Hax - Web - Wget - Arguments

"Making IT happen on the
Information Superhighway"

 

2600 | Antionline | Astalavista | Blacklisted 411 | Crypto-Gram | Defcon | Geeks Are Sexy | IT Conversations | Shoutwire | Slashdot | Tech Tales | The Daily WTF |

Forums | Linux | News |Telephone | Windows |

 

Authentication  
  • --http-user=user --http-passwd=passwd
  • wget ftp://user:password@domain.com/.emacs
  •  

Comment Line  
  • Comments follow the "#" sign, for instance
    # Add in a delay is -w 5
  •  

Concatenate group of files into a single document  
  • Two requirements (-O argument, plus forcefiletype and name)
  • -O
  • --output-document=localtest.htm
  • wget -i w:\if\wget\urls\localtest.txt --progress=bar -w 5 -nc -P w:\blogs\localtest -O --output-document=mavericks.htm
  •  

Continue downloading where you left off, in case of a failure  
  • -c
  •  

Delay, to avoid hammering  
  • -w 5
  •  

Do not overwrite any files we already have, instead rename with file(1), etc  
  • -nc
  •  

Download URLs from a file  
  • wget -i c:\file.txt
  •  

 

Force Filetype  
  • Force filetype to HTML and store in specified directory
    -E
  • --output-document=localtest.avi
    For some reason, content is delivered to location of BAT file
  • --output-document=localtest.htm
    For some reason, content is delivered to location of BAT file
  • --output-document=localtest.html
    For some reason, content is delivered to location of BAT file
  • --output-document=localtest.txt
    For some reason, content is delivered to location of BAT file
  • This format not working right
     -force-html

 

Grab all files needed for proper viewing  
  • - p
  • The "p" argument instructs wget to grab ALL files needed for the page to properly display, for instance, graphics, "CSS" files or "Javascript" files, etc.
  •  

Ignore Robots  
  • -erobots=off
  •  

Logging Results  
  • -o
  • wget -o wget.log
  •  

 

Masquerade as Web Browser  
  • -U "Mozilla/5.0"
  •  

 

Modify content, for local viewing  
  • -k
  •  

 

Progress Bar  
  • --progress=bar
  •  

 

Put all files into same folder  
  • - nd
  • The "nd" argument tells wget to put every file into the same directory (no folders)
  •  

 

Specify the directory name, where files should go  
  • -P c:\mydir
  •  

Time Stamp  
  • -N
  •  

Times to attempt the download  
  • -t 20
  •