Linux command line tips: wget - gHacks Tech News

Linux command line tips: wget

One of the applications I use most often is the wget tool. You don't realize how useful this tool is until you've used and then needed it but not had it available. Wget is, for all intent and purpose, a non-interactive web downloader that allows the user to, from the command line, download files without requiring the user to even be logged in (hence the "non-interactive" in the title.)

Wget can do a lot of things browsers can not do, which makes it an incredibly valuable tool for users who do a lot of downloading. Wget can even re-connect and finish a download if a connection is dropped. Let's take a look and see how to make best use of this very powerful tool.

Installation

Installation for wget is very simple, as it will be found in the standard repositories. You can install wget from either the command line (such as yum install wget or sudo apt-get install wget) or you can also install wget from the Add/Remove Applications tool (Ubuntu Software Center, PackageKit, Synaptic, etc) by doing the following:

  1. Open the tool.
  2. Search for "wget" (no quotes).
  3. Mark for installation.
  4. Apply.

Usage

Remember, wget is a command line only tool (although there are front-ends available) so you will need to either be working from within a terminal window or logged onto a virtual terminal. Now that you have the right tool open, let's see how wget can help you.

The basic usage is:

wget ADDRESS_TO_FILE

Where ADDRESS_TO_FILE is the actual address (URL or IP) to the file you want to download.

The above will download a single file to your hard drive. Now, what if you have a need to do an recursive downloading? For example there is an entire directory of files you need to download. To do recursive downloading on a remote directory the command would be:

wget -r ADDRESS_TO_DIRECTORY/DIRECTORY

The above command will download a directory hierarchy that matches the hierarchy downloaded from. In other words, within the directory you ran the wget command, you will find (using the above as an example) a newly created directory called ADDRESS_TO_DIRECTORY. Within that directory will be all of the subdirectories you downloaded.

If you don't want to download the hierarchy, just the files, you would add the -nd switch like so:

wget -r -nd ADDRESS_TO_DIRECTORY/DIRECTORY

Now the only thing that will download is the files contained within DIRECTORY.

What if the address or directory you are downloading requires a username and password? You can pass that along with the wget command like so:

wget -r -nd --user=USERNAME --password=PASSWORD ADDRESS_TO_DIRECTORY/DIRECTORY

Where USERNAME is the required username, PASSWORD is the required password, ADDRESS_TO_DIRECTORY is the address and DIRECTORY is the directory containing the files.

Now, if you want to run wget in the background (so you don't have to be logged in) you would issue the command as such:

wget -bqc ADDRESS_TO_FILE/FILE

Where ADDRESS_TO_FILE is the address containing the file and FILE is the file name to be downloaded.

Final thoughts

You will find wget to be one of the more useful tools in your Linux toolkit. And once you start using it you will never look back.

 

Advertisement

We need your help

Advertising revenue is falling fast across the Internet, and independently-run sites like Ghacks are hit hardest by it. The advertising model in its current form is coming to an end, and we have to find other ways to continue operating this site.

We are committed to keeping our content free and independent, which means no paywalls, no sponsored posts, no annoying ad formats or subscription fees.

If you like our content, and would like to help, please consider making a contribution:


Previous Post: «
Next Post: »

Comments

  1. fredy said on March 5, 2011 at 3:43 pm
    Reply

    Gracias, cada día he aprendido algo nuevo en tu sitio.

  2. Dan said on March 8, 2011 at 1:56 am
    Reply

    Heck, I use wget in Windows!

Leave a Reply

Check the box to consent to your data being stored in line with the guidelines set out in our privacy policy

Please note that your comment may not appear immediately after you post it.