Wget can do a lot of things browsers can not do, which makes it an incredibly valuable tool for users who do a lot of downloading. Wget can even re-connect and finish a download if a connection is dropped. Let’s take a look and see how to make best use of this very powerful tool.

Installation

Installation for wget is very simple, as it will be found in the standard repositories. You can install wget from either the command line (such as yum install wget or sudo apt-get install wget) or you can also install wget from the Add/Remove Applications tool (Ubuntu Software Center, PackageKit, Synaptic, etc) by doing the following:

1. Open the tool.
2. Search for “wget” (no quotes).
3. Mark for installation.
4. Apply.

Usage

Remember, wget is a command line only tool (although there are front-ends available) so you will need to either be working from within a terminal window or logged onto a virtual terminal. Now that you have the right tool open, let’s see how wget can help you.

The basic usage is:

wget ADDRESS_TO_FILE

Where ADDRESS_TO_FILE is the actual address (URL or IP) to the file you want to download.

The above will download a single file to your hard drive. Now, what if you have a need to do an recursive downloading? For example there is an entire directory of files you need to download. To do recursive downloading on a remote directory the command would be:

wget -r ADDRESS_TO_DIRECTORY/DIRECTORY

The above command will download a directory hierarchy that matches the hierarchy downloaded from. In other words, within the directory you ran the wget command, you will find (using the above as an example) a newly created directory called ADDRESS_TO_DIRECTORY. Within that directory will be all of the subdirectories you downloaded.

If you don’t want to download the hierarchy, just the files, you would add the -nd switch like so:

wget -r -nd ADDRESS_TO_DIRECTORY/DIRECTORY

Now the only thing that will download is the files contained within DIRECTORY.

What if the address or directory you are downloading requires a username and password? You can pass that along with the wget command like so:

wget -r -nd --user=USERNAME --password=PASSWORD ADDRESS_TO_DIRECTORY/DIRECTORY

Where USERNAME is the required username, PASSWORD is the required password, ADDRESS_TO_DIRECTORY is the address and DIRECTORY is the directory containing the files.

Now, if you want to run wget in the background (so you don’t have to be logged in) you would issue the command as such:

wget -bqc ADDRESS_TO_FILE/FILE

Where ADDRESS_TO_FILE is the address containing the file and FILE is the file name to be downloaded.

Final thoughts

You will find wget to be one of the more useful tools in your Linux toolkit. And once you start using it you will never look back.

For both of these needs there is another tool that works as a pretty impressive front-end for the wget command. That tool – Multiget. Now, before you assume Multiget is a Linux-only tool, it’s not. Multiget can be used in Linux, Windows, OS X, and the BSDs. This article, however, is about using Multiget in Linux (in particular – Ubuntu).

Features

Multiget offers the following features:

• Multi-task/multi-thread/multi-server usage.
• User-friendly interface (resembling Flashget).
• Multiple language support.
• Proxy support.
• Extension ignoring/capture.
• FTP anonymous pass.
• Panel icon.

Installation

Fortunately, for Ubuntu users, Multiget can be found in the Universe repository. All you have to do for installation is:

1. Open up your Add/Remove Software tool.
2. Search for “multiget” (no quotes).
3. Select the results.
4. Apply to install.

That’s it.

Usage

Figure 1

To start up Multiget go to the Internet sub-menu of the Applications menu and you will find the Multiget entry. Open that and the main window will start up (see Figure 1). The first step in downloading a file should be fairly obvious – you hit the the New Task button (denoted by the “+” symbol). Simple.

When you hit the New Task button  a new window will appear that asks for certain information. At least in the Linux release of Multiget, you might find a portion of this window that might stump you. As you can see (in Figure 2) in the Basic Info section you have four bits of information to add. The MAIN URL section is obvious – you copy and paste a URl into the this section. But wait a minute – you will find, if copying your URL from your default web browser, that as soon as you right-click a URL, and select Copy Link Location that URL will appear in the MAIN URL section and then automatically moved down to the Mirrors section. This is actually the only means of getting an address into the Mirrors section.

Figure 2

With that in mind, let’s use a few mirrors to quickly download the ISO image of the latest release of Ubuntu. To do this go to the Ubuntu Karmic page page. The first thing you will do is go to the main download page, right click the download link, and click Copy Link Location. Now go back to the main download page and select a mirror and do the same thing. You can add as many mirrors as you like – understanding the more mirrors you add, the faster the download will happen.

Once you have all of your URLs added (make sure you still have a URL in the MAIN URL section) configure the rest of the window (you should configure the SaveTo section at least) and then click OK.

As your file is downloading click on the Progress section in the bottom and you will see how the pieces of the download are coming together. As you can see (in Figure 3) I have five threads running to download the Ubuntu 9.10 ISO image. You can also see the overall progress bar as the green bar starting right under the icon toolbar.

Figure 3

During an active session you can increase or decrease the amount of threads given to a download. This, of course, can speed up your download. Naturally if you give too many threads to a download you might see your overall performance of either your machine or your general network connection degrade. At this point you can decrease the number of threads given to a download. To increase or decrease the number of threads go to the Task menu and select the appropriate entry.

During the download you can click on the Running section to see the information of your download. You can also pause your download by clicking the Pause button.

Final thoughts

Multiget is an outstanding tool to download large files using muliple mirrors and threads. As well it is just an overall outstanding front end for the wget command.