When working remotely or even locally, you often may need to obtain content from an external source. To get such content, especially when you don’t have any other options, you will want to use command line tools to get the job done.
In this article, we are going to review some of the most widely used tools for downloading content via the command line.
Read Also: 9 Most Popular GUI Download Managers for Linux
Wget
We are going to start with one of the most popular tools called wget. It’s a network utility that can be used to download content over HTTP, HTTPS and FTP. Wget can be used in both background and foreground, which makes it useful if you need to leave a download running, even when you are logged off.
This tool comes with plenty of options, that allow you to do an authenticated downloads, recursive downloads with level limits, accepts regular expressions for URLs, allows excludes, accepts URL inputs from a file and many others. The options for wget are really a lot and it is highly recommended to review the tool’s help page by simply running.
$ wget -h
Some useful examples of wget command are:
The most basic download example of wget is:
$ wget https://wordpress.org/latest.zip
Example of downloading from URLs listed in a file. First here is the list of our file:
$ cat list.txt https://wordpress.org/latest.zip https://downloads.joomla.org/cms/joomla3/3-8-5/Joomla_3-9-4-Stable-Full_Package.zip https://ftp.drupal.org/files/projects/drupal-8.4.5.zip
Then you can run the download with:
$ wget -i list.txt
To run a download in a background you can use:
$ wget -b https://wordpress.org/latest.zip
If you want to use wget with FTP to download a single file.
$ wget ftp://user:password@host:/path-to-file/file.txt
A more useful example of this would be to use background and recursive mode so you can obtain all files and folders within a directory.
$ wget -br ftp://user:password@ftp-host:/path-for-download/
Wget is preinstalled on many of the modern Linux distros, but if you need to install it, you can use:
$ sudo apt install wget # Debian/Ubuntu like distros # yum install wget # CentOS/RHEL # dnf install wget # Fedora
Curl
A curl is a tool that can be used to transfer data from or to a server. It supports multiple protocols. According to its man page, the following protocols are supported DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, and TFTP.
As you can imagine, you can do a lot with these. As you probably figured it out, curl supports proxies, user authentication, FTP upload/download, file transfer resume and many many more.
Here are some examples of using curl:
Download a file:
$ curl -O https://wordpress.org/latest.zip <./pre> Download a file to output file by your choice:
$ curl -o wordpress.zip https://wordpress.org/latest.zip
To resume an interrupted download you can use:
$ curl -C - O https://wordpress.org/latest.zip
You can check more useful curl examples here: 15 Tips on how to use curl in Linux.
To install curl, you can use:
$ sudo apt install curl # Debian/Ubuntu # yum install curl # CentOS/RHEL # dnf install curl # Fedora
Aria2
Aria is another multi-protocol download tool. Aria supports HTTP/HTTPS, FTP/SFTP BitTorrent and Metalink. Some of the features that make it different compared to others are that it supports the download of files from multiple locations at the same time, magnet links and is fully featured BitTorrent client.
As a BitTorrent client, it supports DHT, PEX, encryption, Magnet URI, web seeding, selective downloads, and local peer discovery.
Feel free to review the Aria2 download manager article for more detailed usage. Below you can see few examples of aria2 basic usage
Here are some examples of using Aria2
:
Download a torrent file:
$ aria2c http://releases.ubuntu.com/18.10/ubuntu-18.10-desktop-amd64.iso.torrent
Download, using URLs listed in a text file:
$ aria2c -i downloadurls.txt
Resume incomplete download:
$ aria2c -c http://releases.ubuntu.com/18.10/ubuntu-18.10-desktop-amd64.iso.torrent
Download from password protected site:
$ aria2c --http-user=xxx --http-password=xxx https://protectedwebsite.com/file
To install Aria2, you can use the following commands:
$ sudo apt install aria2 # Debian/Ubuntu # yum install aria2 # CentOS/RHEL # dnf install aria2 # Fedora
Axel
The fourth download utility in our list is Axel, attempts to improve the downloading process by using multiple connections for one file. It can use multiple download locations for one download. According to the developers, Axel can increase the download speed of your downloads by 60% and it supports protocols: HTTP/HTTPS, FTP, and FTPS.
We have reviewed Axel in a separate article, which you can find here: How to use Axel as download accelerator to speed up FTP and HTTP downloads in Linux.
In the above article, you can check some download time comparisons between wget, HTTP download, and Axel.
Here are some examples of using Axel:
To perform a simple download with Axel, you can use the following command:
$ axel https://wordpress.org/latest.zip
You can set maximum download speed with the corresponding option --max-speed or short option -s. The value is set in bytes per second:
$ axel --max-speed=512000 https://wordpress.org/latest.zip
To save the file with different name, you can use the -o option to specify the file name:
$ axel -o wordpress.zip https://wordpress.org/latest.zip
If you want to install Axel on your Linux system use the appropriate from the commands below:
$ sudo apt install axel # Ubuntu/Debian # yum install epel release && yum install axel # CentOS/RHEL # dnf install axel # Fedora
Conclusion
This was our list of some of the most widely used download utilities in Linux. Which ones do you use? Why do you prefer those? Share your opinion in the comment section below.
