curl is a powerful, versatile command-line tool that allows you to transfer data to or from a server using a variety of protocols such as HTTP, HTTPS, FTP, and more. Curl, short for “Client URL,” is indispensable for web requests. It enables downloading files, sending data via POST requests, and interacting with RESTful APIs directly from the command line. Its flexibility, coupled with the ability to provide detailed outputs, makes it a favorite among Linux users, often utilized in scripts for automation.
In this tutorial, we will explore the curl command, which, among other functionalities, lets you download data from the web. Note that the examples in this article are tested on Ubuntu 24.04.
Understanding the Linux curl Command
The curl command allows data download and upload through the Linux command line interface. Here is its syntax:
curl [options] [URL...]
The man page describes this command as follows:
curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user inter? action. curl offers a busload of useful tricks like proxy support, user authen? tication, FTP upload, HTTP post, SSL connections, cookies, file trans? fer resume, Metalink, and more. As you will see below, the number of features will make your head spin! curl is powered by libcurl for all transfer-related features. See libcurl(3) for details.
Below are several Q&A-style examples to better illustrate the usage of curl:
Q1. How Does the curl Command Work?
The basic usage is straightforward—pass the URL to the curl command and redirect the output to a file. For example:
curl http://releases.ubuntu.com/24.04/ubuntu-24.04-desktop-amd64.iso.torrent > test.torrent
The -o option can also be used:
-o, --output <file> Write output to <file> instead of stdout.
In the given example, the downloaded data is saved in the ‘test.torrent’ file, while the following progress is displayed on the command line:
As per the man page, the progress meter indicates the amount of data transferred, transfer speeds, and estimated time left, etc., by default during operations. To avoid mixing progress meter and response data, redirect output to a file when involving operations like HTTP POST or PUT requests.
Q2. How to Retain the Same Download File Name?
You can force curl to use the remote file name locally using the -O option:
curl -O http://releases.ubuntu.com/18.04/ubuntu-18.04-desktop-amd64.iso.torrent
This command saves the file under the name ‘ubuntu-18.04-desktop-amd64.iso.torrent’ on your system.
Q3. How to Download Multiple Files Using curl?
To download multiple files, use the -O option for each file URL:
curl -O [URL1] -O [URL2] -O [URL3] ...
An example would be:
curl -O http://releases.ubuntu.com/18.04/ubuntu-18.04-desktop-amd64.iso.torrent -O http://releases.ubuntu.com/18.04/ubuntu-18.04-live-server-amd64.iso.torrent
The progress for both downloads will be displayed consecutively.
Q4. How to Handle URL Redirects?
If a URL redirects to another and causes a “Moved” or “Moved Permanently” error, use the -L option to follow the redirect:
curl -L http://www.oneplus.com
Q5. How to Resume an Interrupted Download?
In case of an interrupted download, resume by using the -C option:
curl -C - -O http://releases.ubuntu.com/18.04/ubuntu-18.04-desktop-amd64.iso
The command resumes the download from where it was interrupted, as illustrated below:
Conclusion
The curl command is a highly effective tool for command-line data transfers. This article provides a glimpse into its capabilities, which extend far beyond what is covered here. For a deeper exploration, consider visiting curl’s manual page.
Frequently Asked Questions (FAQ)
What protocols does curl support?
Curl supports various protocols, including but not limited to HTTP, HTTPS, FTP, FTPS, SCP, SFTP, LDAP, and many more.
Can curl be used for sending POST requests?
Yes, curl allows sending data via POST and is commonly used to interact with RESTful APIs.
Is curl pre-installed on all Linux systems?
While curl is included in many Linux distributions, it is not always pre-installed. However, it can easily be installed via package managers like apt or yum.
Does curl support downloading files over a proxy?
Yes, curl can be configured to download files over an HTTP or SOCKS proxy.
How can I automate tasks using curl?
Curl is often used in shell scripts because it allows automation of data transfer tasks without user interaction.