In this article we will learn how to make HTTP GET requests using cURL, covering the common methods of handling different options and situations.
Table of Contents
- What is cURL
- cURL Syntax
- Perform a Simple GET Request
- Perform a GET Request with Data
- Perform a GET Request with Extra Headers
- Perform a GET Request with Extra Information
- Perform a GET Request to Follow Redirections
- Perform a GET Request and Save the Response
- Perform a GET Request for A Specific Range of Bytes
- Perform an HTTP/2 GET request
What is cURL
curl is one of the most useful command line utilities known to Linux users.
curl is a free and open source tool that allow the transfer of data between a client and a server under the support of multiple protocols (HTTP, FTP, SMTP, POP…), in addition of making various types of requests.
This tool is great for testing API functionalities, downloading files, checking response headers and making HTTP requests. Though many developers are leaning towards other programs such as postman to test APIs, but
curl still a strong option in this domain.
One of the most simple form of used HTTP request among developers, is the HTTP GET method with
In this article we will keep things fairly simple and cover only the GET method using the
curl command tool.
Check the syntax mentioned below of the
curl GET command.
curl [option] [URL]
curl -X GET [option] [URL]
Perform a Simple GET Request
One of the basic HTTP requests that we can perform with the help of curl, is the GET request method minus specifying a body to it. By default, the curl tool sends GET requests without adding the
-X command line argument – (that specifies the request method) – ; this option is used to communicate with the remote server. The command mentioned below is to send a GET request and print out the HTTP response of the URL in question.
The following command is similar to the above given request. We just added the
-X option to it.
curl -X GET localhost:8080/home
If we added the
-I argument, we basically telling the curl tool to only return the HTTP headers of an endpoint without the response body. Perform the below mentioned command, and compare it to the previous command for better understanding.
curl -I localhost:8080/home
Note: that we are using our internal web application for these examples, you can choose any valid web site.
Perform a GET Request with Data
Whenever dealing with HTTP requests, there is a high chance that you will be tweaking your curl command to send data with your request. We can specify these data/ parameters with the
--data command line option. You should also keep in mind that whenever you add the
--data option without adding the
-X option, curl will send a POST request by default; but for our purpose, we are only sending GET requests. So, to achieve this we will use the
-G option with our request. Check the following command.
curl -G -data "usertype=vip&country=usa" localhost:8080/home/config
Note: Sometimes we need to use query parameters with our GET requests; for example, to filter out our search.
-The above given command is equivalent to
We can also add multiple
--data option for each parameter. Check the example below.
curl -G --data "usertype = vip" --data "country = usa" localhost:8080/ home/config
Using cURL Basic Authentication
In case that an endpoint needs a basic authentication, we can simply use the
--user option to pass the username and password. Use the following command to achieve this.
curl -u username:password localhost:8080/home/settings
Perform a GET Request with Extra Headers
The following curl examples demonstrating the ability to add an extra HTTP request header to your requests. To do so, we will be adding the
--header curl option to our request.
curl -H "Accept: application/json" localhost:8080/home/config
Note: The above given request will get JSON data using curl from our web app .
There is also the possibility to add multiple
--header option for each parameter. Check the example below.
curl -H "Accept: application/xml" -H "Content-Type: application/xml" localhost:8080/home/config
Perform a GET Request with Extra Information
--verbose option will make curl output all the details of our request. This simply will generate more information during the process of sending the request and getting its response.
curl --versose -H "Accept: application/json" localhost:8080/home/config
Perform a GET Request to Follow Redirections
Most of the times while using GET requests, you are bound to encounter a
301) redirect response, but in almost all cases we are interested in the final endpoint. So, to make the curl tool follow redirections, you will have to use the
--location command line argument.
curl -L localhost:8080/home/.htm
To override the default max number of redirections, you can simply set a value to the
--max-redirs option, just like so:
curl -L --max-redirs 100 localhost:8080/home/.htm
Perform a GET Request and Save the Response
curl command line option that might come in handy, is
--remote-name. Both options will save the response body of your
curl request. The main difference is that the
-o<code>,</code> --output option will save the file as specified in the
curl request. Check the example for better understanding:
curl -output MyScript.js -L --max-redirs 100 localhost:8080/home/checktime.js
curlcommand will save the response in a file called:
In contrast, the
--remote-name option will save the file as specified by the server. Let’s run the following command:
curl --remote-name -L --max-redirs 100 localhost:8080/home/checktime.js
Note: the above curl command will save the response in a file called:
Downloading a File
As you might have guessed, the
--remote-name curl command options are not just for storing web site pages. Using these
curl command options, we can easily transfer entire files, plus tweaking the process to fit different needs. Let’s download a file using a simple GET request.
curl -O localhost:8080/bucket/Mission_Impossible.mkv
Downloading Multiple Files
curl command options will result in downloading multiple files simultaneously. Let’s check the following example.
curl -O localhost:8080/bucket/Mission_Impossible.mkv -O localhost:8080/bucket/LordOfTheRings.mkv -O localhost:8080/bucket/TheGodFather.mkv -O localhost:8080/bucket/Inception.mkv
Note: we can also use the
--output command option to specify a new file name for each downloaded file. Check the below given example.
curl -o mi.mkv localhost:8080/bucket/Mission_Impossible.mkv -o lor.mkv localhost:8080/bucket/LordOfTheRings.mkv -o gf.mkv localhost:8080/bucket/TheGodFather.mkv -o dreams.mkv localhost:8080/bucket/Inception.mkv
Resuming Downloading Files
We all have been in situations where we are in the process of downloading a large file, then suddenly the process gets interrupted or simply purposely stopped by us. Naturally, we don’t want to lose the previous progress when we decide to resume the download. The
curl command offers us the
--continue-at option to easily resume a broken download where it was left off.
curl -C -o kali.iso localhost:8080/bucket/kali3-cloud-amd64.iso
Checking the Download Progress
As we have seen in previous examples, the curl command doesn’t show us the progress of the download. To overcome this small inconvenience, we just have to add the
--progress-bar command line option just like so:
curl -# -o kali.iso localhost:8080/bucket/kali3-cloud-amd64.iso
Note: The above given request will show the built in progress meter in the terminal screen to demonstrate how the transfer is progressing .
In case you want to suppress the progress meter to make curl don’t show any information while downloading a file, just use the
--silent command flag.
Specifying the Download Speed
--limit-rate parameter, we can limit the transfer rate of our download process. By default,
curl measures the speed in bytes per second, but we can override this default setting by explicitly stating the measurement unit:
G (gigabytes). Follow our next curl request for better understanding.
curl --limit-rate 1M -# -o kali.iso localhost:8080/bucket/kali3-cloud-amd64.iso
Note: We are setting the maximum transfer rate of : one megabytes per second (
Perform a GET Request for A Specific Range of Bytes
The byte-range request method is very useful when requesting a specific section of a large media file. Also it benefits the user for conserving time and bandwidth. For example, a user wanted to download only a small number of pages instead of a very large
--range curl command flag just as so:
curl -r 0-999 --limit-rate 1M -# -o lm_part1.txt localhost:8080/bucket/listofmachines.txt
Note: The above command will get the first
1000 bytes from
For our next example we are going to get the rest of the file starting from where we left off.
curl -r 1000- --limit-rate 1M -# -o lm_partN.txt localhost:8080/bucket/listofmachines.txt
How about we use the
--range option to get a
500 bytes starting from an index? Check the following request:
curl -r 0-499, 1000-1499 --limit-rate 1M -# -o lm_part2.txt localhost:8080/bucket/listofmachines.txt
Note: The above command will get
500 bytes from index 1000.
With byte-range requests, we have the ability to pick up the download at the point when it was broken.
Perform an HTTP/2 GET request
We can also use the
--http2 command line parameter if a server supports the fairly new protocol.
curl -http2 localhost:8080/home/config
Note: if the server support the HTTP/2 protocol, you will get
HTTP/2.0 200 OK in the response body instead of
HTTP/1.1 200 OK
In this how to article, we’ve walked you through the different ways of making GET requests with curl. We hope they may come in handy for you.