How To Make a GET Request With cURL

In this article we will learn how to make HTTP GET requests using cURL, covering the common methods of handling different options and situations.

What is cURL

curl is one of the most useful command line utilities known to Linux users. curl is a free and open source tool that allow the transfer of data between a client and a server under the support of multiple protocols (HTTP, FTP, SMTP, POP…), in addition of making various types of requests.

This tool is great for testing API functionalities, downloading files, checking response headers and making HTTP requests. Though many developers are leaning towards other programs such as postman to test APIs, but curl still a strong option in this domain.

One of the most simple form of used HTTP request among developers, is the HTTP GET method with curl.

In this article we will keep things fairly simple and cover only the GET method using the curl command tool.

cURL Syntax

Check the syntax mentioned below of the curl GET command.

curl [option] [URL]

curl -X GET [option] [URL]

Perform a Simple GET Request

One of the basic HTTP requests that we can perform with the help of curl, is the GET request method minus specifying a body to it. By default, the curl tool sends GET requests without adding the -X command line argument – (that specifies the request method) – ; this option is used to communicate with the remote server. The command mentioned below is to send a GET request and print out the HTTP response of the URL in question.

curl localhost:8080/home

The following command is similar to the above given request. We just added the -X option to it.

curl -X GET localhost:8080/home

If we added the -I argument, we basically telling the curl tool to only return the HTTP headers of an endpoint without the response body. Perform the below mentioned command, and compare it to the previous command for better understanding.

curl -I localhost:8080/home

Note: that we are using our internal web application for these examples, you can choose any valid web site.

Perform a GET Request with Data

Whenever dealing with HTTP requests, there is a high chance that you will be tweaking your curl command to send data with your request. We can specify these data/ parameters with the -d, --data command line option. You should also keep in mind that whenever you add the -d, --data option without adding the -X option, curl will send a POST request by default; but for our purpose, we are only sending GET requests. So, to achieve this we will use the -G option with our request. Check the following command.

curl -G -data "usertype=vip&country=usa" localhost:8080/home/config

Note: Sometimes we need to use query parameters with our GET requests; for example, to filter out our search.

-The above given command is equivalent to localhost:8080/home/config? usertype=vip&country=usa

We can also add multiple -d, --data option for each parameter. Check the example below.

curl -G --data "usertype = vip" --data "country = usa" localhost:8080/ home/config

Using cURL Basic Authentication

In case that an endpoint needs a basic authentication, we can simply use the -u, --user option to pass the username and password. Use the following command to achieve this.

curl -u username:password localhost:8080/home/settings

Perform a GET Request with Extra Headers

The following curl examples demonstrating the ability to add an extra HTTP request header to your requests. To do so, we will be adding the -H, --header curl option to our request.

curl -H "Accept: application/json" localhost:8080/home/config

Note: The above given request will get JSON data using curl from our web app .

There is also the possibility to add multiple -H, --header option for each parameter. Check the example below.

curl -H "Accept: application/xml" -H "Content-Type: application/xml" localhost:8080/home/config

Perform a GET Request with Extra Information

The -v,--verbose option will make curl output all the details of our request. This simply will generate more information during the process of sending the request and getting its response.

curl --versose -H "Accept: application/json" localhost:8080/home/config

Perform a GET Request to Follow Redirections

Most of the times while using GET requests, you are bound to encounter a 3xx (301) redirect response, but in almost all cases we are interested in the final endpoint. So, to make the curl tool follow redirections, you will have to use the -L, --location command line argument.

curl -L localhost:8080/home/.htm

To override the default max number of redirections, you can simply set a value to the --max-redirs option, just like so:

curl -L --max-redirs 100 localhost:8080/home/.htm

Perform a GET Request and Save the Response

Another curl command line option that might come in handy, is -o, --output / -O, --remote-name. Both options will save the response body of your curl request. The main difference is that the -o<code>,</code> --output option will save the file as specified in the curl request. Check the example for better understanding:

curl -output MyScript.js -L --max-redirs 100 localhost:8080/home/checktime.js

In contrast, the -O, --remote-name option will save the file as specified by the server. Let’s run the following command:

curl --remote-name -L --max-redirs 100 localhost:8080/home/checktime.js

Note: the above curl command will save the response in a file called: checktime.js

Downloading a File

As you might have guessed, the -o, --output / -O, --remote-name curl command options are not just for storing web site pages. Using these curl command options, we can easily transfer entire files, plus tweaking the process to fit different needs. Let’s download a file using a simple GET request.

curl -O localhost:8080/bucket/Mission_Impossible.mkv

Downloading Multiple Files

Adding multiple -o, --output / -O, --remote-name curl command options will result in downloading multiple files simultaneously. Let’s check the following example.

curl -O localhost:8080/bucket/Mission_Impossible.mkv -O localhost:8080/bucket/LordOfTheRings.mkv -O localhost:8080/bucket/TheGodFather.mkv -O localhost:8080/bucket/Inception.mkv

Note: we can also use the -o, --output command option to specify a new file name for each downloaded file. Check the below given example.

curl -o mi.mkv localhost:8080/bucket/Mission_Impossible.mkv -o lor.mkv localhost:8080/bucket/LordOfTheRings.mkv -o gf.mkv localhost:8080/bucket/TheGodFather.mkv -o dreams.mkv localhost:8080/bucket/Inception.mkv

Resuming Downloading Files

We all have been in situations where we are in the process of downloading a large file, then suddenly the process gets interrupted or simply purposely stopped by us. Naturally, we don’t want to lose the previous progress when we decide to resume the download. The curl command offers us the -C, --continue-at option to easily resume a broken download where it was left off.

curl -C -o kali.iso localhost:8080/bucket/kali3-cloud-amd64.iso

Checking the Download Progress

As we have seen in previous examples, the curl command doesn’t show us the progress of the download. To overcome this small inconvenience, we just have to add the -#, --progress-bar command line option just like so:

curl -# -o kali.iso localhost:8080/bucket/kali3-cloud-amd64.iso

Note: The above given request will show the built in progress meter in the terminal screen to demonstrate how the transfer is progressing .

In case you want to suppress the progress meter to make curl don’t show any information while downloading a file, just use the -s, --silent command flag.

Specifying the Download Speed

With the --limit-rate parameter, we can limit the transfer rate of our download process. By default, curl measures the speed in bytes per second, but we can override this default setting by explicitly stating the measurement unit: k, K (kilobytes) -m, M (megabytes) -g, G (gigabytes). Follow our next curl request for better understanding.

curl --limit-rate 1M -# -o kali.iso localhost:8080/bucket/kali3-cloud-amd64.iso

Note: We are setting the maximum transfer rate of : one megabytes per second (1 M/s)

Perform a GET Request for A Specific Range of Bytes

The byte-range request method is very useful when requesting a specific section of a large media file. Also it benefits the user for conserving time and bandwidth. For example, a user wanted to download only a small number of pages instead of a very large .pdf document. We can achieve this by using the -r, --range curl command flag just as so:

curl -r 0-999 --limit-rate 1M -# -o lm_part1.txt localhost:8080/bucket/listofmachines.txt

Note: The above command will get the first 1000 bytes from listofmachines.txt

For our next example we are going to get the rest of the file starting from where we left off.

curl -r 1000- --limit-rate 1M -# -o lm_partN.txt localhost:8080/bucket/listofmachines.txt

How about we use the -r, --range option to get a 500 bytes starting from an index? Check the following request:

curl -r 0-499, 1000-1499 --limit-rate 1M -# -o lm_part2.txt localhost:8080/bucket/listofmachines.txt

Note: The above command will get 500 bytes from index 1000.

With byte-range requests, we have the ability to pick up the download at the point when it was broken.

Perform an HTTP/2 GET request

We can also use the --http2 command line parameter if a server supports the fairly new protocol.

curl -http2 localhost:8080/home/config

Note: if the server support the HTTP/2 protocol, you will get HTTP/2.0 200 OK in the response body instead of HTTP/1.1 200 OK

Conclusion

In this how to article, we’ve walked you through the different ways of making GET requests with curl. We hope they may come in handy for you.