Linux curl Command with Practical Examples

Introduction

In this lab, we will explore the powerful curl command, a versatile tool for transferring data over various protocols, including HTTP, FTP, and more. We will start by introducing the curl command and checking its version, then move on to fetching web page content and downloading files using curl. This lab aims to provide practical examples and demonstrate the capabilities of the curl command, which is a essential tool for networking and communication tasks.

The lab covers the following steps:

Introduction to curl Command
Fetching Web Page Content with curl
Downloading Files Using curl

Linux Commands Cheat Sheet

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL linux(("`Linux`")) -.-> linux/BasicFileOperationsGroup(["`Basic File Operations`"]) linux(("`Linux`")) -.-> linux/PackagesandSoftwaresGroup(["`Packages and Softwares`"]) linux(("`Linux`")) -.-> linux/RemoteAccessandNetworkingGroup(["`Remote Access and Networking`"]) linux/BasicFileOperationsGroup -.-> linux/cat("`File Concatenating`") linux/PackagesandSoftwaresGroup -.-> linux/curl("`URL Data Transferring`") linux/PackagesandSoftwaresGroup -.-> linux/wget("`Non-interactive Downloading`") linux/RemoteAccessandNetworkingGroup -.-> linux/ftp("`File Transferring`") subgraph Lab Skills linux/cat -.-> lab-422625{{"`Linux curl Command with Practical Examples`"}} linux/curl -.-> lab-422625{{"`Linux curl Command with Practical Examples`"}} linux/wget -.-> lab-422625{{"`Linux curl Command with Practical Examples`"}} linux/ftp -.-> lab-422625{{"`Linux curl Command with Practical Examples`"}} end

Introduction to curl Command

In this step, we will explore the curl command, a powerful tool for transferring data using various protocols, including HTTP, FTP, and more. Curl is a versatile command-line tool that allows you to interact with web servers, download files, and perform various network-related tasks.

First, let's check the version of curl installed in our Ubuntu 22.04 Docker container:

curl --version

Example output:

curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.0 (+libidn2-2.3.2) libstdc++/9.4.0 libssh/0.9.6/openssl/zlib nghttp2/1.47.0 librtmp/2.3
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

As you can see, the output shows the version of curl, the protocols it supports, and the various features it provides.

Next, let's try using curl to fetch the content of a web page. We'll use the example of fetching the homepage of the curl project:

curl https://curl.se

Example output:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>curl - transfer data with URL</title>
...

The output shows the HTML content of the curl project's homepage, which curl has fetched and displayed in the terminal.

In the next step, we'll explore more advanced use cases of the curl command, such as downloading files and interacting with different protocols.

Fetching Web Page Content with curl

In this step, we will learn how to use the curl command to fetch the content of web pages.

First, let's try fetching the homepage of the curl project again, but this time, we'll use the -o option to save the output to a file:

curl -o curl_homepage.html https://curl.se

Example output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 31748  100 31748    0     0  93644      0 --:--:-- --:--:-- --:--:-- 93644

The -o option specifies the output file name, in this case, curl_homepage.html. The output shows the progress of the download, including the total size of the file, the download speed, and the time taken to complete the download.

You can also use the -s (silent) option to suppress the progress output and only show the fetched content:

curl -s https://curl.se

Example output:

<!doctype html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>curl - transfer data with URL</title>
    ...
  </head>
</html>

The -s option makes the output more concise, as it only shows the fetched content without the progress information.

Another useful option is -I (or --head), which allows you to fetch only the headers of a web page, without the actual content:

curl -I https://curl.se

Example output:

HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 19 Apr 2023 06:34:26 GMT
Content-Type: text/html
Content-Length: 31748
Last-Modified: Fri, 07 Apr 2023 14:37:54 GMT
Connection: close
ETag: "64306f62-7b0c"
Accept-Ranges: bytes

The output shows the HTTP headers, which can be useful for understanding the server's response and troubleshooting any issues.

In the next step, we'll explore how to use curl to download files from the web.

Downloading Files Using curl

In this step, we will learn how to use the curl command to download files from the web.

Let's start by downloading a file from the curl project's website. We'll use the -O option to save the file with the same name as the remote file:

curl -O https://curl.se/download/curl-7.81.0.tar.gz

Example output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 3.8M  100 3.8M    0     0  6901k      0 --:--:-- --:--:-- --:--:-- 6901k

The -O option tells curl to save the file with the same name as the remote file, in this case, curl-7.81.0.tar.gz.

You can also use the -o option to specify a different file name:

curl -o curl_source.tar.gz https://curl.se/download/curl-7.81.0.tar.gz

Example output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 3.8M  100 3.8M    0     0  6901k      0 --:--:-- --:--:-- --:--:-- 6901k

In this case, the downloaded file will be named curl_source.tar.gz.

You can also use curl to download files from FTP servers. Here's an example of downloading a file from an FTP server:

curl -O ftp://ftp.example.com/file.zip

Example output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12.3M  100 12.3M    0     0  6901k      0 --:--:-- --:--:-- --:--:-- 6901k

In this case, the file file.zip will be downloaded from the FTP server ftp.example.com.

Remember to replace the URLs and file names with the ones you want to download.

In the next step, we'll review what we've learned about the curl command and its practical applications.

Summary

In this lab, we learned about the powerful curl command, which is a versatile tool for transferring data using various protocols. We started by exploring the version and capabilities of the curl command installed on our Ubuntu 22.04 Docker container. Then, we used curl to fetch the content of the curl project's homepage, both by displaying the output in the terminal and by saving it to a file. Finally, we discussed how to use curl to download files, which is a common use case for this command.

Linux Commands Cheat Sheet

Linux curl Command with Practical Examples

Introduction

Skills Graph

Introduction to curl Command

Fetching Web Page Content with curl

Downloading Files Using curl

Summary

Other Linux Tutorials you may like