Introduction to Piping in Linux

Piping is a method of taking the output of one command and using it as the input to the second command.

In this tutorial we’ll cover some of the basics of piping in Linux.

Prerequisites

• Access to a Linux machine
• Acting as a non-root sudo user to ensure a secure environment

What is Piping in Linux?

As mentioned above, piping is a form of redirection by which we take the output from one command and pass it on to another command, as input, for further processing.

Piping is denoted with the | symbol.

When working with piping in Linux, it is important to remember that the output flow is unidirectional – from left to right.

It cannot be reversed. The commands are executed simultaneously, one by one, each generating an output for the next command. With the execution of the last command, you get the desired output.

The Piping Syntax

The syntax for piping is as follows:

command1 | command2 | command3

Let’s break down this command to understand it better.

• We have the first command. When the first command is executed, it generates an output known as the STDOUT or Standard Output.
• The output of the first command serves as the input to the second command. The input in Linux is known as STDIN or Standard Input.
• When the second command is executed, it also generates an output.
• The output of the second command is sent as an input to the third command.

So, piping mainly uses the STDIN and STDOUT, which are the built-in data streams in Linux. There is another data stream, STDERR (Standard Error), that is not used in piping.

[powerkit_alert type=”info” dismissible=”false” multiline=”false”]Note: We have mentioned three commands only for the sake of demonstration. You can use as many commands as possible.[/powerkit_alert]

Examples of Piping

Let’s get started with some piping examples.

Using Pipes With Two Commands

We will go ahead and execute the ls -l command to list the files in the current directory:

ls -l

Notice that all directories and files are listed.

Maybe this is not what we wanted or want to narrow down the results to a specific directory, for example, Documents.

This is where piping plays a key role by generating the desired output with the second command.

In the following example we’ll use run ls -l and send its output to a second command, which is grep. The output from ls -l will be taken as input by grep and filtered by our pattern, which is Documents, in this case.

ls -l | grep Documents

Let’s look at another example. This time, we are going to read through a file and display only the selected output.

We have a file named kali.conf in the current folder.

We will read through the file using the cat command and then display the first 10 lines using the head command. To do this, execute the following command:

cat kali.conf | head

Notice that the output displays only the first 10 lines of the kali.conf file.

Using Pipes With Three or More Commands

So far, so good. We have learned the basics of how piping works with two commands.

Let’s now proceed to use three commands.

First, we will find the word “Kali” and then print the number of lines that contain this word. To do this, execute the following command to first find the lines with the word “Kali“:

cat kali.conf | grep "Kali"

Output:

If we count these lines manually, we get a total of 11 lines.

We can filter the output to get the total number of lines rather than printing the lines. To do this, we will execute the following command:

cat kali.conf | grep "Kali" | wc -l

Notice that the output displays only the number of lines rather than the lines with the word “Kali”.

Let’s look at another example to once again read through the kali.conf file.

This time, we will read through the kali.conf file, list the first 10 lines, using the head command, and then further, we will list the last 5 lines of those 10 lines, using the tail -5 command.

cat kali.conf | head

The output displays the first 10 lines. Please note that the blank lines are also counted. In the output, three blank lines are counted in the total 10 lines.

[powerkit_alert type=”info” dismissible=”false” multiline=”false”]
Note: I realize that the head command by itself, when running head kali.conf, will result in the same output as above, without needing to use the cat command. This is just a way I thought would be convenient to demonstrate piping with multiple commands.
[/powerkit_alert]

Let’s now move forward to do read through the last 5 lines of these 10 lines:

cat kali.conf | head | tail -5

Only the last five lines of the 10 lines are displayed. So, how did this happen? Let’s take a close look.

The cat command read through the entire file and sent the output to the head command, which filtered the output to the first 10 lines. Also, it is important to note that the head command, unless the number of lines is specified, by default, shows the first 10 lines. This is the reason we didn’t specify a number when running the head command.

Let’s now take this one step further. We will now try to find the word “Kali” in the output that we generated with the previous command.

To do this, execute the following command:

cat kali.conf | head | tail -5 | grep "Kali"

Only two lines are displayed. This is because these are the only lines that display the word “Kali”.

Next, we don’t want to list the lines, but the total number of lines out of the five lines on which the word “Kali” appears. We will add the wc -l command at the end of the previous command:

cat kali.conf | head | tail -5 | grep "Kali" | wc -l

As you can see, only the number of lines is displayed.

Conclusion

Well done. Hopefully, this tutorial helped you understand some of the basics of piping in Linux. If you encountered any issues, please feel free to leave a comment or contact us, and we’ll get back to you as soon as possible.