Introduction to Piping in Linux

Introduction to Piping in Linux

Piping is a method of taking the output of one command and using it as the input to the second command.

In this tutorial we’ll cover some of the basics of piping in Linux.

Prerequisites

  • Access to a Linux machine
  • Acting as a non-root sudo user to ensure a secure environment

What is Piping in Linux?

As mentioned above, piping is a form of redirection by which we take the output from one command and pass it on to another command, as input, for further processing.

Piping is denoted with the | symbol.

When working with piping in Linux, it is important to remember that the output flow is unidirectional – from left to right.

It cannot be reversed. The commands are executed simultaneously, one by one, each generating an output for the next command. With the execution of the last command, you get the desired output.

The Piping Syntax

The syntax for piping is as follows:

command1 | command2 | command3

Let’s break down this command to understand it better.

  • We have the first command. When the first command is executed, it generates an output known as the STDOUT or Standard Output.
  • The output of the first command serves as the input to the second command. The input in Linux is known as STDIN or Standard Input.
  • When the second command is executed, it also generates an output.
  • The output of the second command is sent as an input to the third command.

So, piping mainly uses the STDIN and STDOUT, which are the built-in data streams in Linux. There is another data stream, STDERR (Standard Error), that is not used in piping.

Examples of Piping

Let’s get started with some piping examples.

Using Pipes With Two Commands

We will go ahead and execute the ls -l command to list the files in the current directory:

ls -l

Notice that all directories and files are listed.

word image 36

Maybe this is not what we wanted or want to narrow down the results to a specific directory, for example, Documents.

This is where piping plays a key role by generating the desired output with the second command.

In the following example we’ll use run ls -l and send its output to a second command, which is grep. The output from ls -l will be taken as input by grep and filtered by our pattern, which is Documents, in this case.

ls -l | grep Documents

word image 37

Let’s look at another example. This time, we are going to read through a file and display only the selected output.

We have a file named kali.conf in the current folder.

I’m running Ubuntu 20.04 and copied it from sudo cp /etc/speech-dispatcher/modules/kali.conf, for demo purposes.

We will read through the file using the cat command and then display the first 10 lines using the head command. To do this, execute the following command:

cat kali.conf | head

Notice that the output displays only the first 10 lines of the kali.conf file.

word image 38

Using Pipes With Three or More Commands

So far, so good. We have learned the basics of how piping works with two commands.

Let’s now proceed to use three commands.

First, we will find the word “Kali” and then print the number of lines that contain this word. To do this, execute the following command to first find the lines with the word “Kali“:

cat kali.conf | grep "Kali"

Output:

word image 39

If we count these lines manually, we get a total of 11 lines.

We can filter the output to get the total number of lines rather than printing the lines. To do this, we will execute the following command:

cat kali.conf | grep "Kali" | wc -l

Notice that the output displays only the number of lines rather than the lines with the word “Kali”.

word image 40

Let’s look at another example to once again read through the kali.conf file.

This time, we will read through the kali.conf file, list the first 10 lines, using the head command, and then further, we will list the last 5 lines of those 10 lines, using the tail -5 command.

cat kali.conf | head

The output displays the first 10 lines. Please note that the blank lines are also counted. In the output, three blank lines are counted in the total 10 lines.

word image 41

Let’s now move forward to do read through the last 5 lines of these 10 lines:

cat kali.conf | head | tail -5

Only the last five lines of the 10 lines are displayed. So, how did this happen? Let’s take a close look.

The cat command read through the entire file and sent the output to the head command, which filtered the output to the first 10 lines. Also, it is important to note that the head command, unless the number of lines is specified, by default, shows the first 10 lines. This is the reason we didn’t specify a number when running the head command.

word image 42

Let’s now take this one step further. We will now try to find the word “Kali” in the output that we generated with the previous command.

To do this, execute the following command:

cat kali.conf | head | tail -5 | grep "Kali"

Only two lines are displayed. This is because these are the only lines that display the word “Kali”.

word image 43

Next, we don’t want to list the lines, but the total number of lines out of the five lines on which the word “Kali” appears. We will add the wc -l command at the end of the previous command:

cat kali.conf | head | tail -5 | grep "Kali" | wc -l

As you can see, only the number of lines is displayed.

word image 44

Conclusion

Well done. Hopefully, this tutorial helped you understand some of the basics of piping in Linux. If you encountered any issues, please feel free to leave a comment or contact us, and we’ll get back to you as soon as possible.

0 Shares:
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
You May Also Like