Important note

These pages are deprecated (latest changes date from end 2001). More recent information may be found at http://www.coresequence.com/training.php. I leave the old pages here as they are because lots of people are still visiting them (eventhough they aren't spell-checked), and because the basics are still useful.

Redirecting I/O

In this section we will discuss the redirecting of input and output. We'll learn how to use output from one command as input for another command using pipes and we'll present some filtering commands.

Standard Input and Standard Output

Most Unix commands read input, such as a file or another attribute for the command, and write output. By default, input is being given with the keyboard, and output is displayed on your screen. Your keyboard is your "standard input" (stdin) device, and the screen is the "standard output" (stdout) device.

The "<", ">" and "|" operators

Sometimes you will want to put output of a command in a file, this is redirecting of output. It is done either using the ">" (greater-than symbol), or using the "|" (pipe) operator which sends standard output of one command to another command as standard input.

As we saw before, the cat command concatenates files and puts them all together to the standard output. By redirecting this output to a file, this file will be created - or overwritten if it already exists, so take care:


tille:~>cat test1 
some words
tille:~>cat test2
some other words
tille:~>cat test1 test2 > test3
tille:~>cat test3
some words
some other words

Some examples using piping of commands:

  • To mail a textfile to somebody:

    cat file | mail somebody@somewhere.com

  • Find a word in a text, display all lines matching pattern1, exclude lines also matching pattern2 from being displayed:

    grep pattern1 file | grep -v pattern2

  • To display output of a directory listing one page at the time:

    ls -la | more

  • To find an entry in a directory:

    ls -al | grep entry

As we saw before, the cat command concatenates files and puts them all together to the standard output. By redirecting this output to a file, this file will be created (or overwritten if it already exists, so take care!).

In other cases you'd want e.g. a file to be the input for a command that normally wouldn't accept a file as an option. This redirecting of input is done using the "<" (less-than symbol) operator.

Below is an example of sending a file to somebody, using input redirection.


tille:~>mail tille@some.net < to_do

If the user "tille" exists on the system, you don't need to type the full address. If you want to reach somebody on the internet, enter the fully qualified address as an argument to mail.

Below a comprehensible example of combining input and output redirection. The file text.txt is first checked for spelling mistakes, and the output is redirected to an error logfile:

spell < text.txt > error.log

Note

Be careful not to overwrite (important) existing files when redirecting output. Many shells have a built-in feature to protect you from that risk: noclobber. See the manual pages for your shell. In bash, you would want to enter the set -o noclobber to your .bashrc in order to prevent accidental overwriting of files.

The pipe command (|) will feed output of one program as input to another. This is a way of sending mail using redirection of output in stead of redirection of input:


tille~>cat to_do | mail tille

The output of the piped command can then be piped into another command, just as long as these commands would normally read input from the standard input and put output to the standard output.

The > > operator

In stead of overwriting, you can also append text to an existing file.

Example:


tille:~>date > > test3

tille:~>cat test3
some words
some other words
Wed Jun  6 09:01:37 CEST 2001

The date would normally put the last line on the screen, now it is appended to the file test3.

Filters

When a program performs operations on input and writes the result to the standard output, it is called a filter. One of the most common use of filters is to restructure output. We'll discuss a couple of the most important filters below.

grep

grep scans the output line per line, searching for matching patterns. All lines containing the pattern will be printed to standard output. This behaviour can be reversed using the -v option.

Some examples: suppose we want to know which files in a certain directory have been modified in May:


tille:~>ls -la | grep May
drwxr-xr-x  12 root   root     4096 May  1 11:07 ../
-rw-rw-r--   1 tille  tille   48333 May 14 20:11 .abw
-rw-rw-r--   1 tille  tille    1839 May  9 18:31 .acrorc
drwx------   2 tille  tille    4096 May  3 15:11 .gnupg/
-rw-------   1 tille  tille     157 May 21 13:46 .gtkrc
drwx------   2 tille  tille    4096 May 23 21:41 .pinepgp/
-rw-r--r--   1 tille  tille   15405 May  2 20:25 .pinerc.old
drwxr-xr-x   3 tille  tille    4096 May  2  2000 .ssh2/
drwx------   3 tille  tille    4096 May 24  1999 .xauth/
drwxrwxr-x   3 tille  tille    4096 May 30  1999 .xvpics/
drwxrwxr-x   2 tille  tille    4096 May 27 20:02 clo/
drwxrwxr-x   2 tille  tille    4096 May 23 21:42 xantippe/

Note

grep is case sensible, use the -i to make no difference between upper and lower case.

Continuing with the previous example, we'd like to know what files were edited in May of this year:


tille:~>ls -la | grep -i may | grep -v 1999 | grep -v 2000
drwxr-xr-x  12 root      root    4096 May  1 11:07 ../
-rw-rw-r--   1 tille    tille   48333 May 14 20:11 .abw
-rw-rw-r--   1 tille    tille    1839 May  9 18:31 .acrorc
drwx------   2 tille    tille    4096 May  3 15:11 .gnupg/
-rw-------   1 tille    tille     157 May 21 13:46 .gtkrc
drwx------   2 tille    tille    4096 May 23 21:41 .pinepgp/
-rw-r--r--   1 tille    tille   15405 May  2 20:25 .pinerc.old
drwxrwxr-x   2 tille    tille    4096 May 23 21:42 xantippe/

See man grep for more information.

The sort command

sort arranges lines in alphabetical order by default. There's many more sort can do, see the manpage for more information. Let's stick to a practical example for now:


tille:~>cat people-I-like 
Mum
Dad
Boyfriend
Grandma
My boss
Auntie Emmy
tille:~>cat people-I-like | sort
Auntie Emmy
Boyfriend
Dad
Grandma
Mum
My boss

The next example shows how to print files to the standard output, smallest first, biggest last:


tille:~>ls -la | grep -i may | grep -v 1999 | grep -v 2000 | sort +4n
-rw-------   1 tille  tille     157 May 21 13:46 .gtkrc
-rw-rw-r--   1 tille  tille    1839 May  9 18:31 .acrorc
drwx------   2 tille  tille    4096 May  3 15:11 .gnupg/
drwx------   2 tille  tille    4096 May 23 21:41 .pinepgp/
drwxr-xr-x  12 root    root    4096 May  1 11:07 ../
drwxrwxr-x   2 tille  tille    4096 May 23 21:42 xantippe/
drwxrwxr-x   2 tille  tille    4096 May 27 20:02 clo/
-rw-r--r--   1 tille  tille   15405 May  2 20:25 .pinerc.old
-rw-rw-r--   1 tille  tille   48333 May 14 20:11 .abw

You could now pipe this listing to lp or lpr to get it printed, or pipe it to mail to send the list to somebody.