Lead Image © Dmitriy Sladkov, 123RF.com

Shell practice: Introduction to the sed stream editor

Quick Edit

Article from ADMIN 28/2015

By Harald Zisler

With sed, you can edit text data without an interactive user interface, using pipes or input redirection. Sed lets you execute extensive editing commands on a single line.

Sed (Stream EDitor) [1] automates repetitive operations on a text file and is especially effective when used in a shell script and with regular expressions (regex). In this article, I send program output to the screen. If you want to participate and practice, simply use the text files provided [2].

Sed Commands

The program calls up and accepts commands from virtually anywhere. You can pass in commands directly or read them in from a file. The data can be piped, redirected, or input from a text file. The output can be sent to the screen (usually stdout), through a pipe to the next command, or redirected to a destination file. (See the "Sed Call Options" box.) To resolve shell variables, you sometimes need to substitute the " for the ' character.

Sed Call Options

Sed simply reads the text file and returns the results through stdout:

sed [COMMAND] [TEXTFILE]

Same with input redirection:

sed [COMMAND] < [TEXTFILE]

Inclusion of sed in one or more pipes:

[PROGRAM1] | sed [COMMAND] | ......

Commands stored in a separate file and read in:

... sed -f [SCRIPT] .....

Output of sed redirected to a text file, but omitting error messages:

... sed [COMMAND] > [TARGETFILE]

The same, but including error messages in the target file:

... sed [COMMAND] > [TARGETFILE] 2>&1

Syntax

The basic syntax structure is shown in Figure 1. Everywhere an editing command should be used, addressing is required. You can provide many addresses as long as doing so doesn't affect clarity. If you want to change "everything except," you can negate addressing with the ! character.

Figure 1: Sed syntax structure.

To put multiple commands on one line use the -e option:

sed -e 'command1' -e 'command2' ... -e 'commandN' ....

Or, you can add these commands from a script file.

Script Files

A script file should have a single line for each statement. For example:

s/Gans//
s/jo/Jo/g

The first line removes the word Gans – rather, it substitutes nothing for the word, but only for the first instance of the search string. The second line substitutes Jo for jo for all instances because of the g option.

To create an executable sed script, include the shebang (#!) interpreter statement on the first line:

#!/bin/sed -f

If you make the script executable (e.g., chmod 700 [SCRIPTNAME]), you can call it like any other program. You wouldn't normally use this option. Rather, you would put sed and any script file calls in a shell script. In some cases, the order of the commands matters. Test your scripts before making them "real" to avoid errors and data loss.

1 2 3 4 Next »