by Rick McMullin

IN THIS CHAPTER

• Creating and Running Shell Programs

• Using Variables

• The Importance of Quotation Marks

• The test Command

• Conditional Statements

• Iteration Statements

• Functions

The last three chapters described how to use the most common Linux shell programs. I mentioned that these shell programs have powerful interpretive programming languages built into them. Now it's time to look at them in more detail.

This chapter describes the fundamentals of shell programming and compares the bash, pdksh, and tcsh programming languages. This chapter covers the following topics:

• Creating and running shell programs

• Using shell variables

• The importance of quotes

• The test command

• Conditional statements

• Iteration statements

This chapter contains several small examples of shell programs. Each new concept or command that is introduced has some example code that further helps to explain what is being presented.

At the simplest level, shell programs are just files that contain one or more shell or Linux commands. These programs can be used to simplify repetitive tasks, to replace two or more commands that are always executed together with a single command, to automate the installation of other programs, and to write simple interactive applications.

To create a shell program, you must create a file using a text editor and put the shell or Linux commands you want to be executed into that file. For example, assume you have a CD-ROM drive mounted on your Linux system. This CD-ROM device is mounted when the system is first started. If you later change the CD in the drive, you must force Linux to read the new directory contents. One way of achieving this is to put the new CD into the drive, unmount the CD-ROM drive using the Linux umount command, and then remount the drive using the Linux mount command. This sequence of steps is shown by the following commands:

umount /dev/cdrom

mount -t iso9660 /dev/cdrom /cdrom

Instead of typing both of these commands each time you change the CD in your drive, you could create a shell program that would execute both of these commands for you. To do this, put the two commands into a file and call the file remount (or any other name you want).

Several ways of executing the commands are contained in the remount file. One way to accomplish this is to make the file executable. This is done by entering the following command:

chmod +x remount

This command changes the permissions of the file so that it is now executable. You can now run your new shell program by typing remount on the command line.

Another way you can execute the shell program is to run the shell that the program was written for and pass the program in as a parameter to the shell. In a tcsh program, this is done by entering the following command:

tcsh remount

This command starts up a new shell and tells it to execute the commands that are found in the remount file.

A third way of executing the commands in a shell program file is to use the . command (in pdksh and bash) and the source command in tcsh. This command tells the shell to execute all the commands in the file that is passed as an argument to the command. For example, the following command can be used to tell bash or pdksh to execute the commands in the remount file:

. remount

To do the same thing in tcsh, you would type the following command:

source remount

Another situation in which a simple shell program can save a lot of time is described in the following example. Assume you were working on three different files in a directory, and at the end of every day you wanted to back up those three files onto a floppy disk. To do this you would type a series of commands similar to the following:

mount -t msdos /dev/fd0 /a

cp file1 /a

cp file2 /a

cp file3 /a

As stated in the example, one way of doing this would be to mount the floppy drive and then type three copy commands, one for each file you wanted to copy. A simpler way would be to put the four commands into a text file called backup and then execute the backup command when you wanted to copy the three files onto the floppy drive.

As is the case with almost any language, the use of variables is very important in shell programs. You saw some of the ways in which shell variables can be used in the introductory shell chapters. Two of the variables that were introduced were the PATH variable and the PS1 variable. These are examples of built-in shell variables, or variables that are defined by the shell program you are using. This section describes how you can create your own variables and use them in simple shell programs.

In all three of the shells I have discussed, you can assign a value to a variable simply by typing the variable name followed by an equal sign and the value you want to assign to the variable. For example, if you wanted to assign a value of 5 to the variable count, you would enter the following command in bash or pdksh:

count=5

With tcsh you would have to enter the following command to achieve the same results:

set count = 5

Notice that you do not have to declare the variable as you would if you were programming in C or Pascal. This is because the shell language is a non-typed interpretive language. This means that you can use the same variable to store character strings that you use to store integers. You would store a character string into a variable in the same way that you stored the integer into a variable. For example: name=Garry (for pdksh and bash)

set name = Garry (for tcsh)

Once you have stored a value into a variable, how do you get the value back out? You do this in the shell by preceding the variable name with a dollar sign ($). If you wanted to print the value stored in the count variable to the screen, you would do so by entering the following command:

echo $count

The shell has knowledge of a special kind of variable called a positional parameter. Positional parameters are used to refer to the parameters that were passed to a shell program on the command line or a shell function by the shell script that invoked the function. When you run a shell program that requires or supports a number of command-line options, each of these options is stored into a positional parameter. The first parameter is stored into a variable named 1, the second parameter is stored into a variable named 2, and so forth. These variable names are reserved by the shell so that you can't use them as variables you define. To access the values stored in these variables, you must precede the variable name with a dollar sign ($) just as you do with variables you define.

The following shell program expects to be invoked with two parameters. The program takes the two parameters and prints the second parameter that was typed on the command line first and the first parameter that was typed on the command line second.

#program reverse, prints the command line parameters out in reverse #order

echo "$2" "$1"

If you invoked this program by entering

reverse hello there

the program would return the following output:

there hello

Several other built-in shell variables are important to know about when you are doing a lot of shell programming. Table 13.1 lists these variables and gives a brief description of what each is used for.

The use of the different types of quotation marks is very important in shell programming. All three kinds of quotation marks and the backslash character are used by the shell to perform different functions. The double quotation marks (""), the single quotation marks (`'), and the backslash (\) are all used to hide special characters from the shell. Each of these methods hides varying degrees of special characters from the shell.

The double quotation marks are the least powerful of the three methods. When you surround characters with double quotes, all the whitespace characters are hidden from the shell, but all other special characters are still interpreted by the shell. This type of quoting is most useful when you are assigning strings that contain more than one word to a variable. For example, if you wanted to assign the string hello there to the variable greeting, you would type the following command:

greeting="hello there" (for bash and pdksh)

set greeting = "hello there" (for tcsh)

This command would store the hello there string into the greeting variable as one word. If you typed this command without using the quotes, you would not get the results you wanted. bash and pdksh would not understand the command and would return an error message. tcsh would assign the value hello to the greeting variable and ignore the rest of the command line.

Single quotes are the most powerful form of quoting. They hide all special characters from the shell. This is useful if the command that you enter is intended for a program other than the shell.

Because the single quotes are the most powerful, you could have written the hello there variable assignment using single quotes. You might not always want to do this. If the string being assigned to the greeting variable contained another variable, you would have to use the double quotes. For example, if you wanted to include the name of the user in your greeting, you would type the following command:

greeting="hello there $LOGNAME" (for bash and pdksh)

set greeting="hello there $LOGNAME" (for tcsh)

This would store the value hello there root into the greeting variable if you were logged in to Linux as root. If you tried to write this command using single quotes it wouldn't work, because the single quotes would hide the dollar sign from the shell and the shell wouldn't know that it was supposed to perform a variable substitution. The greeting variable would be assigned the value hello there $LOGNAME if you wrote the command using single quotes.

Using the backslash is the third way of hiding special characters from the shell. Like the single quotation mark method, the backslash hides all special characters from the shell, but it can hide only one character at a time, as opposed to groups of characters. You could rewrite the greeting example using the backslash instead of double quotation marks by using the following command:

greeting=hello\ there (for bash and pdksh)

set greeting=hello\ there (for tcsh)

In this command, the backslash hides the space character from the shell, and the string hello there is assigned to the greeting variable.

Backslash quoting is used most often when you want to hide only a single character from the shell. This is usually done when you want to include a special character in a string. For example, if you wanted to store the price of a box of computer disks into a variable named disk_price, you would use the following command:

disk_price=\$5.00 (for bash and pdksh)



set disk_price = \$5.00 (for tcsh)

The backslash in this example would hide the dollar sign from the shell. If the backslash were not there, the shell would try to find a variable named 5 and perform a variable substitution on that variable. Assuming that no variable named 5 were defined, the shell would assign a value of .00 to the disk_price variable. This is because the shell would substitute a value of null for the $5 variable.

The back quote marks ('') perform a different function. They are used when you want to use the results of a command in another command. For example, if you wanted to set the value of the variable contents equal to the list of files in the current directory, you would type the following command:

contents='ls' (for bash and pdksh)



set contents = 'ls' (for tcsh)

This command would execute the ls command and store the results of the command into the contents variable. As you will see in the section "Iteration Statements," this feature can be very useful when you want to write a shell program that performs some action on the results of another command.

In bash and pdksh, a command called test is used to evaluate conditional expressions. You would typically use the test command to evaluate a condition that is used in a conditional statement or to evaluate the entrance or exit criteria for an iteration statement. The test command has the following syntax:

test expression

or

[ expression ]

Several built-in operators can be used with the test command. These operators can be classified into four groups: integer operators, string operators, file operators, and logical operators.

The shell integer operators perform similar functions to the string operators except that they act on integer arguments. Table 13.2 lists the test command's integer operators.

The string operators are used to evaluate string expressions. Table 13.3 lists the string operators that are supported by the three shell programming languages.

The test command's file operators are used to perform functions such as checking to see if a file exists and checking to see what kind of file is passed as an argument to the test command. Table 13.4 lists the test command's file operators.

The test command's logical operators are used to combine two or more of the integer, string, or file operators or to negate a single integer, string, or file operator. Table 13.5 lists the test command's logical operators.

The tcsh does not have a test command, but it supports the same function using expressions. The expression operators that tcsh supports are almost identical to those supported by the C language. These expressions are used mostly in the if and while commands, which are covered later in this chapter in the "Conditional Statements" and "Iteration Statements" sections.

The tcsh expressions support the same kind of operators as the bash and pdksh test command. These are integer, string, file, and logical expressions. The integer operators supported by tcsh expressions are listed in Table 13.6.

The string operators that tcsh expressions support are listed in Table 13.7.

The file operators that tcsh expressions support are listed in Table 13.8.

The logical operators that tcsh expressions support are listed in Table 13.9.

The bash, pdksh, and tcsh each have two forms of conditional statements. These are the if statement and the case statement. These statements are used to execute different parts of your shell program depending on whether certain conditions are true. As with most statements, the syntax for these statements is slightly different between the different shells.

All three shells support nested if...then...else statements. These statements provide you with a way of performing complicated conditional tests in your shell programs. The syntax of the if statement is the same for bash and pdksh and is shown here:

if [ expression ]

then

      commands

elif [ expression2 ]

then

      commands

else

      commands



fi

The elif statement is an abbreviation of else if. This statement is executed only if none of the expressions associated with the if statement or any elif statements before it were true. The commands associated with the else statement are executed only if none of the expressions associated with the if statement or any of the elif statements were true.

In tcsh, the if statement has two different forms. The first form provides the same function as the bash and pdksh if statement. This form of if statement has the following syntax:

if (expression1) then

      commands

else if (expression2) then

      commands

else

      commands

ndif

The second form of if statement provided by tcsh is a simple version of the first if statement. This form of if statement evaluates only a single expression. If the expression is true, it executes a single command; if the expression is false, nothing happens. The syntax for this form of if statement is the following:

if (expression) command

This statement could be written using the first form of if statement by writing the if without any else or else if clauses. This form just saves a little typing.

The following is an example of a bash or pdksh if statement. This statement checks to see if there is a .profile file in the current directory:

if [ -f .profile ]

then

      echo "There is a .profile file in the current directory."

else

      echo "Could not find the .profile file."



fi

The same statement written using the tcsh syntax is shown here:

#

if ( { -f .profile } ) then

      echo "There is a .profile file in the current directory."

else

      echo "Could not find the .profile file."

endif

The case statement enables you to compare a pattern with several other patterns and execute a block of code if a match is found. The shell case statement is quite a bit more powerful than the case statement in Pascal or the switch statement in C. This is because in the shell case statement you can compare strings with wildcard characters in them, whereas with the Pascal and C equivalents you can compare only enumerated types or integer values.

Once again, the syntax for the case statement is identical for bash and pdksh and different for tcsh. The syntax for bash and pdksh is the following:

case string1 in

      str1)

            commands;;

      str2)

            commands;;

      *)

            commands;;

esac

string1 is compared to str1 and str2. If one of these strings matches string1, the commands up until the double semicolon (;;) are executed. If neither str1 nor str2 matches string1, the commands associated with the asterisk are executed. This is the default case condition because the asterisk matches all strings.

The tcsh equivalent of the bash and pdksh case statement is called the switch statement. This statement's syntax closely follows the C switch statement syntax. Here it is:

switch (string1)

      case  str1:

            statements

      breaksw

      case  str2:

            statements

      breaksw

      default:

            statements

      breaksw



endsw

This behaves in the same manner as the bash and pdksh case statement. Each string following the keyword case is compared with string1. If any of these strings matches string1, the code following it up until the breaksw keyword is executed. If none of the strings matches, the code following the default keyword up until the breaksw keyword is executed.

The following code is an example of a bash or pdksh case statement. This code checks to see if the first command-line option was -i or -e. If it was -i, the program counts the number of lines in the file specified by the second command-line option that begins with the letter i. If the first option was -e, the program counts the number of lines in the file specified by the second command-line option that begins with the letter e. If the first command-line option was not -i or -e, the program prints a brief error message to the screen.

case $1 in

   -i)

      count='grep ^i $2 | wc -l'

      echo "The number of lines in $2 that start with an i is $count"

      ;;

   -e)

      count='grep ^e $2 | wc -l'

      echo "The number of lines in $2 that start with an e is $count"

      ;;

   * )

      echo "That option is not recognized"

      ;;

esac

The same example written in tcsh syntax is shown here:

# remember that the first line must start with a # when using tcsh

switch ( $1 )

   case -i | i:

      set count = 'grep ^i $2 | wc -l'

      echo "The number of lines in $2 that begin with i is $count"

   breaksw

   case -e | e:

      set count = 'grep ^e $2 | wc -l'

      echo "The number of lines in $2 that begin with e is $count"

   breaksw

   default:

      echo "That option is not recognized"

   breaksw

endsw

The shell languages also provide several iteration or looping statements. The most commonly used of these is the for statement.

The for statement executes the commands that are contained within it a specified number of times. bash and pdksh have two variations of the for statement.

The first form of for statement that bash and pdksh support has the following syntax:

for var1 in list

do

      commands

done

In this form, the for statement executes once for each item in the list. This list can be a variable that contains several words separated by spaces, or it can be a list of values that is typed directly into the statement. Each time through the loop, the variable var1 is assigned the current item in the list, until the last one is reached.

The second form of for statement has the following syntax:

for var1

do

      statements

done

In this form, the for statement executes once for each item in the variable var1. When this syntax of the for statement is used, the shell program assumes that the var1 variable contains all the positional parameters that were passed in to the shell program on the command line.

Typically this form of for statement is the equivalent of writing the following for statement:

for var1 in "$@"

do

      statements

done

The equivalent of the for statement in tcsh is called the foreach statement. It behaves in the same manner as the bash and pdksh for statement. The syntax of the foreach statement is the following:

foreach name (list)

      commands



end

The following is an example of the bash or pdksh style of for statement. This example takes as command-line options any number of text files. The program reads in each of these files, converts all the letters to uppercase, and then stores the results in a file of the same name but with a .caps extension.

for file

do

tr a-z A-Z < $file >$file.caps

done

The same example written in tcsh shell language is shown next:

#

foreach file ($*)

   tr a-z A-Z < $file >$file.caps

end

Another iteration statement offered by the shell programming language is the while statement. This statement causes a block of code to be executed while a provided conditional expression is true. The syntax for the while statement in bash and pdksh is the following:

while expression

do

      statements

done

The syntax for the while statement in tcsh is the following:

while (expression)

      statements

end

The following is an example of the bash and pdksh style of while statement. This program lists the parameters that were passed to the program, along with the parameter number.

count=1

while [ -n "$*" ]

do

      echo "This is parameter number $count $1"

      shift

      count='expr $count + 1'



done

As you will see in the section titled "The shift Command," the shift command moves the command-line parameters over one to the left.

The same program written in the tcsh language is shown next:

#

set count = 1

while ( "$*" != "" )

      echo "This is parameter number $count $1"

      shift

      set count = 'expr $count + 1'

end

The until statement is very similar in syntax and function to the while statement. The only real difference between the two is that the until statement executes its code block while its conditional expression is false, and the while statement executes its code block while its conditional expression is true. The syntax for the until statement in bash and pdksh is

until expression

do

      commands

done

The same example that was used for the while statement can be used for the until statement. All you have to do to make it work is negate the condition. This is shown in the following code:

count=1

until [ -z "$*" ]

do

      echo "This is parameter number $count $1"

      shift

      count='expr $count + 1'

done

The only difference between this example and the while statement example is that the -n test command option (which means that the string has nonzero length) was removed, and the -z test option (which means that the string has zero length) was put in its place.

In practice the until statement is not very useful, because any until statement you write can also be written as a while statement.

bash, pdksh, and tcsh all support a command called shift. The shift command moves the current values stored in the positional parameters to the left one position. For example, if the values of the current positional parameters are

$1 = -r  $2 = file1  $3 = file2

and you executed the shift command

shift

the resulting positional parameters would be as shown here:

$1 = file1  $2 = file2

You can also move the positional parameters over more than one place by specifying a number with the shift command. The following command would shift the positional parameters two places:

shift 2

This is a very useful command when you have a shell program that needs to parse command-line options. This is true because options are typically preceded by a hyphen and a letter that indicates what the option is to be used for. Because options are usually processed in a loop of some kind, you often want to skip to the next positional parameter once you have identified which option should be coming next. For example, the following shell program expects two command-line options--one that specifies an input file and one that specifies an output file. The program reads the input file, translates all the characters in the input file into uppercase, then stores the results in the specified output file.

while [ "$1" ]
do
if [ "$1" = "-i" ] then
infile="$2"
shift 2
elif [ "$1" = "-o" ]
then
outfile="$2"
shift 2
else
echo "Program $0 does not recognize option $1"
fi
done

tr a-z A-Z <$infile >$outfile

---

pdksh offers one iteration statement that neither bash nor tcsh provides. This is the select statement. This is actually a very useful statement. It is quite a bit different from the other iteration statements because it actually does not execute a block of shell code repeatedly while a condition is true or false. What the select statement does is enable you to automatically generate simple text menus. The syntax for the select statement is

select menuitem [in list_of_items]

do

      commands



done

where square brackets are used to enclose the optional part of the statement.

When a select statement is executed, pdksh creates a numbered menu item for each element in the list_of_items. This list_of_items can be a variable that contains more than one item, such as choice1 choice2, or it can be a list of choices typed in the command. For example:

select menuitem in choice1 choice2 choice3

If the list_of_items is not provided, the select statement uses the positional parameters just as with the for statement.

Once the user of the program containing a select statement picks one of the menu items by typing the number associated with it, the select statement stores the value of the selected item in the menuitem variable. The statements contained in the do block can then perform actions on this menu item.

The following example illustrates a potential use for the select statement. This example displays three menu items, and when the user chooses one of them it asks whether that was the intended selection. If the user enters anything other than y or Y, the menu is redisplayed.

select menuitem in pick1 pick2 pick3

do

      echo "Are you sure you want to pick $menuitem"

      read res

      if [ $res = "y" -o $res = "Y" ]

      then

            break

      fi

done

A few new commands are introduced in this example. The read command is used to get input from the user. It stores anything that the user types into the specified variable. The break command is used to exit a while, until, repeat, select, or for statement.

tcsh has an iteration statement that has no equivalent in pdksh or bash. This is the repeat statement. The repeat statement executes a single command a specified number of times. The syntax for the repeat statement is the following:

repeat count command

The following is an example of the repeat statement. It takes a set of numbers as command-line options and prints that number of periods to the screen. This program acts as a very primitive graphing program.

#

foreach num ($*)

      repeat $num echo -n "."

      echo ""



end

The shell languages enable you to define your own functions. These functions behave in much the same way as functions you define in C or other programming languages. The main advantage of using functions as opposed to writing all of your shell code in line is for organizational purposes. Code written using functions tends to be much easier to read and maintain and also tends to be smaller, because you can group common code into functions instead of putting it everywhere it is needed.

The syntax for creating a function in bash and pdksh is the following:

fname () {

      shell commands



}

pdksh also allows the following syntax:

function fname {

      shell commands

}

Both of these forms behave in the same way.

Once you have defined your function using one of these forms, you can invoke it by entering the following command:

fname [parm1 parm2 parm3 ...]

Notice that you can pass any number of parameters to your function. When you do pass parameters to a function, it sees those parameters as positional parameters, just as a shell program does when you pass it parameters on the command line. For example, the following shell program contains several functions, each of which is performing a task associated with one of the command-line options. This example illustrates many of the topics covered in this chapter. It reads all the files that are passed on the command line and--depending on the option that was used--writes the files out in all uppercase letters, writes the files out in all lowercase letters, or prints the files.

upper () {

      shift

      for i

      do

          tr a-z A-Z <$1 >$1.out

          rm  $1

          mv $1.out $1

          shift

      done; }



lower () {

      shift

      for i

      do

          tr A-Z a-z <$1 >$1.out

          rm $1

          mv $1.out $1

          shift

      done; }



print () {

      shift

      for i

      do

            lpr $1

            shift

      done; }



usage_error () {

      echo "$1 syntax is $1 <option> <input files>"

      echo ""

      echo "where option is one of the following"

      echo "p  -- to print frame files"

      echo "u  -- to save as uppercase"

      echo "l  -- to save as lowercase"; }



case $1

in

      p | -p)     print $@;;

      u | -u)           upper $@;;

      l | -l)     lower $@;;

      *)          usage_error $0;;

esac

This chapter introduced you to many of the features of the bash, pdksh, and tcsh programming languages. As you become familiar with using Linux, you will find that you use shell programming languages more and more often.

Even though the shell languages are very powerful and also quite easy to learn, you might run into some situations where shell programs are not suited to the problem you are solving. In these cases you may want to investigate the possibility of using one of the other languages available under Linux. Some of your options are C and C++, which are described in Chapters 27, "Programming in C," and 28, "Programming in C++"; gawk, which is described in Chapter 26, "gawk"; and Perl, which is described in Chapter 29, "Perl."