Page 437
by Cameron Laird
Page 438
"[T]he three great virtues of a programmer: laziness, impatience, and hubris."
Wall & Schwartz, in Programming Perl
Automation enlists a machine—a Linux computer, in the present case—to perform jobs. What makes this definition live, though, and the true subject of this chapter, is attitude. The most important step you can take in understanding mechanisms of automation under Red Hat Linux is to adopt the attitude that the computer works for you. After you've done that, when you realize you're too lazy to type in a telephone number that the machine should already know, or too impatient to wait until midnight to start backups, and when you have enough confidence in your own creativity to teach the machine a better way, the technical details will work themselves out. This chapter offers more than a dozen examples of how small, understandable automation initiatives make an immediate difference. Let them lead you to your own successes.
How can the details work out? Let's look at an example from yesterday (the day before I started to write this chapter).
A client wanted to enhance an online catalog to include thumbnail pictures of the merchandise. After a bit of confusion about what this really meant, I realized that I needed to update a simple database table of products to include a new column (or attribute, or value) that would specify the filenames of the thumbnails. The database management system has a couple of interactive front ends, and I'm a swift typist, so it probably would have been quickest to point and click my way through the two hundred picture updates. Did I do that? Of course not—what happened later proved the wisdom of this decision. Instead, I wrote a shell script to automate the update, which is shown in Listing 22.1.
Listing 22.1. A shell script that updates a database.
1: # picture names seem to look like {$DIR/137-13p.jpg,$DIR/201-942f.jpg,...} The
2: # corresponding products appear to be {137-13P, 201-942F, ...}
3: DIR=/particular/directory/for/my/client
4:
5: # Will we use .gif-s, also, eventually? I don't know.
6: for F in $DIR/*.jpg
7: do
8: # BASE will have values {137-13p,201-942f, ...}
9: BASE=`basename $F .jpg`
10: # The only suffixes I've encountered are `p' and `f', so I'll simply
11: # transform those two.
12: # Example values for PRODUCT: {137-13P, 201-942F, ...}
13: PRODUCT=`echo $BASE | tr pf PF`
14: # one_command is another shell script, that passes a line of SQL to
Âthe DBMS.
15: one_command update catalog set Picture = "`$DIR/$BASE.jpg'" where Product
Â= "`$PRODUCT'"
16: done
Page 439
As it turned out, the team decided within a couple days that the pictures needed to be in a different directory, so it was only a few seconds' work to update the penultimate line of the script and add a comment, such as
...
# Do *not* include a directory specification in Picture; that will be
# known only at the time the data are retrieved.
one_command update catalog set Picture = "`$BASE.jpg'" where Product = Â"`$PRODUCT'"
done
and rerun it. It's inevitable we'll someday have more pictures to add to the database or will want reports on orphaned pictures (those that haven't been connected yet to any product), and this same script, or a close derivative of it, will come into play again.
Let's work through the example in Listing 22.1 in detail, to practice the automation mentality.
Do you understand how the script in Listing 22.1 works? Chapter 21, "Shell Programming," explains shell processing, and Appendix B, "Top 50 Linux Commands and Utilities," presents everything you're likely to need about the most commonly used UNIX utilities. You can always learn more about these by reading the corresponding man pages or any of the fine books available on shell programming. The most certain way to learn, of course, is to experiment on your own. For example, if you have any question about what man tr means by "…translation…," it's an easy matter to experiment, such as with
tr pf PF <<HERE abcopqOPQ FfpPab HERE
and conclude that you're on the right track when you see the following:
abcoPqOPQ FFPPab
This is one of the charms of relying on shells for automation; it's easy to bounce between interaction and automation, which shapes a powerful didactic perspective and a check on understanding.
The sample product catalog script in Listing 22.1 is written for sh processing. I strongly recommend this be your target for scripts, rather than ksh, csh, or bash. I much prefer any of the latter for interactive, command-line use. In automating, though, when I'm often connecting to hosts that don't use Red Hat Linux, availability and esoteric security issues have convinced me to code using constructs that sh, and therefore all the shells, recognizes. Default Red Hat Linux installations link /bin/sh and /bin/bash. All the work in this chapter, though, is written so that it will function properly no matter what the details are of your host's configuration. Chapter 21 gives more details on the differences among shells, and the page
Page 440
http://starbase.neosoft.com/~claird/comp.unix.shell/shell.html supplements this with a few remarks targeted particularly to readers of this chapter. Incidentally, if you have a question that the URLs given here don't answer, e-mail me at claird@neosoft.com, and I'll update the pages available on the World Wide Web.
Did I really include the in-line comments, the lines that begin with #, when I first wrote the script in Listing 22.1? Yes. I've made this level of source code documentation a habit, and it's one I recommend to you. If your life is at all like mine, telephones ring, coworkers chat, and power supplies fail; I find it easier to type this much detail as I'm thinking about it, rather than risk having to re-create my thoughts in case of an interruption. Also, it's much easier to pick up the work again days or weeks later. Writing for human readability also eases the transition when you pass your work on to others.
Listing 22.1 begins by assigning a shell variable
DIR in line 3. It's good practice to make such
an assignment, even for a variable (apparently) used only once. It contributes to self-
documentation and generally enhances maintainability; it's easy to look at the top of the
script and see immediately on what magic words or configuration in the outside environment
(/particular/directory/for/my/client, in this case; see line 3) the script depends.
Many of the jobs you'll want to accomplish involve a quantifier: "change all…," "correct every…," and so on. The shell's looping constructs, for and while, are your friends. You'll make almost daily use of them.
basename and tr are universally available and widely used. tr, like many UNIX utilities, expects to read standard input. If you have information in shell variables, you can feed tr the information you want, either through a pipe from echo, as in
echo $VARIABLE | tr [a-z] [A-Z]
or an equivalent, or with a so-called HERE document, such as
tr [a-z] [A-Z] <<HERE $VARIABLE HERE
or perhaps by creating a temporary file:
echo $VARIABLE >$TMPFILE tr [a-z] [A-Z] $TMPFILE
one_command, as invoked in line 15 of Listing 22.1, is a two-line shell script I had written earlier in the day to process SQL commands. Why not in-line the body of that script here? Although that's technically feasible, I have a strong preference for small, simple programs that are easy to understand and correspondingly easy to implement correctly. one_command already has been verified to do one small job reliably, so the script lets it do that job. This fits with the UNIX tradition that counsels combining robust toolkit pieces to construct grander works.
In fact, notice that the example in Listing 22.1 shows the shell's nature as a "glue" language. There's a small amount of processing within the shell in manipulating filenames, and then most