In this chapter you will be introduced to Perl and its origins in the Unix operating system. We will explore the initial reasons for Perl's creation as a handy systems administration tool kit. Then, we touch on how Perl has evolved into a CGI (common gateway interface) language that can be used with the Windows NT server. Finally, I will cover what you will need to operate Perl on your system, from the hardware requirements to the software necessary, as well as the limitations Perl has. This should ready you for getting your feet wet in the next chapter by actually learning how Perl works.
While this book demonstrates how Perl is used to give Web pages more depth in their interactivity, Perl wasn't created to do this. It was originally created because someone wanted to make their job of administering a large network a little easier. All the programming tools available then didn't work as well as he or she needed them to, so out popped Perl. The man who created Perl was Larry Wall. Perl stands for Practical Extraction and Report Language. Some time ago Larry Wall wanted to develop a better way to create debugging reports. He was working in the UNIX operating system trying to produce bug reports from a Usenet-new-like hierarchy of files. He first tried the UNIX utility awk to do this, but he couldn't get the results he needed. Awk is a common UNIX utility that uses regular expressions to manipulate and process text. Building on this application, Wall developed the first version of Perl. This new tool proved handier than he had first anticipated, so he gave copies of this new language to some other UNIX programmers. They worked with Perl and made it perform more tasks. Wall then developed Perl to a state where he could release it as a general systems administrator utility for the greater UNIX world.
Interest in Perl grew, so versions of Perl for other operating systems were developed, one of which was ported to the Windows NT operating system. Wall has referred to Perl's development as the melting of one magic bead after another into each other, so that the end result contains more magic than all the beads added together separately.
Portability can also refer to the ability of a file to move from one operating system to another. For example, when you take an MS Word document from a Macintosh computer to a PC you are still able to work on the file on the new computer with little modification necessary, making MS Word documents highly portable between operating systems that can use MS Word.
Wall, with the help of others, has crafted Perl into five different versions, each one improving on the previous. Perl is now available in version 5.001 for Windows NT.
The world of Perl is now a considerable size, with the support of a 211-page manpage (the name given to an online manual), which can be found at http://www.perl.com/perl/info/documentation.html#online in these formats:
In addition, there are several books devoted to Perl, and six
Perl newsgroups:
| NOTE |
Porting and portability refer to the process of taking an existing language, application, or file type that runs on one operating system and translating the source code so it can be used on another, totally different, system. Sometimes in this translation process certain features of the original language are lost because of the limitations of the new operating system. With the different portings of Perl, some features are modified or omitted. These omissions are usually well documented, and these unsupported features in Perl for Windows NT can be found in a table in Appendix |
There are many other online resources for Perl, some of which are detailed later in this chapter.
In the beginning Perl was designed to be a data reduction language. This means that it could steer through an assortment of files, examine large amounts of text, call up commands to get fluid, or dynamic, data, and output reports based on this information. Previously, programmers have had to resort to dedicated C programs to accomplish this. If you have ever had to program in C you will realize that, given a choice, no one who is not a programmer at heart wants to have to use C if they can help it. Perl speeds up its processing time by being self-compiling, making it very attractive to the rest of us. Perl is also compatible with C.
As more programmers used Perl it was discovered that it could also manipulate files quite easily. It is a language that gives the programmer the ability to deal with the files themselves, separate from what they contain. This has made Perl very popular, because it allows you to both change a file's contents and its very nature with the same utility.
What makes Perl even more popular is that it is free, and very easy to find and download for your computer. Details about where to find Perl are covered later in this chapter.
Perl is distributed under both the GNU General Public License and the Artistic License agreement. When you install Perl you can choose which agreement you are going to use Perl under. Read both carefully before you decide which fits your situation best. A full copy of each is found in Appendix A to this book.
The world that produced Perl is like yours; it is made of information
that you need, distribute, and manage every day. It doesn't matter
that Perl originated in the UNIX world, because the problems it
was created to solve are familiar to anyone working with a network,
regardless of platform.
| NOTE |
Your computer, its network, and the Internet at large is where information is posted, traded, sold, and given away in many different formats, depending on each user's needs. All this myriad data shares one common trait: it is based on text. When I use the word text I don't want to limit its meaning to only letters that we use to make words. This is just a small part of what text is. In this book text refers to any character, be it letter, number, or symbol, that the computer can take in, process, and/or output. Text is the building block for each file stored in your server's memory and can be interpreted into various outputs based on the user requests. The Common Gateway Interface (CGI) works only with text, moving it in and out of your server, making text fluency a must for any language used for CGI tasks. That language has to be one that fundamentally works with text on all levels: a language such as Perl. |
When we use the Web, each of us looks for newer and better ways to receive and present our information. Moving away from static Web pages, we have discovered how to use CGI specifications to create interactive pages for our Web sites. If text is the basis of this interactive information flow, then we need a language that can both deal with text quickly and efficiently, and which can also manipulate the text files themselves. Welcome to Perl.
Originally Larry Wall wanted a powerful but simple way to output reports. Wall was working on a bug reporting system that would work on a hierarchy of file directories. He first tried the UNIX utility awk. Awk is used by UNIX programmers to manipulate structured data and produce formatted reports. Awk's main power lies behind the use of regular expressions, a syntax that is common to UNIX utilities. Regular expressions are a powerful way to manipulate text, and this power is harnessed by Perl. The gory details about regular expressions in Perl are dealt with in Chapter 3
Perl was developed to help UNIX users with tasks that are too heavy, or too portability-sensitive, for a shell. With UNIX, the operating system is utilized through a shell, like the Bourne or Korn shell. The shell provides an interface where the UNIX user can run applications, rename files, and perform various housekeeping tasks. This is different from the Windows method of interfacing that Microsoft has developed, and more like using DOS or MS-DOS.
In this instance you can compare using a UNIX shell to using MS-DOS. Both appear on the screen in the form of a text-driven terminal screen and both operate from the command line. Often software used by Windows is set up using MS-DOS first to extract and install the various files involved. The commands available to you in MS-DOS to perform tasks beyond this are fairly limited. For a larger job, like automating a login sequence, you have to use a short program, sometimes called a script.
Instead of creating an entire specialized program using a high-level language like C, you might want to write a simple script using a mid-level, or scripting, language. A simple comparison will illustrate the difference between the two languages.
Listing 1.1 is a script in C to replace all the occurrences of the letter "o" in a string[]:
Listing 1.1 Example of a Script in C
#include <memory.h>
#include <string.h>
#include <stdio.h>
int ch = 'o';
int newch = 'p';
char string[] = "The quick brown dog jumps over the lazy fox";
void main()
{
char *pdest;
int result;
printf( "String to be searched:\n\t\t%s\n", string );
printf( "Search char:\t%c\n", ch );
printf( "Replace with char:\t%c\n", newch );
pdest = strchr( string, ch );
while ( pdest != NULL )
{
result = pdest - string + 1;
printf( "Result:\t\t%c found at position %d\n\n", ch, result );
string[result]=newch;
pdest = strchr( string, ch );
}
printf( "The new String:\n\t\t%s\n", string );
}
Listing 1.2 is the comparable script in Perl. It is obvious-even to the inexperienced programmer-which language is easier to use when handling text at this level.
In the UNIX world, Perl was created as just such a language. Its purpose was to handle these mid-level jobs that would be too much for the shell, and too little to justify using C.
This is not to imply that MS-DOS is exactly like a UNIX shell. There are several major differences between them, the biggest of which is how they differ in the method by which they handle the resources available to the computer. This comparison of the two terminal windows is only meant on the conceptual level, and using a UNIX shell is very similar in experience to using MS-DOS, for those of you who have never worked with UNIX.
Listing 1.2 Perl Script
#!/usr/bin/perl $string = "The quick brown dog jumps over the lazy fox"; $ch = 'o'; $newch = 'p'; print "String to be searched:\n\t\t$string\n"; print "Search Char: $ch\n"; print "Replace with Char: $newch\n"; $string =~s/$ch/$newch/g; print "The new string:\n\t\t$string\n";
a UNIX shell is very similar in experience to using MS-DOS, for
those of you who have never worked with UNIX
| NOTE |
A real examination of MS-DOS and UNIX is a little outside the scope of this book, but if you want to learn more about UNIX and UNIX shells, you can try these books published by O'Reilly & Associates, Inc.: Grace Todino, John Stang, and Jerry Peek's Learning the UNIX Operating System Daniel Gilly's UNIX in a Nutshell: System V Edition Bill Rosenblatt's Learning the Korn Shell Dale Dougherty's sed & awk |
The initial purpose of Perl has grown by leaps and bounds. Now found among Perl's various applications is the ability for Perl to act as a CGI language on an Internet or an intranet server to help provide Web services and systems administration tools. It is this application of Perl that interests us most in this book.
Perl creates a more succinct and clear way of doing many tasks that C, or one of the UNIX shells, does, but without a lot of fuss. Unlike a compiled language like C or C++, Perl is an interpreted script language. One of the things that means is that it has a much shorter development cycle.
Perl is also flexible. It runs on many different server setups without needing much, if any, reconfiguration. Perl is available for the Sun OS, AIX, Windows NT (WinPerl), and even Macintosh platforms.
Perl, especially where CGI is concerned, is extremely portable. If you want to use a script on a different operating system, you just pack up your Perl file and bring it along. Think about Perl scripts as though they were Lego blocks, and consider how you can use different blocks that have their own function to build new structures, then take them apart to build something different. Not only can you develop Perl libraries to assist you in creating more complex scripts, but you can also bring your work home with you. Those of you in the freelance world are quite familiar with the pressure placed on you to solve problems quickly. Perl lets you develop a very efficient bag of tricks that you can take from project to project that will let you spend more of your time solving the problem instead of hammering out unique scripts each and every time.
Perl is a language that is easy to read. The commands in a Perl script are much more accessible than many other languages, usually because other older languages have kept archaic and mystical aspects that just confuse most people.
Perl is also quite powerful because it combines the versatility and efficiency of a shell language with the ability to make low-level system calls that you would expect from a C program. Perl can work with many different text search tools, such as WAIS or Glimpse, relational databases, such as Sybase and Oracle, and object-relational packages like Illustra. This is a point that no Web programmer should ever forget when dealing with the many different executables on the market. Though your software ranges in its standards and preferences, Perl can work with each one of them. Perl functions on a level below a programmable "layer" between the user and an information source, like a text file.
It must be stated though that not all of your CGI problems will be solved with Perl. Often, a combination of CGI languages is needed to overcome bug hurdles. There are no hard and fast programming rules with CGI, so a clear understanding of the client-server cycle, and its related protocols, should be taken to heart.
While Perl was originally designed with the UNIX operating system in mind, there are now several other UNIX-like operating systems that can handle it, Windows NT being one of them.
For being such a relatively new programming language, Perl has a very active programming community behind it. In fact, it's hard to find someone who has anything bad to say about this language. There are many sites on the World Wide Web that have Perl FAQ lists, archives, and script libraries. Some of the more popular of these are listed below:
For Windows NT you need to use a version of Perl that has been ported to the Windows NT environment. You will probably want to install Perl on any workstations connected to your server locally (in a LAN or WAN configuration) so that you can utilize Perl on your own network as well.
On the Web you can find Perl for Windows NT (and lots of other NT goodies) at http://www.perl.hip.com/.
This is the primary place to learn about all Windows NT Perl 5.001 concerns. They also house two e-mail lists, revolving around Windows NT Perl 5.001, that can be subscribed to by going to the above URL and registering. You can also join by sending an e-mail message to: majordomo@mail.hip.com with this text in the body (not the subject line) of the message:
subscribe ntperl
and the announcement list is joined by sending a message to:
majordomo@hip.com
with the message:
subscribe ntperl_announce
in the message body.
A strong and loyal support group has developed around Perl as it has grown in popularity. A good source of information relating to all aspects of Perl can be found on the newsgroups already discussed in this chapter. Larry Wall and other Perl architects post articles regularly. Wall and the other Perl architects are quite keen on advancing Perl's power, and they very actively try to solve bugs and add requested features to upcoming versions of Perl. While sending e-mail messages to Wall directly generally gets a response, the best way to communicate your wants and needs, or report a bug, is to post to the newsgroups listed above.
You can also access Perl gurus by sending a request to perl-users-request@virginia.edu, where a human will send you information on how to receive the Perl Users Digest which contains answers to many questions, including several Perl scripts in every issue, as well as the guidelines to how it works. There is also an extensive Perl FAQ listing at http://www.perl.com/perl/faq/index.html. Where else can you get this level of customer support, especially for something that's free?
There are several considerations that need to be dealt with before you install Perl on your system, such as the hardware and software requirements. These considerations are outlined here, as well as the inclusion of a security alert that will affect where you decide to store the Perl interpreter perl.exe.
Thanks to the dedicated people at hip.com there is a wonderful version of Perl ready for Windows NT.
The system requirements needed for running Perl 5.001 on Windows NT are a Windows NT Workstation 3.5 or an Advanced Server 3.5 or later, with at least 2MB of free disk space. The recommended amount of free space for maintaining a working Perl system is at least 10MB. Perl for Windows NT has been constructed with Microsoft Visual C++, version 2.0. The Windows NT Perl 5.001 source release has included the source files and test scripts. There is also a makefile that can be used to build Windows NT Perl 5.001 from your Software Development Kit, or SDK, if you regularly use one. If some of this doesn't make sense to you, don't worry, just install Perl from the command line using install.bat. You will need at least 15MB of free disk space to house Perl 5.001.
In order to use Perl, you need to be able to run either the Windows NT 3.51 server or workstation operating system. You should have a copy of Microsoft's Internet Information Server to make full use of CGI functions like Server Side Includes, and an NT Web or HTTP service like the one that comes with IIS, or the EMWAC HTTP server. There are many other software toolkits out there that will make your job of designing and administering a Web site on NT easier.
Installation instructions are also found in the INSTALL.TXT file of Perl 5.001. A successful installation begins by starting the install from the Perl 5.001 directory. First, run the executable "install" program from the command line in MS-DOS. The "install" program will add Perl-related entries to the Registry; create the directory \perl\bin; add the file perl.exe and then update its name to perl.{version number}.exe; add \perl\bin to your path; and create a log of the installation events in \Perl\5.001\install.log. Next, copy any of your own Perl scripts or libraries to the \perl\bin directory. This should allow you to run your scripts from any other directory.
Alternatively, you can build Windows NT Perl 5.001 from the source release using the makefile provided. If you are using Visual C++, start Microsoft Visual C and load Perl\5.001\src\perl.mak. You can build both perlglob and Perl using the assigned targets that have been predefined for your platform. If you are using SDK, run nmake from the Perl\5.001\src directory. This builds Perl.exe and PerlGlob.exe using the Perl\5.001\src\makefile. Perl.exe is the actual Perl interpreter, while PerlGlob.exe is involved with processing the regular expressions in your Perl scripts. If you do not regularly use an SDK to install software of this nature, please use the install.exe to put Perl on your computer.
Regardless of which installation method you use, the next step is to test the Perl binary by running the test scripts. Go to the directory Perl\5.001\scr\ntt and run "test." This will test the Perl installation and create a report on all the functions tested, as well as any functions not currently in operation.
The final step, if you do not use the SDK, is to run "install" again, which is in Perl\5.001. This copies perl.exe to the Perl\5.001 directory and completes the rest of the installation process as was outlined previously. It is important to install the perl.exe again so that any corrections made during the testing phase become permanent.
While Windows NT Perl 5.001 was created for Windows NT 3.51, it should run on Windows 95. Since this was not the primary environment for Windows NT Perl 5.001, Windows 95 has not been tested thoroughly and has a few known bugs.
The scripts that are converted from pl2bat into batch files don't run from the command line. The batch file uses cmd.exe, the Windows NT command shell, to run Perl. Replacing all cmd.exe with command.com should solve this hitch.
Win32 extensions, like Win32 Registry and OLE, might not work properly on Windows 95 with Perl 5.001 installed.
The Perl 5.001 install program does not work on Windows 95. These must be implemented manually. To install Perl 5.001 on Windows 95:
For those of you who have worked with Perl on other platforms, Appendix A has a list of Perl 5.001 functions that are not supported for Windows NT. The list entitled "Unsupported Routines" lists the various network routines, system V compatible IPC routines, I/O routines, filesystem routines, process related routines, and miscellaneous routines not included in this porting of Perl.
To ensure quick and easy installation of Perl 5.001 on your system, read these tips to avoid unnecessary hazards.
The first thing that you should know is once you have installed Perl you have a choice to make. You may not want to choose, but you have it to make anyway. To execute Perl scripts, Windows NT has to know that it is a Perl script, and where to locate the support files and executables that will make your Perl script work. In traditional Perl the first line of the program is:
#!/usr/bin/perl
where the #! symbols are a command understood by UNIX. This command lets UNIX know where the Perl interpreter is to be used with this script. The single # symbol signifies a comment line in Perl. In Windows NT Perl this does not work. Windows NT does not recognize the #! command. To get around this you can rework any imported Perl scripts into batch files, or you can associate the file format of Perl files ".pl" to the Windows NT. Since most of the Perl you will come across in other books and on the Internet use the #! symbols in their first line, associating the .pl file format will be the suggested choice in this book. The batch method is explained later, in case batch files better fit your CGI needs.
To associate the .pl extension with perl.exe use File Manager:
Figure 1.1 : Associating the Perl file extension in Windows NT.
Many administrators have had serious security problems running Perl on their NT servers. The reason for this is the location of perl.exe on your system. Figure 1.2 shows the Perl home page security announcement that outlines the problem, and offers a solution.
Figure 1.2 : Urgent security announcement from the Perl language home page.
The following is an urgent security announcement as found on the Internet:
How'd you like to let anyone anywhere run any program they feel like on your system, even sending you new ones of their own devising? Sound frightening?
Well, that's what's going on out there.
Despite months of lobbying corporations, individuals, and the net at large about the perl.exe?FMH.pl problem, it continues to get worse. In the spirit of the Satan network checker, here's something that will help you find out whether you have the problem. It's called latro, a program that anyone can use to run any program they feel like on any system so unfortunate as to have ignored those warnings. If I hadn't written it, someone else would have.
You may argue that I've just given a lock-picking kit to the unwashed masses. Perhaps this is so, but far better that everyone should have the same resources at their disposal than that merely the thieves should have them. This way at least the locks might get fixed.
Already several people have posted to USENET about how one can use Alta Vista to find these sites. It's only a matter of time before these sites get, um, visited. Hopefully someone will construct a list of these and notify them. This is, of course, just a fraction of the vulnerable sites.
Let's clean it up out there, guys. Nefarious users could even ship over their own PC binaries and run them on your system, which means that if you aren't careful, they might do something useful like forcibly upgrade you to Linux. Of course, then the perl.exe?FMH.pl travesty magically goes away, along with a whole lot of other problems. :-)
This problem probably affects only amateur machines: those running Microsoft or Apple operating systems. UNIX and Plan9 systems should be largely unaffected, so at least most professional systems should be safe from this attack. But never underestimate the power of human stupidity when it comes to using technology they don't understand. There are also loads of sites out there with other interpreters than Perl in their cgi-bins, including shells, tcl, Python, and so forth. This has got to stop.
CERT has been notified of the issue, and will doubtless deal with it eventually.
For more information regarding this security problem please see Appendix E.
The reasons for choosing Perl as your CGI language are many. The first is its popularity. There are so many people using it that the number of Perl libraries is huge and growing every day. Perl routines are often included in many Web Server packages ready to use. Often new Internet applications include Perl gateway routines free of charge.
There are also numerous areas of support out there, as have been listed throughout this chapter (even more of which can be found in Appendix F, "Perl Sources").
These sources include Web sites where Perl creator Larry Wall (as well as many other key players in Perl's development) has been known to hold court.
Finally, Perl routines can be seen to be like mix and match modules (not to be confused with actual Perl modules, the collections of scripts). A Perl script can often be used in a new configuration with an intent never meant for the original script. These smaller scripts can stay quietly in the library waiting to be used for many other tasks.
While you may get caught up in singing the praises of Perl once you start using it, you should realize that it has limitations. Perl is ideal for doing what it does, because that's what it was designed to do. While various industrious individuals are constantly expanding its powers, Perl is still only a mid-level language that can't compete with a language like C or C++ when it comes to bigger applications. At least not yet.
This section details some of the limitations of Perl from a technical perspective, and may only make a lot of sense to those of you with some programming experience. These limitations are important to note, if only to have for future reference. (This information is taken from the Perl Home Page's "Seven Deadly Sins of Perl" article.)
While these disadvantages to Perl might seem over your head at the moment, their opaque meaning will come to those who get into Perl in a big way and then find out that there are limitations.
There is still quite a lot of documentation on Perl 4.0 and its many patch levels. Now that Perl 5.001 is in use, and 5.002 is not far behind, you might be asking whether you need both to run old Perl scripts. Perl is designed to incorporate all of its previous scripts in new releases, so anything you might have, or find, that was written for any patch level of Perl 4.0 will work with Perl 5.001.
Patching is what you sometimes do to earlier releases of software to update it to the current version. It is especially popular in shareware and freeware. As an NT user you don't need to worry about patching an old release of Perl because the new 5.001 version is ready and complete when you download it.
Already a version of Perl 5.002 is kicking around somewhere. Soon it will be available, and you can be sure that it will include even more functionality and fixes of old bugs. The huge ground support for Perl that exists already, and the communal attitude Perl developers and programmers share, ensures Perl's continued use well into the next decade. The time you take learning Perl, and continuing to develop that knowledge, will not go to waste, and will be useful for years to come.
Our introduction to Perl has covered quite a few things. We learned a little about how Perl came from the UNIX world into Windows NT, creating special translation concerns for NT users to be aware of. We also touched on how Perl is supported on all levels of development and programming with newsgroups and Web sites (more of which are mentioned through the book and in Appendix F), including the Windows NT port.
There is also the issue of why Perl is fundamentally a good CGI language because both Perl and CGI deal with text, a point that will be expanded on in the chapters to come. Now that you know where Perl came from and a little bit about what it is, you're ready to jump right into the language itself.