Platinum Edition Using HTML 4, XML, and Java 1.2
(Publisher: Macmillan Computer Publishing)
Author(s): Eric Ladd
ISBN: 078971759x
Publication Date: 11/01/98

The following, then, are the broad steps of the CGI process, simplified for clarity:

1.  Your browser shows the HTML document containing the form.

2.  You enter data into the form as needed and then click the Submit button.

3.  Optionally, a script in the browser performs client-side validation of the form’s contents.

4.  The browser decodes the URL and contacts the server.

5.  Your browser requests the document file from the server.

6.  The server translates the URL into a path and filename.

7.  The server “realizes” that the URL points to a program instead of a static file.

8.  The server prepares the environment and launches the script.

9.  The script executes and reads the environment variables and STDIN.

10.  The script sends the proper MIME headers to STDOUT for the forthcoming content.

11.  The script sends the rest of its output to STDOUT and terminates.

12.  The server notices that the script has finished and closes the connection to your browser.

13.  Your browser displays the output from the script.

It’s a bit more complicated than a normal HTML retrieval, but that’s essentially how CGI works. The scripts become extensions to the server’s repertoire of static files and open up the possibilities for real-time interactivity.

Where CGI Scripts Live

Like any other file on a server, CGI scripts have to live somewhere. Depending on your server, CGI scripts may have to live in one special directory. Other servers let you put scripts anywhere you want.

Typically—whether required by the server or not—Webmasters put all the scripts in one place. This directory is usually part of the Web server’s tree, often just one level beneath the Web server’s root. By far, the most common directory name is cgi-bin, a tradition started by the earliest servers that supported CGI. UNIX hacks will like the “bin” part, but because the files are rarely named *.bin and often aren’t in binary format anyway, the rest of the world rolls its eyes and shrugs. Today, servers usually enable you to specify the name of the directory and often support multiple CGI directories for multiple virtual servers (that is, one physical server that pretends to be many different ones, each with its own directory tree).

Suppose that your UNIX Web server is installed so that the fully qualified path name is /usr/bin/https/webroot. The cgi-bin directory would then be /usr/bin/https/webroot/cgi-bin. That’s where you, as Webmaster, put the files. From the Web server’s point of view, /usr/bin/https/webroot is the directory tree’s root. So if a file in that directory is named index.html, you’d refer to that file with an /index.html URL. A script called myscript.pl in the cgi-bin directory would be referred to as /cgi-bin/myscript.pl.

On a Windows or Windows NT server, much the same thing happens. The server might be installed in C:\Winnt35\System32\Https, with a server root of D:\Webroot. You’d refer to the file Default.htm in the server root as /Default.htm; never mind that its real location is D:\Webroot\Default.htm. If your CGI directory is D:\Webroot\Scripts, you’d refer to a script called Myscript.exe as /Scripts/Myscript.exe.

---

NOTE:  Although URL references always use forward slashes—even on Windows and Windows NT machines—file paths are separated by backslashes here. On a UNIX machine, both types of references use forward slashes.

---

For the sake of simplicity, assume that your server is configured to look for all CGI scripts in one spot and that you’ve named that spot cgi-bin off the server root. If your server isn’t configured that way, you might want to consider changing it. For one thing, in both UNIX and Windows NT, you can control the security better if all executables are in one place (by giving the server process execute privileges only in that directory). Also, with most servers, you can specify that scripts may run only if they’re found in the cgi-bin directory. This enables you to keep rogue users from executing anything they want from directories under their control.

CGI Server Requirements

CGI scripts, by their very nature, place an extra burden on the Web server. They’re separate programs, which means the server process must spawn a new task for every CGI script that’s executed. The server can’t just launch your program and then sit around waiting for the response; chances are good that others are asking for URLs in the meantime. So the new task must operate asynchronously, and the server has to monitor the task to see when it’s done.

•  The overhead of spawning a task and waiting for it to complete is usually minimal, but the task itself will use system resources—memory and disk—and also will consume processor time slices. A popular site can easily garner dozens of hits almost simultaneously. If the server tries to satisfy all of them, and each one takes up memory, disk, and processor time, you can quickly bog your server down so far that it becomes worthless.

•  In addition, consider the matter of file contention. Not only are the various processes (CGI scripts, the server itself, and whatever else you may be running) vying for processor time and memory, they may be trying to access the same files. A guestbook script, for example, may be displaying the guestbook to three browsers while updating it with the input from a fourth. (Nothing exists to keep the multiple scripts running from being the same script multiple times.) The mechanisms for ensuring a file is available—locking it while writing and releasing it when done—all take time: operating system time and simple computation time. Making a script foolproof this way also makes the script bigger and more complex, which means longer load times and longer execution times.

Does this mean you should shy away from running CGI scripts? Not at all. It just means you have to know your server’s capacity, plan your site, and monitor performance on an ongoing basis. No one can tell you to buy a certain amount of RAM or to allocate a specific amount of disk space. Those requirements will vary based on what server software you run, what CGI scripts you use, and what kind of traffic your server sees. However, following are some general rules for several operating systems that you can use as a starting point when planning your site.

Windows NT

The best present you can buy your Windows NT machine is more memory. Although a Windows NT Server runs with 12MB of RAM, it doesn’t run well until it has 16MB and doesn’t shine until it has 32MB–64MB. Adding RAM beyond 64MB probably won’t make much difference unless you’re running a few very hungry applications—for example, database applications such as Access or SQL Server. If you give your server 32MB of fast RAM, a generous swap file, and a fast disk, it can handle a dozen simultaneous CGI scripts without sweating or producing a noticeable delay in response. In most circumstances, it also helps to change Windows NT Server’s memory management optimization from the default Maximize Throughput for File Sharing to Balance. This tells Windows NT to keep fewer files in cache, so more RAM is immediately available for processes.