Platinum Edition Using HTML 4, XML, and Java 1.2
(Publisher: Macmillan Computer Publishing)
Author(s): Eric Ladd
ISBN: 078971759x
Publication Date: 11/01/98

	One feature to look for in an indexing search engine is the capability to add words to the stop words list. On my site, for example, I would like to add the name of our company, ACNielsen, to the list. Because this word is mentioned in almost every file on the site, it doesn’t make sense to waste indexing space, nor do we need to enable users to search for this term.

Controlling Items to Index An important feature of any search engine is the capability to determine which files on your site to include in a search. If your site is like many, various directories are either password-protected or developmental in nature and not linked to the main pages. Certainly you don’t want files from these directories to turn up when users search your site.

Two approaches generally are used to control what material is indexed by the search engine. Either you specify all the directories you want to search, or you specify only those directories you want to exclude from searches. This latter approach usually results in less maintenance for the Webmaster. An even easier method is for the indexing engine to automatically skip directories protected with an access control file, such as the .htaccess file used by the NCSA Web server. This way you don’t have to remember to include or exclude new directories as they are added to your site.

As mentioned in the section “Using Keywords and Descriptions in HTML META Tags” earlier in this chapter, some indexing search engines enable you to use the <META> tag to control how a page is indexed. Using this tag, you place a page description and keywords in the heading of your documents. The search engine then gives this information special treatment when it performs its index.

Evaluating Search Engine Security Concerns

You must think about two main security concerns when implementing a search facility on your site:

•  Does the search engine itself represent a threat to site security?

•  Does the search engine allow users access to information that ordinarily they are prevented from seeing?