| Previous | Table of Contents | Next |
by Tim Parker
If you have an Internet connection, eventually you are going to want to access Usenet and its newsgroups. Usenet is one of the most dynamic (and often controversial) aspects of the Internet. Although you can set up, access, and work with newsgroups with access to the Internet, most Linux users are probably more interested in using Usenet specifi-cally.
Usenet was originally developed to facilitate the provision of discussion groups (called “newsgroups” in Usenet jargon). A newsgroup lets any user with access to the system participate in a public dialog with everyone else. Usenet is supported on millions of networks, in hundreds of countries, and reaches hundreds of millions of users.
Any machine that can attach itself to the Internet either directly, through a gateway, or through a forwarding service (such as an online service provider) can become part of Usenet. All that is required to use Usenet is the software that downloads and uploads the newsgroup mail and a reader package that lets users read and write articles.
The software that implements the passing of Usenet messages over local area networks from one machine to another is the Network News Transfer Protocol (NNTP). Using NNTP, your Linux machine can interact with any others that handle the news. NNTP software is an integral part of most Linux versions, so there is no additional software to purchase or look for.
There are two components to a news service for any Linux machine: actually getting the newsgroups to your machine is the responsibility of the transport software (usually CNews for UUCP connections or NNTP for TCP connection), then the newsreader assembles and presents the articles to the user. The original news system relied completely on UUCP, so much of the news software was designed for UUCP and then modified later to accommodate alternative methods. Since most of you will be using TCP/IP, we’ll concentrate on NNTP in this chapter.
To transfer news from one machine to another, a technique called flooding is used. One machine calls another and transfers all the news articles. The machine that just received the news calls another and transfers the articles again. The news articles flow across the networks in this manner, moving from machine to machine, instead of all the machines polling a single main news source. Each machine maintains a list of other sites that it can contact to transfer mail. Each connection to another machine is called a newsfeed.
Each machine can generate new articles as the system’s users interact with newsgroups. When new articles are created, the machine checks its list of newsfeeds and calls them to transfer the new mail. Since each article generated by a newsreader has a list of the machines that it has passed through (called the Path), the local machine knows whether the remote sites on its newsfeed list have already seen the article or not. As articles move from machine to machine, each machine adds its own identifier to the article’s Path field, using the UUCP bang-style notation.
A restriction may be placed on the machines that can be sent an article by an entry in the Distribution field of the header. For example, if you write an article that you want to stay within your local area network, you can specify this in the distribution field of the message when you write it. Then, when a newsfeed to a machine outside the local area network is created, the Distribution field prevents the article from being sent.
To help prevent duplicates of articles moving around Usenet, each article has its own unique identifying number, called a message ID (which sits in the Message-Id field in the article header). The message ID is a combination of a unique number and the name of the machine that the article was originally posted on.
These message ID numbers are used by each machine when a connection to a newsfeed is established. A file on each system called history contains a list of all article ID numbers that the local system has. When the two machines communicate with each other, they can check the history file to find out if the message should be sent. This is involved in a news transfer protocol called ihave/sendme.
With the ihave/sendme protocol, one machine sends a list of all the message ID numbers it currently has and waits for the other machine to identify the ones it wants. These are transferred one at a time in response to “sendme” messages. Then the process can be reversed to update the other machine. This type of protocol works well but does involve a lot of overhead in the communications process. For that reason (coupled with the generally slow lines used by UUCP modem links), ihave/sendme protocols are not often used when a very large newsgroup transfer is to take place at regular intervals. You wouldn’t want to use ihave/sendme to transfer 100MB of articles every day, for example, because it is so slow.
An alternative used for large transfers is batching of articles, wherein one machine simply sends everything it has to another. The receiving machine then performs a check of the newly arrived articles to see if it already has them. By comparing message ID numbers, the machine can discard duplicates. This tends to be a faster method for transferring, although it does have more processing overhead for the receiving machine when it crunches the newly arrived batch.
For network-based news access, there are three ways to get articles from another machine. Using NNTP, your machine can download articles you want using a technique similar to the ihave/sendme protocol (called pushing the news). Your machine can also request specific newsgroups or articles from the remote based on the date of arrival (called pulling the news). Alternatively, you can interact on an article-by-article basis with the remote, never downloading the articles to your local machine. This is called interactive newsreading and works only when you have a newsfeed you can log in to (which is common these days).
| Previous | Table of Contents | Next |