Sams Teach Yourself Network Troubleshooting in 24 Hours:Hour 12 UNIX Networking Basics

Socket to Me

Sometimes, a mere stopping and starting of a daemon won’t help. In the case of inetd, you may be in a situation where the cure may be worse than the disease. In such cases, you’ll need finer diagnostics.

Enter netstat -a. This command is to network sockets what ps is to processes (remember from Hour 1, “The Telephone Analogy: Becoming Familiar with Basic Networking Concepts,” that a socket is like a phone extension that a service listens to for calls). netstat lists each socket that’s being used for a current connection or being listened to for a connection. It tells you whether things are backlogged, where they are backlogged from, and which socket is in use on both sides of the call. netstat -an will allow you to look at the numeric values only, which is valuable when you want to keep name services out of the picture. Let’s look at a specific example.

Suppose someone calls and tells you that he can’t get into FROTZ. FROTZ is the Financial Remuneration with Overwhelming Trillions of Zeros system. He’s very important to your place of employment, Frobozzco, so you’re alarmed when someone can’t get in. The user at the other end of the line is incoherent with rage and will not answer any of your questions.

You know by this person’s name that he’s in the Finance department. You can’t get an answer to the basic question “Are other people having problems?” Therefore, you decide to find out for yourself. You quickly check the network map and see that Finance lives on the subnet 200.1.1.0. In order to determine for yourself whether anybody else is having problems, you log in to the FROTZ UNIX host and type the following:

    netstat -a | grep 200.1.1

You’re rewarded with this:

   Proto send-q recv-q Local Address          Foreign Address    (State)
   tcp   0      0      frotz.frob.com.telnet  200.1.1.10.1673  ESTABLISHED
   tcp   0      0      frotz.frob.com.telnet  200.1.1.25.1975  ESTABLISHED
   tcp   0      0      frotz.frob.com.telnet  200.1.1.27.1772  ESTABLISHED
   tcp   0      0      frotz.frob.com.telnet  200.1.1.29.1968  ESTABLISHED
   tcp   0      0      frotz.frob.com.telnet  200.1.1.33.1492  ESTABLISHED
   tcp   0      0      frotz.frob.com.telnet  200.1.1.34.1444  ESTABLISHED
   tcp   0      0      frotz.frob.com.telnet  200.1.1.35.2855  ESTABLISHED

I’ve included the column headings for clarity; you wouldn’t actually get them when searching for an address. It becomes apparent very quickly that people from the 200.1.1 network are in, and they’re working just fine. (Actually, you could have used the who command, which, on most UNIX machines, tells you who is logged in and where they are logged in from. However, netstat will show you any service, not just Telnet.) You now know that people from 200.1.1.0 are logged into the system, and you suspect that the user’s problem is workstation related.

proto is the protocol. In most cases, it will be tcp or udp, TCP being the equivalent of a phone call (circuit oriented) and UDP being the equivalent of tossing notes back and forth to each other (connectionless). You can find more on UDP in Hour 15, “Firewall and Proxy Server Basics,” and Hour 18, “Lots of Different People in Your Neighborhood: In-Depth Application Troubleshooting.”

send-q and recv-q are representations of holding places for sending and receiving data in the host’s memory. You can think of them just the way you do a print queue; they hold stuff while waiting for processing. Unlike a print queue, they typically will be empty during normal operation. That is, these values will typically be 0 for local area networks, because local nets move pretty fast.

What if they’re not zero? Well, a changing “send queue” can mean that the other end is processing data but is keeping up somehow. This is usually a normal state for a LAN print server; it really is a print queue, so it processes some data, then catches up, gets some more data, and keeps going. If you see a nonchanging, non-zero send queue for one socket but not others, it usually means that something on the other end has stopped accepting data.

A non-zero receive queue can mean that something on the UNIX host itself is running out of resources, and it’s temporarily unable to process the incoming data. In practice, this is relatively rare.

The local address is, of course, the server you’re typing netstat on. In this case, because we’re discussing the Telnet service, the full address with extension is wefrotz. frobozz.com.telnet. Had we used netstat -an, it would have shown something like 192.168.55.10.23 (Telnet being socket 23). The foreign address is the other address—the client machine. The socket doesn’t matter as much here—just about any high-numbered socket that isn’t already in use can be used on the client side.

You can count the number of client/server sockets in use at any given time, say, for an imap mail server, by typing this:
netstat -a | grep imap | wc -l

You can find out which services your UNIX machine is offering to the world by typing this:
netstat -a | grep LISTEN

This will show you which services are listening for new connections. Because the service names are usually close to or exactly the same as the program names (http service/httpd program), you can easily figure out which program is responsible for a given service.

Table of Contents