Network Grep

You might want to inspect your network at a very detailed level for a number of legitimate reasons. Much of the time, it’s to debug an application that’s misbehaving and connecting to a server on the wrong port, or it might be that a colleague has noticed a slowdown on a particular network link, and you need to diagnose where the sudden flood of multidirectional traffic is coming from.

On the other hand, you might need to check the exact nature of an attack to perform some real-time forensic diagnostics to circumvent it. Leaving the networking aspects entirely aside for a moment, even an admin solely responsible for systems and not networks (an exceptionally rare remit these days, admittedly) needs a highly functional packet sniffer available at all times. Because we rely so heavily on connectivity for our multifaceted Internet usage, it’s imperative that an admin can inspect the contents of their network deeply and interpret the results proficiently.

The all-pervasive networking tool tcpdump is undoubtedly still the champion of packet sniffers, but for certain scenarios, I much prefer an equally lightweight package called ngrep , sometimes called simply network   grep . As its name suggests, ngrep does for networks what grep does for files; it’s a highly functional network pattern-matching tool that helps the user sort the wheat from the chaff, and on a busy network, you need a great deal of assistance to determine what the seemingly endless splurge of characters quickly running up the screen actually mean.

What’s the Difference?

When I first started looking at networks in any great detail, I was initially attracted to ngrep because its command structure seemed to be in plain English. It uses words unlike this tcpdump example, which doesn’t exactly make sense at first glance:

# tcpdump -vv -i eth1 'tcp[13] & 2 = 2'
Matching either SYN only or SYN-ACK datagrams

Instead, these looked more like a demonstration of why I should have listened to my mathematics teacher properly when I was still in school. If you’re familiar with regular expressions, then you’ll know one of the aspects that made tcpdump so popular was its flexibility. On the other hand, ngrep follows the same path but appears to offer more of a grep-style filtering, which, having used grep frequently, to my mind at least feels more intuitive to use. However, you don’t need to be strictly purist, and using both tcpdump and ngrep can provide a great deal of invaluable functionality.

Words and Numbers

To begin, I’ll look at some simplistic filtering rules that make ngrep so attractive. To access a network interface fully, you will need elevated privileges (e.g., su or sudo   -s ) to fully achieve that status before running the examples below. For those of you less concerned with repetitive strain injury, simply prefix sudo to your command lines.

If you’re concerned about email traffic and need to watch all TCP traffic closely using the SMTP port, then you could construct a command line such as:

# ngrep -d any port 25

Here, the SMTP example shows that (in more recent libpcap library versions, at least) you can ask ngrep to listen on all the available interfaces at once; otherwise, you might just specify -d   DEV or, for example, -d   eth1 instead to specify a particular network interface.

Now I’ll expand on that first command a little and add more switches to the example. By omitting the -d   any parameter, the ever-trusty ngrep will assume a default interface, usually eth0 . Just append it as above if the examples that follow aren’t what you need.

You can drill down into any HTTP traffic on your network link by mentioning port 80. Additionally, you can isolate one sender IP address that is sending the port 80 traffic. Notice the src   host syntax:

# ngrep port 80 and src host 12.34.56.78

Moving on from a single IP address, imagine that you have so much data from that single IP address that you want to refine it even further and this time specify a destination address, too. In this case, your example would look like this:

# ngrep port 80 and src host 12.34.56.78 and dst host 98.76.54.32

The dst   host appendage followed by the destination IP address is, I hope, self-explanatory. If you see fit, you can then easily interchange the host element with net ; if you use the CIDR format, your command line might then look like this instead:

# ngrep port 80 and src net 12.34.56.0/24 and dst net 98.76.54.32/27

Master Class

By now, I hope you can see how it’s possible to wade through even the heaviest floods of network traffic and still discern what’s going on and from where. One of the more granular functions of ngrep is its ability to pick out certain pieces of information quickly from the deluge of data that’s streaming up the screen. For unencrypted logins, this works a treat. I sincerely hope it’s only in a LAN environment that you are still using Telnet, but if you need to hunt down the login prompt to a Telnet server, you can use this:

# ngrep -t -wi "login" port 23

Running this command spawns ngrep under the default network interface and offers the following information in addition underneath:

filter: (ip or ip6) and ( port 23 )
match: ((^login\W)|(\Wlogin$)|(\Wlogin\W))

Here, ngrep is saying it will listen for both IPv4 and IPv6 traffic on port 23 for Telnet. The match is the pattern for which ngrep is searching. The -w switch tells ngrep to match the regular expression (login , in this case), and the -i switch means “ignore case sensitivity” on that regular expression. If you’re stopping a steady flow of traffic shooting up your screen with Ctrl+C, then it’s useful to have a time reference when you’re scrolling back through the data, and that’s exactly what the -t parameter should do, with timestamps for each match in the form: YYYY/MM/DD HH:MM:SS.UUUUUU.

Flick a Switch

Before I look at more examples, I’ll take a breather and look a little at the some of the other available switches that ngrep supports.

If you’re keeping a keen eye on all network traffic, you might even have the need to look at empty packets, which are usually discarded because they have no actual payload through which to search. By adding -e to the command, then despite the added regular expression, you can still catch empty packets on the network, which could be of a malicious nature.

Conveniently, in the same vein as the stalwart grep , you could simply add -v to reverse the filter to see packets that don’t match the prescribed pattern.

I mentioned using tcpdump in hand with ngrep, and the -l option works nicely for this. If you’ve captured and saved a large dump of network data to a file with tcpdump, then you can run ngrep over the top of that data file and use its simple, yet powerful, searching functionality to do so. Using the example from above, you can search for Telnet logins from within a pre-saved tcpdump dump file:

# ngrep -wi "login" port 23 -I <filename>

By enabling -X , you can inform ngrep that you’re looking for a hexadecimal pattern, and not plaintext, which is useful for more advanced searching.

Finally, how about dumping directly from ngrep onto a text file of your choice? It’s a simple maneuver and involves the -O parameter. The nice thing about this feature is that you can still see all of the required data on your screen and still store it in a pcap-compatible data file for later (the highly portable libpcap library format).

Lead by Example

Now we can gather our newly found knowledge and apply some of these switches to what will hopefully prove to be useful examples. Some of these are available in more detail on the ngrep website, if you get stuck or are curious, but I’ll cover a few others, too.

Returning to the Telnet login example above, think about an unencrypted and cleartext FTP login sequence and how you might go about pattern matching such a session taking place on your network:

# ngrep -wi -t -d eth0 'user|pass' port 21

The FTP login session capture is frighteningly simple, as is the Telnet login capture, and highlights precisely why everything for which you can justify a little extra complexity is encrypted on networks these days.

Apparently ngrep can also delve into the payloads of packets using regular expressions. I won’t go too far into the details of the next regular expression, but it’s an example of how you might study a pre-recorded network dump file:

# ngrep -t -O <filename> '~.*(\*|\[[^]]*)'

To spot an HTTP attack that involves endless HTTP POST commands, you add a caret in front of the regular expression, which instructs ngrep to look only for POST s at the beginning of the payload associated with the packet.

ngrep -t '^(POST) ' 'dst port 80'

Well Refined

One of the most important features of ngrep is its ability to sort the wheat from the chaff, as I mentioned previously. If you looked at raw port 80 traffic, you would see lots of useful information, as well as lots of potentially useless information that doesn’t help you decipher what’s travelling across your network link. This HTTP sniffing example is going to be noisy in terms of output,

# ngrep port 80

whereas this example, which uses the clever byline functionality, helps boil down the screeds of information efficiently:

# ngrep -W byline port 80

The byline function is the epitome of simplicity and wraps text when a new line is spotted, making those raw HTTP packets significantly easier to read with the human eye. It differentiates the packet headers and their associated payload nicely, too.

Reaction Time

On my travels, I once came across a useful tool called tcpkill . In the past, I have used it on a Linux router to drop specific connections between hosts that are unnecessary or malicious. It might surprise you to know that ngrep offers exactly that functionality too; that is, it lets you capture and disconnect certain network traffic, disrupting the TCP flow between hosts by sending a set number of RSTs. In this case, the ngrep manual offers the following entry for the -K parameter and mentions tcpkill as well:

-K num Kill matching TCP connections (like tcpkill). The numeric argument controls how many RST segments are sent.

The Beginning of the End

This little insight might tempt you to turn to ngrep the next time you’re looking for something on your networks. The clarity of its output and its minuscule installation footprint make it an indispensable tool. I haven’t gone into any detail relating to ngrep’s formatting of binary (hexadecimal) traffic, but it’s certainly impressive and, again, uses a familiar grep structure. Combined with its other functionality, ngrep is undoubtedly a force to be reckoned with.