Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Friday November 28, 2003
08:14 PM

You just do what you have to do

[ #16066 ]

For several years I have been meaning to write a program to parse the output of tcpdump, which I like ot use to figure out how certain web transactions work (so I can automate them). A lot of other people have written these things too, but sometimes I just have to write it myself. Sometimes life is like that.

Up until now, I simply dealt with the hex dumps, which do not really bother me as much as the thought of reading a hex dump when it is so easy not to. In any case, I am use to reading these sorts of things because I use them to do things like figure out the iTunes Music Library format, and a lot of my time in academia was spent looking at binary formats (because everyone has their own).

For some reason, this output worries people. They get vary pained expressions on their faces, unless, like my cats, they just look at me as if to say "So when's lunch?"

sudo /usr/sbin/tcpdump -lX -i en1 -s 512 dst port 80 or src port 80
 
18:50:11.671997 10.0.1.4.50692 > useperl.org.http: P 1:619(618) ack 1 win 33984 <nop,nop,timestamp 332830 3298546934> (DF)
0x0000   4500 029e 67fe 4000 4006 8885 0a00 0104        E...g.@.@.......
0x0010   4223 faaf c604 0050 8f9a e018 8df5 2ab5        B#.....P......*.
0x0020   8018 84c0 d401 0000 0101 080a 0005 141e        ................
0x0030   c49b d4f6 4745 5420 2f20 4854 5450 2f31        ....GET./.HTTP/1
0x0040   2e31 0d0a 486f 7374 3a20 7573 652e 7065        .1..Host:.use.pe
0x0050   726c 2e6f 7267 0d0a 5573 6572 2d41 6765        rl.org..User-Age
0x0060   6e74 3a20 4d6f 7a69 6c6c 612f 352e 3020        nt:.Mozilla/5.0.
0x0070   284d 6163 696e 746f 7368 3b20 553b 2050        (Macintosh;.U;.P
0x0080   5043 204d 6163 204f 5320 5820 4d61 6368        PC.Mac.OS.X.Mach
0x0090   2d4f 3b20 656e 2d55 533b 2072 763a 312e        -O;.en-US;.rv:1.
0x00a0   302e 3129 2047 6563 6b6f 2f32 3030 3330        0.1).Gecko/20030
0x00b0   3330 3620 4361 6d69 6e6f 2f30 2e37 0d0a        306.Camino/0.7..

That is not so tough to read, but I wanted to use Perl to make it easier. First, I need to get rid of the (normal) human readable stuff. I just change the big X to a small x in the tcpdump switches. I could just grab the converted characters from the big X output, but then I have to jump through a lot of hoops to figure out where the newlines go.

This output, which is less busy, troubles people a bit more. I guess some people skipped out on that class on ASCII while in high school (or whatever form or level or whatever the rest of the world calls it).

sudo /usr/sbin/tcpdump -lx -i en1 -s 512 dst port 80 or src port 80
 
18:55:29.808364 10.0.1.4.50695 > useperl.org.http: P 1:619(618) ack 1 win 33984 <nop,nop,timestamp 333467 3298578717> (DF)
                         4500 029e 6b5a 4000 4006 8529 0a00 0104
                         4223 faaf c607 0050 16ba 2fc2 a1e6 e685
                         8018 84c0 aecf 0000 0101 080a 0005 169b
                         c49c 511d 4745 5420 2f20 4854 5450 2f31
                         2e31 0d0a 486f 7374 3a20 7573 652e 7065
                         726c 2e6f 7267 0d0a 5573 6572 2d41 6765
                         6e74 3a20 4d6f 7a69 6c6c 612f 352e 3020
                         284d 6163 696e 746f 7368 3b20 553b 2050
                         5043 204d 6163 204f 5320 5820 4d61 6368
                         2d4f 3b20 656e 2d55 533b 2072 763a 312e
                         302e 3129 2047 6563 6b6f 2f32 3030 3330
                         3330 3620 4361 6d69 6e6f 2f30 2e37 0d0a

After that, I only need to convert all of that back to the human readable characters. Easy-peasy. When SourceForge updates their CVS servers, you can see the source for my httpsniffer program, whose output is a bit more friendly. People still look at me funny, but maybe I should get them HTTP: The Definitive Guide for the holidays, just as a gentle nudge that they really should know how the world works, you know?

Time: 18:13:56 Src: 10.0.1.4.51750 Dst: service.cddb.com.http
POST http://service.cddb.com//service/2.0/product/1 HTTP/1.0
Host: service.cddb.com
Accept: application/octet-stream
User-Agent: SL CDDB 2.0
Content-Type: application/octet-stream
Content-Length: 573
-------------------------------------------------------------------------
    Time: 18:13:57 Src: service.cddb.com.http Dst: 10.0.1.4.51750
HTTP/1.1 200 OK
Date: Sat, 29 Nov 2003 00:13:58 GMT
Server: KZSM/1.0
Content-length: 1205
Connection: close
Content-type: application/octet-stream

And Bob's your uncle.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • It understands a good bit of many different protocols and can dump out something similar. It can also filter based on their contents (i.e. I just want to see HTTP traffic that's POST requests ...).

    Ethereal.com [ethereal.com]
    • I saw that, and I wanted to install it on my FreeBSD machine, but my ports is out of date, so the ethereal version it knows about is gone. Not only that, but my version of cvsup no longer works, so I can't update ports until I update that.

      Of course, these things really are not all that hard, but my ability to remember how to do all this stuff has greatly suffered this year. Heck, it took me 15 minutes last week to figure out where the Trash was on Mac OS X :)

      Ethereal does look very cool though.