Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

paulm (5067)

paulm
  (email not shown publicly)
http://paulm.com/
Jabber: Tantrix@jabber.org
Hello! I'm Paul Makepeace [paulm.com], a regular on London.pm [pm.org]. Perl's my primary language I code in (since '95 I think); people even pay me [paulmakepeace.com] for it! Although I'd say I'm a developer I spend a substantial chunk of time being a sysadmin for my own machines (I'm also one of london.pm.org's root), all Debian boxen.

This journal is my first foray into a technical annotation of my perl life...

Journal of paulm (5067)

Tuesday June 08, 2004
07:32 PM

Stripping certain requests from log files

[ #19154 ]
I have this service that performs a HEAD request on a few sites I run, every few minutes. The problem is that it inflates my page hit count. Being one of those problems that is too annoying to do properly, the following one-liner (line broken for a shot at clarity) came out surprisingly easily,

for i in www.paulm.com-access.log.*gz; do
  gunzip $i; f=`echo $i | perl -pe 's/.gz//'`;
  perl -ni -e 'print unless /^ftp.itransact/' $f;
  gzip $f& echo $f;
done


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Stripping certain requests from log files 2 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • zgrep -v '^ftp.itransact'

    Of course once you add in a shell loop to iterate over the files and re-gzip them it starts to look like your original code :-)

    • Yeah, true. zgrep's pretty handy for scanning but not so much use for in-place mods, so far as I've seen. I'm sure it's possible with some of bash's more esoteric (for me!) constructs... But why bother when perl's there, heh. It's funny watching these little hacks have perl appear in there somewhere (f=`perl ...`), then as more functionality is needed, more of it gets shoe-horned into the perl invocation until *pop* it's all in perl and it gets re-written "properly":

      #!/bin/sh
      exec perl $0.pl

      :-)

Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.