NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.
All the Perl that's Practical to Extract and Report
about I/O disciplines and I should retire to the Tassili-in-Aijer
mountains in Sahara as a camel herder :-)
-- Jarkko Hietaniemi
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.
Splunk? (Score:1)
If you can afford it, splunk looks good.
http://www.splunk.com/ [splunk.com]
If not, I like
http://search.cpan.org/dist/Log-Log4perl-Appender-Spread/ [cpan.org]
for log collecting and then use the hashed spread groups to insert into multiple MyISAM tables in parallel, or in multiple servers.
If using a single server, a merge table on top gives you a nice global view for quick and dirty stuff. For more complex processing, you can use a simple map-reduce-style operation where you query your multiple mysql log databases in parallel and reduce their results into the final one.
Best regards,
life is short
Reply to This