Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

acme (189)

acme
  (email not shown publicly)
http://www.astray.com/

Leon Brocard (aka acme) is an orange-loving Perl eurohacker with many varied contributions to the Perl community, including the GraphViz module on the CPAN. YAPC::Europe was all his fault. He is still looking for a Perl Monger group he can start which begins with the letter 'D'.

Journal of acme (189)

Wednesday August 03, 2005
02:22 PM

OSCON Day 3: Terabytes of Business Intelligence...

[ #26069 ]
I've stayed in the same room for Terabytes of Business Intelligence: Design and Administration of Very Large Data Warehouses on PostgreSQL by Josh Berkus and Joseph Conway. The first case study is on weblog analysis data with ad-hoc reports on one year of data, with large nightly ETL batch loads. It's all quite interesting, showing all the problems faced on the way to a final solution, ranging from server memory and disk allocation, query reoptimising and fiddling with kernel versions. To handle the adhoc querying, they produced aggregate tables at ETL time. One trick mentioned is that VACUUM is a big IO load so avoid it. The second case study was on equipment performance data, which was using NAS mounted using NFS (!) with jumbo frames. Interestingly, this used PostgreSQL's table inheritance which I haven't found useful in the past but maybe I should have another look. The rest was fairly routine data warehouse stuff, but with some notes on upcoming features such as BitmapScan and constraint elimination, which was interesting. Best quote: "... and you notice, there's no WHERE clause".
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.