Stories
Slash Boxes
Comments

NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

use Perl

All the Perl that's Practical to Extract and Report

use Perl Log In

[ Create a new account ]

acme (189)

acme
(email not shown publicly)
http://www.astray.com/

Leon Brocard (aka acme) is an orange-loving Perl eurohacker with many varied contributions to the Perl community, including the GraphViz module on the CPAN. YAPC::Europe was all his fault. He is still looking for a Perl Monger group he can start which begins with the letter 'D'.

Journal of acme (189)

Friday January 14, 2005

06:25 AM

Fast, light-weight databases

[ #22711 ]

I like general case solutions, like Plucene and DBM::Deep but sometimes you just have to get slightly lower level and do it all yourself. Why, you ask? For speed of course. I've just redone my recipe website, which has a lot of recipes, is used a lot (mostly by American housewives), and is hosted on a bitty box.

Enter CDB_File, a "Perl extension for access to CDB databases". Now, ignoring the fact that CDB is djbware, it rocks. The files are small, and they are very fast to read. Thus I do lots of processing and indexing on my build scripts and then suddenly you can search at lightning speed for orange wine, and find Orange-wine wafers. Note the related recipes on the right, generated with Search::ContextGraph.

Mmmmm, food and Perl. Happy birthday richardc!

List all Journal entries

Fast, light-weight databases More | Login | Reply

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Without JavaScript enabled, you might want to use the classic discussion system instead. If you login, you can remember this preference.

Recipes and American housewives (Score:1)

by osfameron (3135)

In one of the early novels, Michael Dibdin notes Aurelio Zen's dismay at the idea of using a cookery book. Italians cook the food they learnt in their family/regional tradition, and for the Venetian Zen it would be as odd to look in a cookery book for an Umbrian speciality as it would a Chinese, or Martian one. (Contrasted with his American girlfriend who cooks a baffling variety of recipes from across the world).

--
osfameron
Back to CDB? (Score:2)

by Matts (1087)

I recall we had a conversation about 2 years ago about how you decided not to use CDB files for some reason. Why the change of heart?
- Re:Back to CDB? (Score:2)
  
  by acme (189)
  
  The reason at the time was that CDB_File was leaking memory, so I switched to BerkeleyDB. However, I checked out CDB_File again and lo and behold it doesn't leak, it does what it says on the tin, and it's nice and fast. After doing some more performance tests, one of the nice things about CDB over BDB is that my CDB files are 2.5x smaller.
BerkeleyDB (Score:1)

by perrin (4270)

It probably doesn't matter for your recipe site, but BerkeleyDB is typically faster, if you use the BTree mode. Even faster if you use method calls instead of TIE. There's a benchmark you can play with here [perlmonks.org]. Make sure to look for the update at the bottom of the thread.
- Re:BerkeleyDB (Score:2)
  
  by acme (189)
  
  And lo did I do some more benchmarking. Bear in mind that my data has numeric keys and largish objects serialised via Storable. 'deep' is DBM::Deep. 'bdb' is BerkeleyDB::Hash. 'bdb_btree' is BerkeleyDB::Btree with "pack "N" as a key filter and 'cdb' is 'CDB_File'. First time I run the benchmark:
  
  Rate deep bdb bdb_btree cdb deep 0.810/s --
  - Re:BerkeleyDB (Score:1)
    
    by perrin (4270)
    
    I feel a bit silly trying to optimize a recipe search, but... I suspect your use BerkeleyDB is not optimal. Take a look at this benchmark code [fastmail.fm] for an example of high-speed use of BerkeleyDB.