Stories
Slash Boxes
Comments

NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

use Perl

All the Perl that's Practical to Extract and Report

use Perl Log In

[ Create a new account ]

TeeJay (2309)

TeeJay
(email not shown publicly)
http://www.aarontrevena.co.uk/

Working in Truro
Graduate with BSc (Hons) in Computer Systems and Networks
pm : london.pm, bath.pm, devoncornwall.pm
lug : Devon & Cornwall LUG
CPAN : TEEJAY [cpan.org]
irc : TeeJay
skype : hashbangperl
livejournal : hashbangperl [livejournal.com]
flickr :hashbangperl [flickr.com]

Journal of TeeJay (2309)

Monday October 04, 2004

04:36 AM

Just write a simple reverse index

[ #21176 ]

Heard any good strategies for super-fast catalog-database search results?

Yes - just write a simple or weighted reverse index. Its trivial in any relational database and works accross tables and can be weighted by key columns or object attributes.

Don't use native RDBMS or SQL like unless you want your site to be as unsearchable as the millions of others out there.

Which reminds me - perl.com, and many perl sites (including search.cpan and perldoc.com) still have sucky searches - who else is interested in getting together and trying to improve them?

I am planning to port my current searchable class to Class::DBI, which might help. I will probably look at a module to provide a similar functionality in a procedural way to non-OO code.

List all Journal entries

Just write a simple reverse index More | Login | Reply

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Without JavaScript enabled, you might want to use the classic discussion system instead. If you login, you can remember this preference.

Perl.com (Score:1)

by chromatic (983)

I'm interested in improvements to Perl.com; I'm exploring ways to improve the searching and findability of articles across the entire O'Reilly Network. Please feel free to send me ideas.
- Re:Perl.com (Score:2)
  
  by TeeJay (2309)
  
  see my response on the blog - I couldn't even find my article on searching through perl.com, found it straight away with google.
  It would be nice if perl.com could showcase good perl solutions.
  If I knew what perl.com was using I would be happy to make suggestions.
  
  --
  
  @JAPH = qw(Hacker Perl Another Just);
  print reverse @JAPH;
Maybe these can help (Score:1)

by melo (332)

http://search.cpan.org/~tjmather/DBIx-FullTextSearch-0.73/
http://search.cpan.o rg/~tmtm/Class-DBI-mysql-FullTextSearch-0.09/
http://search.cpan.org/~snowhare/ Search-InvertedIndex-1.14/

I used the last one with good results.
--

--
life is short
- Re:Maybe these can help (Score:2)
  
  by TeeJay (2309)
  
  Yup the last one is a reverse index. They work nicely and are trivial to write and customise.
  I wrote a handy class that implements cross-object context specific searching - Class-Indexed-0.01 [cpan.org]
  Its based on the article I wrote for perl.com
  The others aren't a solution - native RDBMS fulltext searching is only useful for really trivial cases - if your data is only in a single table and has a very simple structure so you are only interested in 1 or 2 columns and they don't change - move beyond that and nati
  
  --
  
  @JAPH = qw(Hacker Perl Another Just);
  print reverse @JAPH;
  - Re:Maybe these can help (Score:1)
    
    by perrin (4270)
    
    It depends on how the database implements the full text search. Often it is just a reverse index under the covers, with some stemming and phrase support bolted on.
    I agree that a reverse index is typically the way to go for custom searching. It scales well and is very customizable.
    - Re:Maybe these can help (Score:2)
      
      by TeeJay (2309)
      
      I have yet to see an RDBMS native fulltext search that is as good as a handwritten one - all are held back by a total lack of customisation or weighting and can only apply to a single table. Very few provide features like stemming although they can provide some simple phrase matching and global weightings (i.e. common words may score less, etc).
      As I said, even a good native RDBMS can be improved on trivially with a competent custom reverse index, as stemming, etc are trivial in perl. Native RDBMS are only
      
      --
      
      @JAPH = qw(Hacker Perl Another Just);
      print reverse @JAPH;