use Perl

• Alternative to directory structure (Score:2)

  by mirod (2686)

  We really need an alternative to complement the directory structure. Something like a light database that indexes files on their type, date and other keywords (I am not even talking about indexing the content here), so you can just select the files you want to list. I am really tired of looking all over the system for files when I know what I want but not where it is.

  Of course the problem is to generate as many of the keywords automagically, with the user being able to add more in a simple way... quite a t
  • Re:Alternative to directory structure (Score:3, Insightful)

    by Matts (1087)

    The problem is speed. Ever worked up to or past midnight (or whenever your slocate cron job runs)? The system slows to a crawl as it thrashes all over your filesystem. Now imagine that also doing full text indexing? Yikes.
    
    Plus I think there are a lot more questions with complex filesystems than there are answers, for instance the problem with sending a file from such a filesystem via email.
    
    Personally I'd rather keep filesystems simple, and work out funkier ways to actually work over the top of those. Mayb
    • Re:Alternative to directory structure (Score:2)

      by mirod (2686) on 2002.03.14 7:15 (#5887)

      The problem is speed

      I realize this, plus I am not so sure full text indexing is necessary. Not if you can narrow down the list of files to grep by date, type, and maybe header fields for emails.

      I guess it's buzzword time ;--) What we need first is probably ontology to tell us how to describe files better than just through a list of names (directories and file name) and a 3 letter extension (or a Mime type, or neither). Then we can start having fun implementing all sorts of funky file systems, or layers on top of the file system (DBD::SQLite works just fine BTW, I love not having to worry about rights management when I write the prototype of a product).

      Reply to This

      Parent

      • Re:Alternative to directory structure (Score:4, Interesting)

        by ziggy (25) on 2002.03.14 9:26 (#5894) Journal

        The problem is speed

        I realize this, plus I am not so sure full text indexing is necessary.

        I read an article on O'Reillynet some time back about a user who was migrating to OS X (it didn't jump out at me looking at the index [oreillynet.com]). One of the really nasty things about dumb full-text indexing is full-text indexing every damn file -- including MP3s. :-)

        Taking the BFS approach, full-text indexing is within reach, once you intelligently focus on text files. After all, what's the point of indexing executables, PDFs or PNGs? The heuristics to reduce the number of files to be indexed are quite simple. After all how many files do you touch on a daily basis that are text files? And how many of them actually change in any given day?

        Also, the speed cost at 2am for running slocate or mklocatedb is partially because it is dumb, and partially because it does everything. Amortizing the cost over the course of the day such that only active files are checked shouldn't impose such a huge penalty.

        And, of course, DBD::SQLite is likely part of the solution to the problem. The BFS idea of making the filesystem into a database sounds nice, but I don't think the *NIX world is quite ready for that just yet. :-)

        Reply to This

        Parent

        • Re:Alternative to directory structure (Score:1)

          by pudge (1)

          MP3s, JPGs, and PNGs have comments in them. Might you want those indexed?
          • Re:Alternative to directory structure (Score:2)

            by ziggy (25)

            The comments indexed? Sure.

            Full-text indexing the rest of the binary data structures in those file formats is not likely to turn up any other hidden messages though. :-)
      • Re:Alternative to directory structure (Score:2)

        by darobin (1316)

        I guess it's buzzword time

        Man, we have it, we have it!!! I mean throw away those old clunky ideas of a Semantic Web, what we want is a Semantic File System, YAY!!! Lets get a startup rolling, lets become billionaires :-)

        Seriously though, I'm not sure what but this strikes me as a good idea. Maybe it's those mushrooms from last night...

        --
        

        -- Robin Berjon [berjon.com]
      • Re:Alternative to directory structure (Score:1)

        by belg4mit (967)

        Funny how nmh has pick for that :-D
        
        alias pscan "scan `pick !:*`"
        pscan --from Bob
        pscan --subject Perl
        
        ...

        --
        Were that I say, pancakes?
        • Re:Alternative to directory structure (Score:1)

          by belg4mit (967)

          Of course mgrep might make a more
          intuitive name...?

          --
          Were that I say, pancakes?