use Perl

• bogofilter (Score:1)

  by inkdroid (3294) on 2003.02.04 10:41 (#16691) Homepage Journal

  Yeah, I think this highlights why bayesian analysis is better than rule based approaches. It's harder to tweak the message so it slips by. Also, it's my understanding that bogofilter can scaled much better on a high volume MTA than SpamAssasin...but I don't know this for a fact so perhaps I should keep quiet.

  Reply to This

  • Re:bogofilter (Score:2)

    by Matts (1087)

    No it's true. SpamAssassin's rules are slow.
    
    On the flip side, bogofilter is a personal filter, so it's not going to perform that well on larger installations, which kind of breaks the point of being so much faster, doesn't it?
    • Re:bogofilter (Score:1)

      by inkdroid (3294)

      bogofilter can be run from procmail or invoked by your mta just like spamassasin. It just needs a nice corpus of spam/non-spam. What do you mean by being a "personal filter"?
      • Re:bogofilter (Score:2)

        by Matts (1087)

        Bayesian filters are personalised. They learn your ham/spam types. If you add in John-From-Marketing's ham then it massively decreases the effectiveness of the filter (because of the lists he subscribes to) for you. They don't stop working, but they're not as effective as SpamAssassin at large installations.
        
        See my talk on this at the spam conference.
        • Re:bogofilter (Score:1)

          by inkdroid (3294)

          URL?
          • Re:bogofilter (Score:3, Informative)

            by Matts (1087)

            http://axkit.org/docs/presentations/spam/ [axkit.org]
      • Re:bogofilter (Score:1)

        by chromatic (983)

        Being a Bayesian filter, it assumes there's a clear line between spam and non-spam. The more people you put in the same filter, the fuzzier that line may get. It's attempting to emulate your personal in-your-head filter.
        • Re:bogofilter (Score:1)

          by inkdroid (3294)

          Yeah, but everyone gets their own.
          • Re:bogofilter (Score:1)

            by chromatic (983)

            Hence the phrase "personal filter". :)
    • Re:bogofilter (Score:1)

      by godoy (2167)

      Speed is important when you're dealing with lots of messages.
      
      BTW, it is also important to reduce server load if you try implementing some automated way to handle spam.
      
      I've been thinking about writing something that will be run from an alias such as $USER-spam and $USER-ham (it will have to check the origin of the message: only the user himself can send messages to these addresses) and will classify the message as one of those using the user's database. Then, procmail or some other thing can compare messag

      --
      -- Godoy.
      • Re:bogofilter (Score:2)

        by Matts (1087)

        With SpamAssassin 2.50 (nearly ready) you can have per-user bayesian databases. But that doesn't scale to a company with (say) 20,000 users. You can't expect everyone to train their systems.