Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Matts (1087)

Matts
  (email not shown publicly)
I work for MessageLabs [messagelabs.com] in Toronto, ON, Canada. I write spam filters, MTA software, high performance network software, string matching algorithms, and other cool stuff mostly in Perl and C.

Journal of Matts (1087)

Wednesday May 14, 2003
05:18 AM

C-R spam solutions

[ #12187 ]

It seems that Challenge-Response solutions to the spam problem are getting more popular, with lots of people using systems like ASK, TMDA and several commercial systems (including Earthlink now offering C/R to their home users).

Now I hate C/R systems. With a passion. I absolutely will not respond to them. They go in the trash. I don't get them very often but I get them more and more. I think they have the potential to seriously damage email communication as we know it. And I'm not alone in this opinion.

But, it occurred to me that maybe I'm being too strict. I'm thinking back to the early days of the web, when we also predicted that cookies had dire consequences, and were a really fundamentally bad idea, and would damage the web as we knew it. But they didn't really. I use cookies every day, and use them in the web sites I develop.

I think the above is wrong though. We're all familiar with C/R systems in mailing lists - you get a challenge when you subscribe to confirm that your email address is valid and you really wanted to subscribe. That confirms that you want to receive mails from the list. But with a C/R anti-spam system you're confirming that you want to send mails to somewhere, and that seems backwards to me. If I didn't want to send something there I wouldn't have, umm, sent something there, and I'd really rather not be questioned about it, thanks.

As a company we need to be able to deal appropriately with the rise in C/R systems, and be able to talk about them sensibly if we're going to dismiss them as a solution, so feel free to post any pros/cons to this entry. I have a list of my own somewhere but I'm interested in other people's opinions.


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

C-R spam solutions 15 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.

  • Like you, I encounter these periodically and I don't like them either. The particularly annoying scenario I encountered recently was that someone sent me email. I replied to their question and was then expected to leap through hoops to shepherd my reply through their challenge response system.

    • Re:Not a fan of C/R Email (Score:4, Interesting)

      by inkdroid (3294) on 2003.05.14 6:59 (#20095) Homepage Journal

      That's a poorly designed C/R system. The C/R systems that I've seen (and liked somewhat) will automatically whitelist an email address when an email is *sent* to that address. This means that people within an organization can send an email to whoever they want, and when that person responds it will go straight through.

      If email lists sent from the same address that you sign up at, then C/R systems that use the whitelisting technique above would have no problems receiving emails sent from legit email lists. The problem is that the list software I'm familiar with (listserv, ezmlm) have you sign up at different addresses than the list which email is sent from. I imagine this could be adjusted somehow though. If C/R systems were to catch on.

      I must be different because the first time I got a challenge from a C/R system I was intruiged and thought it was rather interesting. Presumably the challenge could also contain some question that it would be hard for spammers to automate an answer to. Perhaps if I understood more why people disliked the systems then I would understand more. As with implementations of SMTP, there are applications that do a sloppy job, and some that do a better job. The same will certainly be true for C/R systems as well.

      Matt, just to play devil's advocate here: are you against C/R systems because (if widely adopted) they would make spam filtering no longer a viable business niche? Heresy!

      • Re:Not a fan of C/R Email (Score:5, Insightful)

        by Matts (1087) on 2003.05.14 9:22 (#20108) Journal
        Haha!

        No, I'm against them because spammers lie. They forge the reply address to someone innocent. That means that the amount of spam I receive is going to double (as I count C/R requests as spam).

        I have a good mind to respond to C/R's that are the result of a joe-job against my domains just to prove this point.

        Mostly though I'm against them because I consider it impolite. A C/R anti-spam system is a cost shifting exercise. It shifts the recipients cost of spam to everyone who sends him email. That cost *should* be pushed to the spammers, not to legitimate users.

        I have a list of other problems, but it's a few pages ;-)

        • Just to take my devil's advocacy a bit further &evilgrin; When a spammer lies, using another persons email address as the From, then the content of their original email would not be delivered, thus reducing the incentive to send it in the first place. The few C/R systems I've interacted with did not include the content of the email I sent as part of the challenge.

          Sure, someone else would get the "challenge"...But if that person was also using a C/R system then they would never need to know...would the

          • Re:Not a fan of C/R Email (Score:4, Insightful)

            by vsergu (505) on 2003.05.14 10:05 (#20112) Journal
            The idea is for the spammer to use 'From' addresses taken from the same group the 'To' addresses came from (for example, they appeared on a web page together). Then there's a chance the addresses have been whitelisted previously and the spam will get through.

            And everyone not using C/R gets bombarded with ever-increasing numbers of challenges.

            C/R is a poorly thought-out attempt at solving the spam problem. It needs to be stopped before it makes the situation worse.
            • Oh, I hadn't thought of that. Duh. I guess one solution to that problem would be to only position C/R systems in between the Internet and an organizations Intranet: so internal emails would never go through it, and email posing as internal email with common username would be rejected. But you are right, this gets way to complicated way too fast, and if widely adopted would result in way too much complexity.
              • When I said the same group, I didn't necessarily mean the same domain. I'm not really talking about people in the same organization, just people who seem to be associated in some way and thus might be on each other's whitelists. Such people may show up on a web page together, for example.
            • You know what'd be an interesting twist - if you look at all the headers and MD5 checksum a concatonation of the server routes, from origin to destination and when you add the e-mail to a whitelist, you also associate it with the MD5 checksum of the traffic route. Any spammer forging the 'From' address won't be able to reproduce the route.

              Jason
          • So in order for C/R to work everyone has to implement it?
    • This is exactly why I don't like them either. As the other person said, if there was an automatic whitelisting of the e-mail that the person sent TO expecting a reply FROM that would then bypass the C/R I'd be okay with it.

      As it is, it tends to get in the way of communication, which is ostensibly what the internet is all about. The more difficult we make communication, the less of it is likely to happen. Basic principle of quantum laziness. :)
    • I have yet to reply to one, though so far the only ones I remember encountering have been when some moron has subscribed to a mailing list but neglected to whitelist it and then every time anyone posts they get the challenge (at least it hasn't gone to the list address). I really wish they could be nipped in the bud, but I fear the plague will spread.

      As far as I'm concerned, anyone who wants to offload their spam problem onto me isn't worth writing to. Handle your spam yourself, and don't bother your cor

  • Sender vs. Receivers (Score:5, Interesting)

    by ziggy (25) on 2003.05.14 9:02 (#20107) Journal
    The difference between C/R with mailing lists and C/R with sending email is quite different. As you point out, C/R with mailing lists works because you are opting in to receive messages, while poorly designed anti-spam C/R systems put barriers to sending messages.

    Sounds to me like we don't quite understand the problem fully. Or, rather, the current C/R anti-spam solutions are simple, obvious and totally wrong.

    To get the desired effect, a general purpose anti-spam system needs to work on the receiver's end, so sender-side C/R is broken as designed. I think that the issue C/R systems are trying to automate is whitelist management for the receiver. With that in mind, a good whitelist management tool would:

    • auto-whitelist any message where I send mail to the sender
    • understand that mailing list messages are fundementally different from individual messages
    • know that I'm tired of getting jokes that have been forwarded 14 times from my dad, but I still want to get email from him
    • filter out all mail from blacklisted addresses
    Add all that up, and any whitelist management tool is going to let spam through, but it can reduce the amount of irritating and useless (possibly non-spam) email I receive.

    I think the answer here is to quarantine messages from unknown senders. If I get more than, say, 5 messages from someone without responding to a single one (or automagically adding the sender to my whitelist), then all future messages from that sender should go into a "gray" folder, because chances are pretty good I don't want to read this message. That should accomplish the desired goal: automate maintenance of whitelists, blacklists and greylists.

    • I completely agree that C/R systems should treat mailing lists differently than regular email. Fortunately, almost all mailing lists set the header "to:" field to the name of the mailing list. A C/R system should not challenge a message if the header "to:" is not to the user of the C/R system. It should put these messages into a special bin and give the user an easy way to say that this "to" is a mailing list (and the messages should be allowed to pass through). This is what knowspam.net does (I am the
  • It appears that Challenge Response technology for blocking spam is patented technology [mailblocks.com] so only licensees of Mailblocks (Inc?) will be able to use it to annoy honest law abiding (non spamming) citizens. Thank goodness for that :-)
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.