Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

scrottie (4167)

scrottie
  scott@slowass.net
http://slowass.net/

My email address is scott@slowass.net. Spam me harder! *moan*

Journal of scrottie (4167)

Thursday August 16, 2007
04:42 AM

Ask.com emailed me asking to be unblocked from perldesignpat

[ #34128 ]

So, do I be a bastard and ignore them? Or do I unblock them?

Argument for ignore them: Shitloads of robots blatantly ignore robots.txt and cause vast amounts of trouble (usually wasted CPU and network resources). Ask in particular got stuck in a spammer tarpit that generates infinite numbers of bogus pages with bogus email addresses on them designed to clog the works for spambots. It was clearly flagged as off limits in the robots.txt but Ask choose to ignore that. I don't want to send the message that it's okay to be a bad netizen and then later redeem yourself by virtue of making money (off of other people's content, no less). As far as I'm concerned, they were a bad bot then and they're a bad bot now and they can rot in hell. Also, even though my email address is clearly listed on perldesignpatterns.com, for some reason this... woman had to contact my publisher. What the hell? If someone is published once, the only valid way to reach them is through their publisher? They're crackmonkeys and everyone should ban them. Also, I probably firewalled a bunch of IPs of theirs to enforce the robots.txt ban when the robots.txt ban didn't cut it and I got tired of seeing hits from them scroll by and I don't want to wade through and figure out which IPs to unblock. Hey, maybe that's why she had to talk to my publisher. Also, the tone was whiney and unapologetic... if she had inquired about they had done in the past to reassure me of a resolution, I'd have more sympathy, but requesting the same access their competitors have just doesn't fly. Truth of the matter is bots already outnumber humans 10:1 and GOOG drives ten times more traffic than any other bot. If I start letting other bots in, it'll be 100:1 before I can blink.

Argument for unblocking them: She emailed me. That's about it.

Crud. Fuggit. I have better things to do. I already wasted enough of my time banishing them in the first place. It's their own damn fault. They run disrespectful bots, getting firewalled and shitlisted and having access to less content on the 'net is the natural result.

-scott

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Yeah. Me and the devil on my shoulder are buds. He's got like exclusive privileges from his lobbying efforts or something. I know.

    -s
  • Hi

    Your company was automatically banned from the website because your automation ignored the published rules for interactively with the website, and behaved in a way that triggered an explicit test for spam robots.

    I'm happy to re-enable your access to the webiste, but I will need confirmation that the relevant behaviour in your spidering system.