Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Within 90 minutes, and Google have already come sniffing for branes:

    66.249.66.14 - - [03/May/2007:11:05:20 +0100] "GET /~nick/branes.cgi HTTP/1.1" 200 1761 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

    Is there any particular reason that Google spiders use.perl rapidly? Is it partly because use.perl itself has a high pagerank?

    • Pages that update often get spidered often. That sucks, by the way, when your page has randomized elements.
    • Yeah, but more importantly, what did googlebot score? :-)

      I got 13.

      -Dom

      • The POST form put it off, and it didn't go beyond the first page. Either that, or it only likes Python flavoured branes.

        Does Googlebot follow meta HTTP-EQUIV=refresh in pages? If so, maybe I should put one in the introduction page too, to lure it all the way to the end.

    • It could just be coincidence, couldn't it? Have you checked how often the bot comes back?