Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

slanning (5049)

slanning
  (email not shown publicly)
http://search.cpan.org/~slanning/
Scott Lanning is currently working in Amsterdam at a hotel-booking company. The following interviews and commentaries are for entertainment only. The views and opinions expressed therein do not necessarily represent the views of his employer or even himself.

Journal of slanning (5049)

Wednesday February 20, 2008
10:27 AM

headless gecko

[ #35703 ]
Looks like someone is realizing one of my dreams with Crowbar:

Crowbar is a web scraping environment based on the use of a server-side headless mozilla-based browser.


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

headless gecko 4 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.

  • FYI, Javascript support for WWW::Mechanize has recently begun to emerge. You'd have to do some Googling and installation of experimental versions to use it; IIRC it uses a plugins mechanism for WWW::Mechanize which exists only in an experimental version. I believe it can use a Perl-based Javascript engine or Perl bindings for Mozilla's Javascript engine.

    Not sure if this is relevant to you or not, but maybe someone else will be glad to know. :)

    --
    J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
  • I'm not sure why this problem has not already been solved with the modules Win32::IE::Mechanize [cpan.org] (on Windows, uses the core engine of MS Internet Explorer) and its descendant Mozilla::Mechanize [cpan.org] (uses the gecko core). I believe that both support execution of embedded Javascript I know for sure that Win32::IE::Mechanize does.
    • Check out who the author of Moz::Mech is, and who the author of the post up above is...I'm sure he knows what the issues are. I'm guessing that M::M actually opens a browser window, and that he wants to not have to do that.
      • Ah, right. I only have experience with Win32::IE::Mechanize which does not open any windows, but not with Mozilla::Mechanize. Odd that nobody I've ever talked to about this module ever mentioned that fact.
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.