Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Just use Apache with Squid (or another balanceable proxy). mod_proxy's ProxyRemote directive lets you pick another proxy to use, and Squid can be load balanced quite well.
    • Can Squid store the files on disk in a structure that mirrors the master servers? Part of what I am looking for is a way to preload the cache with files from the CD or DVD. I also want to be able to sync parts of the mirror with rsync or wget.
      • by Juerd (1796) on 2007.06.04 3:18 (#55411) Homepage
        Why would you worry about the storage structure? As long as everything appears to be the same through http, does the physical layout on disk matter?

        No, Squid won't use a similar structure, but you can preload anyway, by just requesting the relevant URIs.

        If you're not loading from a normal internet site, though, you may have to set up a tiny http server for this purpose yourself. Or perhaps another proxy :)
        • Not really. Squid’s directory structure doesn’t look like what it has cached, but it’s pretty easy to hack some Perl to bulk-feed a directory structure plus given base URI into the Squid cache dirs.