All the Perl that's Practical to Extract and Report
use Perl Log In
davorg (18)
dave@dave.org.uk
http://dave.org.uk/
Yahoo! ID: daveorguk (Add User, Send Message)
Hacker, author, trainer
Technorati Profile [technorati.com]
Introducing Perlanet
A couple of years ago I started building feed aggregation sites (aka "planets"[1]) using Plagger. Once you've installed Plagger, it's pretty simple to configure new planets.
Notice I say "once you've installed Plagger". Plagger is one of those CPAN modules which installs about half of CPAN. The CPAN dependencies page currently gives you a 26% chance of successfully installing Plagger.
The reason for that is clear. Plagger is an incredibly powerful tool. It is an all-purpose tool for slicing and dicing web feeds. It also includes special-case plugins for creating web feeds for many sources that don't currently publish them.
So the net result is that Plagger is large and often hard to install. And most people (or, at least, I) don't use a fraction of its power. I wish I didn't have to install all of Plagger's dependencies in order to just grab a few feeds and combine them into a planet site. Perhaps the Plagger team needs to look at breaking up the distribution into smaller parts that do less.
[Update: Miyagawa points out that the vast majority of Plagger's dependencies are optional, so I'm overstating the case here. Sorry about that.]
Recently I moved a lot of my sites to a new server. And I really didn't want to go through the pain of installing Plagger. Therefore all of my planets have been dead for a couple of months. And that has been bothering me. But a couple of days ago (prompted by something that Paul Mison wrote) I decided to do something about it.
I a few hours of spare time I wrote Perlanet[2]. It doesn't do anything at all complex. It reads data from a YAML file, uses XML::Feed to parse the feeds and then the Template Toolkit to generate the web page (oh, and XML::Feed again to generate the combined feed).
It's very simple. And it's new, so there may well be bugs. But it's there if you find it useful. It's already powering a revived planet davorg. My other planets should come back to life over the next few days.
I'm sure I'll be adding more features over the coming weeks, but the main point is to keep it simple.
[1] After Planet the Python software that (as far as I know) first popularised this method of aggregating data.
[2] Yes, I know it's a terrible name. But it seemed obvious to me and now I can't shake it.
--Larry Wall in stab.c from the perl source code
Thanks, links question (Score:1)
I have a question about the del.icio.us integration. In one case several links were aggregated into one post. In other cases, which link got it's own post.
What's the difference? Or, why do it both ways?
Re: (Score:2)
I've been using the delicious "daily blog posting" tool to wrap up a day's links and add them as a single entry to my blog. But the planet includes both the blog feed and the raw delicious feed. So delicious links appear twice on the planet.
I'll going to turn off the delicious daily posting tool.
Thank you (Score:2)
I hope to be able to use this someday. I've been needing such a tool but on a lower scale than plagger. Unfortunately I'm so busy with $NEW_JOB I'm not sure when I'll get to try it out. :)
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Validation (Score:1)
There are, however, two RT tickets (#33881 [cpan.org], adding accessors for some required feed types, and #29684 [cpan.org], which stops 'convert' generating empty summaries) which will help you get towards a valid feed.
I really should write up the other changes I had to make to genenerate a valid Atom feed [feedvalidator.org]. I think convert needs help with applying an updated date to RSS feeds, for example. There are also issues with RSS and Atom taking different values for th
Re: (Score:2)
I'll apply the patches from RT locally and wait for the next release of XML::Feed to fix the problems.
[1] Often a pointless task on a planet as so much of the markup is out of your control.
Re: (Score:1)
Similar attributes cause issues with the Atom feed, but they don't prevent validation, merely cause warnings, because Atom does at least recognise the concept of extensibility.
Plagger dependencies (Score:2)
http://search.cpan.org/src/MIYAGAWA/Plagger-0.7.17/META.yml [cpan.org]
Plagger's "requirement" modules are all generic, from Cache, DateTime and LWP to HTML, XML and URI fetch modules. They are all necessary to all kinds of data sources and output as well. Other "plugins" are all pure perl and you can just skip installing the dependencies, which are disabled by
Re: (Score:2)
You're absolutely right (of course!) The vast majority of the pre-requisites are optional. I had forgotten that.
I still think, however, that you're bundling too much stuff together. If I was bundling Plagger I'd create a core distribution that just read feeds, combined them and published new ones. I'd then relegate all
Re: (Score:2)
The only reason why we haven't done this was probably because we had never got to the point "yes we're done", and I just didn't want to see people uploading their random Plagger plugins to CPAN that will eventually be unmaintained, abandoned, out of sync with core, and in a poor quality code etc, etc.
It doesn't take you a minute to name a few Catalyst modules that are "out-of-date" or "was a total mistak
failed to install (Score:1)
First, I tried several alternatives for using PAR, building a packed file on my Linux laptop and uploading to the FreeBSD server. That failed in part because PAR failed to detect all the dependencies, including some of the XML::Feed namespace modules and some of the DateTime modules. It was also overly conservative about which modules it through n
Re: (Score:2)
The dependencies checker indicates that XML::Feed has a 44% chance [cantrell.org.uk] of installing correctly whereas Plagger has a 26% chance [cantrell.org.uk]. Of course, Plagger uses XML::Feed so Plagger never going to be easier to install than XML::Feed.
I had exactly the same problem with XML::LibXML that you did. My server is a Fedora Core 6 system and I install all of my modules using rpm (buildi
Optional extras (Score:2)
The likelihood of success calculated by cpandeps only takes into account the mandatory dependencies (those in 'requires' and 'build_requires') and ignores those that are merely recommended - taking recommended modules into account would in fact *reduce* the chance of success. It's also worth noting that the link you give is for the dependency tree and test results when using latest version of perl (5.10.0) and for any operating system. It's always a good idea to change the filters to match your perl versi
Re: (Score:1)