Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

dagolden (6584)

dagolden
  (email not shown publicly)
AOL IM: xdaveg (Add Buddy, Send Message)

Journal of dagolden (6584)

Wednesday April 09, 2008
02:03 PM

Oslo Hackathon 2008 -- wrap-up writeup

[ #36113 ]

First, I'd like to thank The Perl Foundation for sponsoring my trip to the hackathon. I'd also like to thank Linpro for providing a wonderful venue as well as on-site food and refreshments throughout the hackathon. This is my wrap-up summary so they have a sense of what their contributions helped make possible.

Project 1 -- Test::Reporter transport plugins

I worked with rjbs on adding "transport" plugins to Test::Reporter to provide better options for how reports are sent to CPAN Testres. The existing transports (Mail::Send and Net::SMTP) were extracted into separate modules (Test::Reporter::Transport::Mail::Send, etc.). Plus we created three new transports:

  • Test::Reporter::Transport::HTTPGateway -- provides HTTP to email transport via a the new Test::Reporter::HTTPGateway module

  • Test::Reporter::Transport::Net::SMTP::TLS -- provides TLS and authenticated SMTP (a long-standing wishlist item)

  • Test::Reporter::Transport::File -- reports saved as files in a directory for offline CPAN Testing and manual submission

Unfortuately, the HTTPGateway transport is only a temporary step towards a better CPAN Testers as the gateway still just relays to email. But people can set up their own gateways for now or perhaps a central server will be deployed somewhere.

Using transport plugins for all of these with a consistent API makes it easy for clients like CPAN::Reporter to switch between them based on configuration settings. E.g., in .cpanreporter/config.ini

transport=HTTPGateway http://gateway.example.com/http2email/

Project 2 -- CPAN Metabase for CPAN Testers 2.0

The big issue with CPAN Testers "1.0" (what we currently have) is that it uses email to cpan-testers@perl.org for submissions and the only archive for reports is effectively the perl.cpan.testers newsgroup. For a while, I and others were talking about creating CPAN Testers 2.0 to move to HTTP submission and a centralized, indexed, searchable database.

I'd hoped to interest people in Oslo in thinking through what it might require or even working on some pieces of it. What I found was that there are a lot of areas where people are looking to collect 'meta' information about CPAN distributions, each with their own approach and APIs for gathering, storing or publishing in a very application-specific way.

That got me and rjbs started designing a more general solution, which we've called a CPAN 'metabase'. This would work for CPAN Testers 2.0 or for other projects that want to store/publish distribution-level information.

After a few days of work, we achieved:

  • General design of a system (class hierarchy, architecture, etc.) to meet at least initial envisioned needs with the flexibility to expand over time

  • A simple, working proof-of-concept -- with filesystem-based storage and indexing and the ability to store and retrieve a simple "fact" (a text string) via a web client.

Next steps are to refine, standardize and document the classes and APIs; to add additional capabilities to the proof-of-concept; and to get a 0.01 release to CPAN for community feedback.

Other contributions

  • Moral support for Adam Kennedy's time-travelling efforts to build the April Strawberry Perl

  • Feedback on TAP diagnostic semantics and the implication for downstream consumers like CPAN::Reporter

  • Figured out a hack with the Berkeley DB libraries to help Tux build DB_File on Strawberry Perl

  • Participated in a best practice discussion/argument on environment variables, 'xt' directories, etc.

Things that came up that I didn't get around to working on

  • CPAN::Reporter::Smoker -- a couple people (Gabor and someone else?) said it would be helpful if C::R::S could take a specific list for a work queue instead of trying to smoke only the latest things on CPAN. Now that someone asked for it, that gets bumped up on my priority list

  • Salve brought up using boilerplating as a way of discussing and later disseminating best practices so I showed him a boilerplate tool in my repository (not yet released). He and rjbs thought it had promise given easy customization so I'm going to try to finish the documentation and get a 0.01 release out soon

  • Schwern brought up the CPAN::PERL5INC taint bug in recent devel versions of CPAN.pm

  • Various people were working on codifying and releasing modules for various parts of the PAUSE/CPAN tools. rjbs has a work project along these lines and brian d foy's work to index BackPAN is driving similar efforts. There was some discussion of making it a YAPC::NA hackathon project

Other things I learned (not necessarily Perl)

  • git is awesome. I had been looking for a project to force me to live with it and get over the learning curve. Since Test::Reporter lives in git and rjbs knew it already, I got my chance. rjbs and I were routinely developing simultaneously on the same code base, merging back and forth several times a day and despite the huge flux in our early development code, it "just worked"

    The best part was that during the flight home, we continued hacking, using a git repository on a usb drive to exchange and merge our branches. We'd just walk it between our seats about every hour or so

  • Developing Perl on a Ubuntu virtual machine on Windows is a vastly better, faster experience than native Perl on Windows. (It would be interesting for someone to figure out what's behind that)

  • rjbs showed me the wonders of screen (and screen + irssi). I don't know how I could have gone so long without learning how cool it is.

  • Norwegians are friendly. The organizers, Linpro staff and people on the streets were welcoming and helpful whenever we need it

Other thanks

  • To Pobox (via rjbs) for a temporary mailbox for testing TSL and authenticated SMTP

  • To the BBC for backstage.bbc.co.uk swag

  • To jonasbn for being a sounding board as rjbs and I drew crazy metabase design diagrams

  • To rjbs for putting in a tremendous effort in a short period of time on projects that he wasn't even focusing on before the hackathon and for teaching me a lot of new tools

-- dagolden

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Hi David,

    thanks for the write-up. You guys really did accomplish a lot of stuff in the short time.

    Cheers,
    Steffen

    P.S.: Somewhen, I need to get off my arse (read: take some vacation and set aside the cash for travelling) and participate in some hackathon.