Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

barbie (2653)

barbie
  reversethis-{ku. ... m} {ta} {eibrab}
http://barbie.missbarbell.co.uk/

Leader of Birmingham.pm [pm.org] and a CPAN author [cpan.org]. Co-organised YAPC::Europe in 2006 and the 2009 QA Hackathon, responsible for the YAPC Conference Surveys [yapc-surveys.org] and the QA Hackathon [qa-hackathon.org] websites. Also the current caretaker for the CPAN Testers websites and data stores.

If you really want to find out more, buy me a Guinness ;)

Links:
Memoirs of a Roadie [missbarbell.co.uk]
[pm.org]
CPAN Testers Reports [cpantesters.org]
YAPC Conference Surveys [yapc-surveys.org]
QA Hackathon [qa-hackathon.org]

Journal of barbie (2653)

Friday April 24, 2009
04:17 AM

New CPAN Testers Reporting Preferences

The CPAN Testers Preferences site has for sometime now only offered the ability to allow authors to receive the Daily Summary Report. This was a reaction to the authors who did not wish to receive the deluge of reports that they had started to receive, with the increased volume of testing and reporting done by CPAN Testers. However, there have been several authors who have asked to still receive the individual reports, while others are happier to receive less frequent reporting.

After several test runs over the last couple of weeks, the site can now offer the following reporting options:

  • Daily Summary Report
  • Weekly Summary Report (choose your prefered day)
  • Monthly Summary Report
  • Individual Reports

The Daily Summary Report remains as is, and has received no changes. The Weekly Summary Report and Monthly Summary Report simply provide the same style of reporting as the Daily Summary report, just over a longer period. Note that you can choose which day you wish to receive the Weekly Summary Report too.

Lastly we have the Individual Reports. Essentially this allows the author to get the reports direct from the tester again. However, it comes with several caveats. Firstly the process behind the scenes to generate the mails is initially only going to be run once a day. Once any issues that surface over the next few weeks are resolved, this will become more frequent. Secondly, the mailer process does not receive the reports directly from the tester, it is not a mail relay in the traditional sense. The latter therefore could cause problems.

The process for Individual Reports is to read the reports from the CPAN Testers database and regenerate the original email. Except it isn't the original email, as it isn't sent from the testers original IP. This is where problems may arise. The mails are sent from the CPAN Testers server, and any receiving server that implements SPF checking, will undoubtably reject the mails. This is perhaps most likely to affect reports from testers that use GMail or Hotmail accounts. Some authors have a challenge/request email setup, which could prove troublesome unless you're able to whitelist the CPAN Testers server in some way. However, if you've actively selected to receive Individual Reports, then I would hope that you are prepared for this. Lastly, we have the issue of dubious email addresses used by testers. While 'root@localhost' can get through some systems, it may get rejected by a few.

Over the next few weeks I shall attempt to watch for any issues (as above or otherwise), and will review how the mails are sent out. If specific domains or addresses prove too troublesome, then I may have to implement a blacklist to prevent some reports from being sent. If you wish to receive the Individual Reports and spot any issues client side, please let me know and I will look into it. If you have suggestions for improving the reporting process further, then please let me know too. Though always bear in mind this system currently operates for almost 4,000 authors :)

Monday April 06, 2009
10:12 AM

CPAN to git helper

As I have a lot of CPAN distributions I want to port to git, I decided to try and make things a little easier (and fun) along the way. As I don't have a previous VCS repo for my distributions (I once did, but it died a long time ago), I currently rely on snapshots and backing up the current workspace. The snapshots happen to be the tarballs I upload to CPAN, and are kept in the current workspace so they get backed up too.

This made for writing a simple tool to firstly create a new git repo, unwrap each snapshots in turn, importing the contents and removing obsolete files, then doing the same for the current workspace. I did think about adding some code to login to Github, create the new project and then push the master and tags to Github, but I think that might be going a bit too far for the moment.

Any road up, if you have a similar setup, and want to port your CPAN distributions to a git repo that can be moved to Github, my little script might go a long way to getting you off the ground. The usual disclaimers about me not being responsible for breaking your code apply :)

Sunday April 05, 2009
10:30 AM

CPAN Testers now on Github

On Saturday I upgraded git-core on the cpantesters.org server. As a consequence none of the repos on the server now work :( Thankfully I have my local copy, and a backup copy, so nothing has been lost. To make life easier for myself, in case it happens again, I've now ported all the CPAN Testers distros that I look after onto Github. To see them all (together with a few extra repos) check my profile page.

In the coming weeks I'll also be storing all my other CPAN distros on Github, so you'll then have another way to send me patches :)

Saturday April 04, 2009
04:37 AM

CPAN Testers Stats - March Summary - A Hundred Million Suns

CPAN Testers Statistics

Okay so there weren't exactly a hundred million, but Chris Williams did pass the 1 million reports submitted mark on 10th March 2009. Andreas König also passed the 500,000 reports submitted mark. Together with Slaven Rezić, David Golden and Dave Cantrell, these 5 CPAN testers now account for 2/3rds of all the reports now submitted to the CPAN Testers database. It was fantastic to have 4 of the 5 guys all meet up in Birmingham recently. I did mean to get a group photo during the hackathon, but sadly forgot :( Thanks to all the testers, for helping to make the CPAN one of the most admired code repositories in the world.

As mentioned above, we had a few stars meet up in Birmingham at the end of March for the 2009 QA Hackathon, all of whom made the event extremely worthwhile. The event featured 3 days of brainstorming, coding and testing. It did also feature some eating, drinking and a little bit of sleeping along the way too. The full results and thoughts of the hackathon will no doubt appear on blogs and in write-ups in the coming weeks, but I for one am really please to have been a part of it. CPAN Testers was just one of the featured projects, which saw several ideas come together. CT2.0 is firmly within our sights and we'll be looking to unleash it to testers very soon.

With the hackathon over, and the fact that we've now had two very successful QA Hackathons, it would be nice to continue the event next year. There is a possibility that another European location might be able to host the event next year, but there is no reason why there can't be a US one too. I'm sure there are several US developers that would love to take part, but just can't afford the airfare to Europe. As there isn't a formal committee to decide a location, it's pretty much up to a local group to decide they would like to host the event. Perhaps you work for a company willing to provide a venue and wifi, or have funding to find somewhere equally suitable. Either way attendees from past hackathons would be delighted to hear from you. If you do want to host a QA Hackathon, please get in touch with JJ, as he holds the DNS for the qa-hackathon.org domain, and would be only too pleased to point '2010.qa-hackathon.org' in the direction of your web server.

During the hackathon the Metabase, that David Golden and Ricardo Signes originally devised in Oslo, got a wider audience, which meant it also got some meat and flesh on its bones, as well as giving everyone the chance to pick holes in some of the design decisions. Thankfully David and Ricardo did a very good job in the first place, so it was mostly ensuring that the edge cases were covered, and that we had an agreed method to migrate existing testers and the testing infrastructure. Looking at David and Ricardo's work so far, I think we have something very exciting to look forward to. There was a lot of CPAN Testers related effort over the weekend, with Chris and Jos working together on CPANPLUS related tasks, pulling in David Golden to review how to abstract the reporting code into a common distribution for CPANPLUS and CPAN.pm. Tux and Andreas looked at the structured reports, specifically looking for better ways to parse the environment to provide much more detailed reporting, even more than we have previously managed with CPAN Testers. Rich Dawe, working from David and Ricardo's design, wrote the transporter that will now takes the structured data and feeds it into the Metabase. The Metabase itself got a name change, as during the discussions, Ricardo realised that it could eaily be used for much more than just CPAN. It still needs work, but I think it's been a very productive few days, and I looking forward to getting the Metabase finally online. If only for the fact that we will finally be able to switch off the SMTP and NNTP parts of the current system.

A couple of updates I made behind the scenes during the hackathon, included the switch to using PAUSE for the direct mirror and adding in more reporting capabilities to the Authors Notification system, that currently produces the Daily Summary Reports. With the former, Andreas' script now updates the local CPAN Testers CPAN mirror every 60 seconds, which means the database on the backend of the CPAN Testers infrastructure is now very up to date. Unfortunately due to the time taken to update all the web files, you don't get to see the benefits as quickly as I'd like, but it's on my TODO list to improve the processing. For the author reporting, shortly you should be able choose weekly and monthly summary reports, as well as the ability to switch on individual reports again. The individual reports were switched off by the testing tools, due to the number of authors that were unhappy receiving them. However, there are still several authors that would like them. If all goes well with the final round of testing, expect an announcement next week.

We topped 143 testers submitting reports last month, so thank you once again to everyone involved. The mappings this month included 18 total addresses mapped, of which 10 were for newly identified testers.

Friday April 03, 2009
12:35 PM

The Fastest Mirror?

During the QA Hackathon last weekend, Andreas had a task for me. I run a CPAN mirror on the CPAN Testers server, which currently uses funet.fi as its source. However, Andreas has written some code that has the ability to update directly from PAUSE. After tests this week, I now have the /authors and /modules directories updating from PAUSE and the others from funet.fi

But that's not the full story. Currently funet.fi updates from PAUSE every 30 minutes, but PAUSE updates much more frequently than that, as authors upload their latest releases. As such Andreas' script allows me to update every 60 seconds, so the mirror is as update as possible. Actually that's not true either, as Andreas suggested I should update every 20 seconds!

The mirror is now public, with http, ftp and rsync access. If you choose to use it, please look after it :)

Thursday April 02, 2009
03:07 AM

The Matrix Rehosted

Last night Slaven emailed me to let me know that the CPAN Testers Matrix that he runs had lost its domain. Thankfully he still has the site running on a new IP. As such from here on in, if you have the old URL bookmarked or referenced in documentation or on websites, you'll now need to use the sub-domain previously set up under the cpantesters.org domain.

Tuesday March 24, 2009
12:01 PM

2009 QA Hackathon Update - 24th March 2009

Dear All,

Welcome to the last update before the 2009 QA Hackathon.

SPONSORSHIP & FUNDING

Before starting out on the update, we wanted to make sure you all knew who to thank for helping to put on the QA Hackathon. With final costs coming in last week, our budget slightly went over the edge. However, thankfully we have some wonderful sponsors and we're back on course for a very successful weekend of hacking.

So a big thank you to:

  • Birmingham Perl Mongers
  • Dijkmat
  • Digital Craftsmen
  • $foo Magazin
  • Nestoria

As well as a thank you to the following companies for funding their employees to attend:

  • PROCURA B.V.
  • Booking.com

TRAVEL

Please make sure that you are familiar with the Directions Guide [1], and where the hotel is located (Jury's Inn, Broad Street), so you can at least find your way initially.

FRIDAY EVENING

On Friday, I plan to be at the hotel for about 1pm, and will aim to be around in the lobby after checking in. Once those arriving by air around lunchtime have had a chance to check-in and freshen up, we'll head over to the The Wellington. We'll be passing a couple of places to eat if anyone fancies a late lunch. Unfortunately The Wellington doesn't serve food, however you are allowed to bring your own :) It is also worth noting that The Wellington is A Real Ale Pub, and usually has 10-15 guest and local ales on tap.

We have tables booked at The Thai Orchid for 7pm, for 18 people. The Wellington and The Thai Orchid are virtually opposite each other on Bennetts Hill, off New Street. If you get there slightly earlier than the rest of us, ask for the table for 'Birmingham Perl Mongers'.

After the meal, we'll be walking slightly down the hill, to The Hill, where some of the later arrivals will hopefully catch up with the rest of us.

See also Friday Evening Events page [2]. Note that directions have been added to this page, to both pubs and the restaurant, from The Jury's Inn.

SATURDAY

The QA Hackathon will kick-off at 9am at The City Inn. We'll have some initial outlines for everyone for what they hope to work on during the weekend, then splinter off into working groups as appropriate.

During the day, there will be refreshments provided during the morning and afternoon, with a buffet served for lunch.

The room is booked until 5.30pm, at which time we will break, with everyone heading back to their hotels to freshen up. We will meetup at 6.30pm in the Jury's Inn lobby, before heading over to Brindleyplace and the Handmade Burger Co. for food. After the meal, we will be heading over the canal to the Flapper & Firkin. Advice: DO NOT fall into the canal ... it isn't nice!

Note that Digital Craftsmen have also put some money into the kitty to buy the first round. The Flapper & Firkin is hosting an unsigned band festival this weekend, so we may head to The Malthouse if it proves too busy.

See also Saturday Evening Events page [3].

SUNDAY

The day will begin at 9am at The City Inn, finishing at 5.30pm. Refreshments and buffet provided as per Saturday.

After the day's hacking, we will again meet in the Jury's Inn lobby for 6.30pm before heading up Broad Street to Shimla Pinks. After the meal we will be heading back to Brindleyplace to All Bar One.

See also Sunday Evening Events page [4].

MONDAY

For the final day we will begin at 9am at The City Inn, finishing at 5.30pm, with refreshments and buffet provided as per Saturday. We'll aim to have a wrap-up session during the afternoon, so everyone can present a round up of what they have achieved over the weekend.

For those not heading home immediately after the hackathon, the rest of us will meet in the Jury's Inn lobby for 6.30pm, then head over to The Mailbox and Bar Room Bar. After the meal we can either stay in Bar Room Bar, or head over to Penny Blacks, which is also in The Mailbox.

See also Monday Evening Events page [5].

QUESTIONS

If you have any further questions, please email us at organisers@qa-hackathon.org.

LINKS

[1] http://qa-hackathon.org/wiki/DirectionsGuide
[2] http://qa-hackathon.org/wiki/FridayEvening
[3] http://qa-hackathon.org/wiki/SaturdayEvening
[4] http://qa-hackathon.org/wiki/SundayEvening
[5] http://qa-hackathon.org/wiki/MondayEvening

THANKS

JJ and myself would like to thank everyone who is taking part in the hackathon this weekend, as well as all our sponsors. Without you all, this would not be happening.

We look forward to seeing you all, and hope you all have pleasant and safe journeys getting here.

See you soon...

Thanks,
The 2009 QA Hackathon Organisers.

04:54 AM

I am not inhuman!

I got this in my inbox this morning:

THIS IS UNSOLICITED, THUS SPAM !

STOP THIS *NOW* AND REMOVE ME FROM THIS LIST. I CAN KEEP TRACK OF CPAN DISTS MYSELF.

This is in response to a CPAN Testers daily summary report. The above is cut-n-paste from the original. I then discover I'm in the middle of a mud-slinging regarding the daily summary because it doesn't go far enough.

Is it really that difficult to realise I'm actually a real person, and actually quite responsive most of the time. Politeness goes a long way. Treating me like the scum of the earth doesn't.

Sunday March 22, 2009
01:49 PM

[ANNOUNCE] Birmingham.pm Technical Meeting - Wednesday 25th

Event:   Birmingham.pm Technical Meeting
Date:    Wednesday 25th March 2009
Times:   from 7pm onwards (see below)
Venue:   The Victoria, 48 John Bright Street, Birmingham, B1 1BN.
Details: http://birmingham.pm.org/tech/next

Talks:

  • Whizzy new JavaScript stuff [Stuart Langridge]
  • PDF Manipulation with Perl [JJ]

Details

This month we welcome a returning guest speaker, Stuart Langridge, who will be taking a look at some of the cool things in JavaScript that you never knew about, plus some of the things planned for the language for the future. JJ will also be taking a look at a few ways to create PDF documents using Perl.

As per usual, this month's technical meeting will be upstairs at The Victoria. The pub is on the corner of John Bright Street and Beak Street, between the old entrance to the Alexandra Theatre and the backstage entrance. If in doubt, the main entrance to the Theatre is on the inner ring road, near the Pagoda roundabout. The pub is on the road immediately behind the main entrance. See the map link on the website if you're stuck.

As always entry is free, with no knowledge of Perl required. We'd be delighted to have you along, so feel free to invite family, friends and colleagues ;)

Some of us should be at the venue from about 7.00pm, usually in the backroom downstairs. Order food as you get there, and we'll aim to begin talks at about 8pm. I expect talks to finish by 9.30pm, with plenty of time for discussion in the bar downstairs.

Venue & Directions:

The Victoria, 48 John Bright Street, Birmingham, B1 1BN
- Pub Details
- Picture
- Google Map

The venue is approximately 5-10 minutes walk from New Street station, and about the same from the city centre. On street car parking is available see full details and directions on the website.

Times:

These are the rough times for the evening:

  • food available until 9.00pm
  • talks: 8.00-10.00pm
  • pub closes: 11.00pm

Please note that beer will be consumed during all the above sessions ;)

Friday March 20, 2009
11:00 AM

A New Future

After 6 years at ML, I shall be moving on from helping to protect the world from malware. My last day will be 31st March (the day after the QA Hackathon ends), and as yet I have no idea what I'll be doing next. My immediate plan is to work on lots of Open Source stuff during April, but beyond that I'm just going to take the time to find the right job for me.

It's a rare opportunity to take a step back and look at what you want to do, so I'm going to do my best not to rush into any decision too quickly.

Having said that, if you know of anyone who could put a Perl geek to good use, my CV is available online.