barbie's Journal http://use.perl.org/~barbie/journal/ barbie's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:00:31+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 barbie's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~barbie/journal/ Announcing CPAN Testers 2.0 http://use.perl.org/~barbie/journal/40435?from=rss <p>After 6 months of development work, following 2 years worth of design and preparation, CPAN Testers 2.0 is finally live.</p><p>With the rapid growth in CPAN Testers environments and testers over the past few years, the previous method of posting reports to a mailing list had reached a point where the scalability was no longer viable. This was recognised several years ago and discussions for a new system had already begun, with the view that reports should be submitted via HTTP.</p><p>At the Oslo QA Hackathon in 2008, David Golden and Ricardo Signes devised the Metabase, with the design work continuing at the Birmingham QA Hackathon in 2009, where David and Ricardo were able to bring others into the thought process to work through potential issues and begin initial coding. A number of releases to CPAN and Github followed, with more people taking an interest in the project.</p><p>The Metabase itself is a database framework and web API to store and search opinions from anyone about anything. In the terminology of Metabase, Users store Facts about Resources. In the Metabase world, each CPAN tester is a User. The Resource is a CPAN distribution. The Fact is the test report. Today that&#8217;s just the text of the email message, but in the future it will be structured data. The Metabase specifies data storage capabilities, but the actual database storage is pluggable, from flat files to relational databases to cloud services, which gives CPAN Testers more flexibility to evolve or scale over time.</p><p>Meanwhile the CPAN Testers community was also attracting more and more interest from people wanting to be testers themselves. As a consequence the volume of reports submitted increased each month, to the point that the perl.org mail server was struggling to deal with all the mailing lists it hosted. The cpan-testers mailing list was submitting more posts in one day than any other list submitted in a month (in a year in some cases). Robert and Ask, very reasonably, asked if the testers could throttle their submissions down to 5k report posts a day, and set a deadline of 1st March 2010 to switch off the mailing list.</p><p>David Golden quickly took on the task to envisage a project plan, and work began in earnest in December 2009. With less than 3 months to the cut-off date, there was a lot of work to do. David concentrated on the Metabase, with Barbie working on ensuring that the current cpanstats database and related websites could move to the Metabase style of reports. Despite a lot of hard work from a lot of people, we unfortunately missed the 1st March deadline. Having throttled report submissions to a more manageable level, and although not complete, the target for HTTP submissions was in sight, Robert and Ask were very understanding and agreed to keep us going a little while longer.</p><p>Throughout March and April a small group of beta testers were asked to fire their submissions at the new system. It ironed out many wrinkles and resulted in a better understanding of what we wanted to achieve. The first attempts at retrieving the reports from the Metabase into the cpanstats database began in April, and again highlighted further wrinkles that needed to be addressed. After a month of hard testing and refinement, we finally had working code that went from report submission by a tester, storage into the Metabase, retrieval into the cpanstats database and finally presentation on the CPAN Testers family of websites.</p><p>During June the process was silently switched from testing to live, allowing reports to be fed through into the live websites. Due to the ease with which the new style reporting fit into the existing system, the switch largely went unnoticed by the CPAN testers community as well as the Perl community. A considerable success.</p><p>The CPAN Testers eco-system is now considerably larger than those early days of simply submitting handwritten reports by email to a mailing list, and the work to get here has featured a cast of thousands. Specifically for CPAN Testers 2.0, the following people have contributed code, ideas and effort to the project over the past six months:</p><ul> <li>Andreas K&ouml;nig</li><li>Apocalypse</li><li>Ask Bj&oslash;rn Hansen</li><li>Barbie</li><li>Chris Williams</li><li>Dan Collins</li><li>David Cantrell</li><li>David Golden</li><li>Florian Ragwitz</li><li>H.Merijn Brand</li><li>Jon Allen</li><li>Lars D&#618;&#7431;&#7428;&#7435;&#7439;&#7457; &#36842;&#25289;&#26031;</li><li>L&eacute;on Brocard</li><li>MW487</li><li>Nigel Horne</li><li>Ricardo Signes</li><li>Richard Dawe</li><li>Robert Spier</li><li>Serguei Trouchelle</li><li>Shlomi Fish</li><li>Slaven Rezi&#263;</li></ul><p>Barbie and David would like to thank everyone for their involvement. Without these guys CPAN Testers 2.0 would not have been possible. Thanks to everyone, we can now look forward to another 10 years and more of CPAN Testers.</p><p> <a href="http://www.cpantesters.org/">CPAN Testers</a> now holds over 7.5 million test reports covering nearly 11 years worth of testing Perl distributions. There have been over 1,000 testers in that time, and every single one has helped the CPAN Testers project to be the largest single community supported testing system of any programming language. For a full list of everyone who has contributed, visit the <a href="http://stats.cpantesters.org/testers.html">CPAN Testers Leaderboard</a>. A huge thank you to everyone.</p><p>With the Metabase now online and live, we can now announce an absolute deadline to close the mailing list. This is currently set as 31st August 2010. After this date all submissions via email will be rejected, and testers will be encouraged to upgrade their testing tools to take advantage of the new HTTP submission system. Many of the high volume testers have already moved to the new system, and we expect nearly everyone else to move in the next month. We will be tailing the SMTP submissions to catch those who haven't switched, such as some of the more infrequent testers, and warn them of the deadline.</p><p>More work is planned for CPAN Testers, from further validation and administration of reports, to providing more functionality for alternative analysis and search capabilities. Please check the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a> for our regular updates.</p><p>If you'd like to become a CPAN Tester, please check the <a href="http://wiki.cpantesters.org/">CPAN Testers Wiki</a> for details about setting up a smoke testing environment, and join the <a href="http://lists.perl.org/list/cpan-testers-discuss.html">cpan-testers-discuss mailing list</a> where many of the key members of the project can offer help and advice.</p><p>You can find out more about CPAN Testers at two forthcoming conferences. David Golden will be presenting <a href="http://www.oscon.com/oscon2010/public/schedule/detail/13759">"Free QA! What FOSS can Learn from CPAN Testers"</a> at OSCON and Barbie will be presenting <a href="http://conferences.yapceurope.org/ye2010/talk/2891">"CPAN Testers 2.0 : I love it when a plan comes together"</a> at YAPC::Europe.</p><p>CPAN Testers is sponsored by Birmingham Perl Mongers, and supported by the Perl community.</p><p>You can now <a href="http://blog.cpantesters.org/press-release-ct20-20100705.pdf">download the full and complete Press Release</a> from the CPAN Testers Blog. If you have access to further IT news reporting services, please feel free to submit the Press Release to them. Please let us know if you are successful it getting it published.</p><p>Cross-posted from the <a href="http://blog.cpantesters.org/diary/83">CPAN Testers Blog</a> </p> barbie 2010-07-05T09:50:22+00:00 journal Birmingham.pm Technical Meeting - Wednesday 26th May 2010 http://use.perl.org/~barbie/journal/40364?from=rss <code> Event:&nbsp;&nbsp;&nbsp;Birmingham.pm Technical Meeting<br> Date:&nbsp;&nbsp;&nbsp;&nbsp;Wednesday 26th May 2010<br> Times:&nbsp;&nbsp;&nbsp;from 7pm onwards (see below)<br> Venue:&nbsp;&nbsp;&nbsp;The Victoria, 48 John Bright Street, Birmingham, B1 1BN.<br> Details:&nbsp;<a href="http://birmingham.pm.org/tech/next">http://birmingham.pm.org/tech/next</a> <br> </code> <p> <b>Talks:</b> </p><ul> <li>Accelerated web development with Catalyst [Richard Wallman]</li><li>CPAN Testers 2.0 - "I love it when a plan comes together" [Barbie]</li></ul><p> <b>Details</b> </p><p>This month we welcome a returning guest speaker, Richard Wallman, who will be taking a look at how Catalyst has eased the development lifcycle of websites, from his own experiences. In addition I'll be looking at the progress of the CPAN Testers 2.0, and looking at some of the near future plans for CPAN Testers.</p><p>As per usual, this month's technical meeting will be upstairs at The Victoria. The pub is on the corner of John Bright Street and Beak Street, between the old entrance to the Alexandra Theatre and the backstage entrance. If in doubt, the main entrance to the Theatre is on the inner ring road, near the Pagoda roundabout. The pub is on the road immediately behind the main entrance. See the map link on the website if you're stuck.</p><p>As always entry is free, with no knowledge of Perl required. We'd be delighted to have you along, so feel free to invite family, friends and colleagues<nobr> <wbr></nobr>;)</p><p>Some of us should be at the venue from about 7.00pm, usually in the backroom downstairs. Order food as you get there, and we'll aim to begin talks at about 8pm. I expect talks to finish by 9.30pm, with plenty of time for discussion in the bar downstairs.</p><p> <b>Venue &amp; Directions:</b> </p><p> The Victoria, 48 John Bright Street, Birmingham, B1 1BN<br> - <a href="http://www.pubs247.co.uk/pubPages/pub.asp?id=24358&amp;from=&amp;promotion=">Pub Details</a> <br> - <a href="http://birmingham.pm.org/images/upload/victoria.jpg">Picture</a> <br> - <a href="http://www.google.co.uk/maps?hl=en&amp;client=firefox-a&amp;q=the+victoria+pub&amp;near=Birmingham&amp;radius=0.0&amp;cd=1&amp;cid=52482921,-1893619,7492755984503563963&amp;li=lmd&amp;z=14&amp;t=m">Google Map</a></p><p>The venue is approximately 5-10 minutes walk from New Street station, and about the same from the city centre. On street car parking is available see full details and directions on the <a href="http://birmingham.pm.org/tech/next">website</a>.</p><p> <b>Times:</b> </p><p>These are the rough times for the evening:</p><ul> <li>food available until 9.00pm</li><li>talks: 8.00-10.00pm</li><li>pub closes: 11.00pm</li></ul><p>Please note that beer will be consumed during all the above sessions<nobr> <wbr></nobr>;)</p> barbie 2010-05-24T19:26:06+00:00 journal CPAN Testers Summary - December 2009 - The Wall http://use.perl.org/~barbie/journal/40083?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/diary/64">CPAN Testers Blog</a>.</p><p>Last month CPAN Testers was finally <a href="http://blog.cpantesters.org/diary/61">given a deadline</a> to complete the move away from SMTP to HTTP submissions for reports. Or perhaps more accurately to move away from the perl.org servers, as the amount of report submissions has been affecting support of other services to the Perl eco-system. The deadline is <b>1st March 2010</b>, which leaves just under 2 months for us to move to the CPAN Testers 2.0 infrastructure. Not very long.</p><p> <b>David Golden</b> has now put together a <a href="http://www.dagolden.com/index.php/614/project-plan-for-cpan-testers-2-0-migration/">plan of action</a>, which is being rapidly consumed and worked on. The first fruits of which has been an update to the <a href="http://www.cpantesters.org/">CPAN Testers Reports</a> site. The ID previously visible on the site, refering to a specific report, is now being hidden away. The reason for this is that the current ID refers to the NNTP ID that is used on the perl.org NNTP archive for the <i>cpan-testers</i> mailing list. This ID is specific to the SMTP submissions and includes many posts which are not valid reports. As such we will be moving to a GUID as supplied by the Metabase framework, with existing valid SMTP submitted reports being imported into the Metabase. The NNTP ID will eventually be completely replaced by the Metabase GUID across all parts of the CPAN Testers eco-system, including all the databases and websites. As such you will start to see a transition over the next few weeks.</p><p>The second change which has now been implemented, is to present the reports via the <a href="http://www.cpantesters.org/">CPAN Testers Report</a> site and not the NNTP arcive on the perl.org servers. Currently the presentation of a report (e.g. <a href="http://www.cpantesters.org/cpan/report/6524841">this report for App-Maisha</a>) is accessed via the reports pages for a distribution or an author, but will also be accessible in a similar manner across all the CPAN Testers websites. There are a large batch of early reports that are currently missing from the database, but these are being updated now, and will hopefully be complete within the next few days. If you have any issues with the way the reports are presented, including any broken or missing links from other parts of the site, please let me know.</p><p>In all this change, there is one aspect that may worry a few people, and that is the <i>"Find A Tester"</i> application. For the next few months it will still exist, but the plan is to make the Reports site more able to provide tester contact information. In addition to this the testers themselves will soon have the ability to update their own profiles. Initially this will be used to link email addresses to reports and then map those email addresses to a profile held wihtin the Metabase, but in the longer term will be used to help us manage the report submissions better.</p><p>David Golden is concentrating on the Client and Metabase parts of the action plan, and I am working on porting the websites and 'cpanstats' database. If you have any free time and would like to help out, please review the <a href="http://www.dagolden.com/index.php/614/project-plan-for-cpan-testers-2-0-migration/">action plan</a>, join the <a href="http://wiki.cpantesters.org/wiki/MailingLists"> <i>cpan-testers-discuss</i> mailing list</a>, and please let us know where you'd like to help. There is a lot of work to be done and the more people involved, the better the spread of knowledge in the longer term.</p><p>After David announced the <a href="http://www.mail-archive.com/cpan-testers-discuss@perl.org/msg01550.html">deadline last month</a>, all the testers have throttled back their smoke bots. This saw a dramatic reduction in the number of reports and page being processed, and enabled the Reports Page Builder to catchup with itself, to the point it was frequently having less than a 1000 request waiting. That changed yesterday with the changes to the website, as every page now needs to be updated. It typically takes about 5 days to build the complete site, so this quiet period will help allow the Builder to rebuild the site, without adversely affecting the currently level of report submissions. Expect the site to reach a more managable level of processing some time next week. To help monitor the progress of the builder, a new part of the Reports site, <a href="http://www.cpantesters.org/home/status">The Status Page</a>, now checks the status of all outstanding request every 15 minutes, providing a 24 hour persepctive and a week long perspective.</p><p>A new addition to the family was also launched recently, the <a href="http://analysis.cpantesters.org/">CPAN Testers Analysis</a> site, which <b>Andreas K&#246;nig</b> has been working on, to help authors identify failure trends from reports for their distributions. Read more on <a href="http://blogs.perl.org/users/andreas_konig/2010/01/so-you-got-mixed-results-from-cpantesters.html">Andreas' blog</a>.</p><p>Last month we had a total of 168 tester addresses submitting reports. The mappings this month included 22 total addresses mapped, of which 2 were for newly identified testers. Another low mapping month, due to work being done on CPAN Testers as a whole.</p><p>My thanks this month go to <b>David Golden</b> for finding the time to write an action plan, and <b>his wife</b> for allowing him the time to write it, as well as working on all the other areas involving the CPAN Testers and the Metabase<nobr> <wbr></nobr>:)</p> barbie 2010-01-07T13:54:38+00:00 journal CPAN Testers Summary - November 2009 - Abbey Road http://use.perl.org/~barbie/journal/39976?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a>.</p><p>In November we reached the <a href="http://blog.cpantesters.org/diary/58">6 million reports</a> submitted mark. It's quite staggering how many reports are being submitted these days. It's now roughly 1 million reports every 3 months! So expect a 10 million reports post some time in August 2010<nobr> <wbr></nobr>:)</p><p>Now that we are producing so many reports, while there is a desire to get more reports from less tested operating systems, Tim Bunce recently highlighted his interest in getting reports that included a diverse set of Perl configuration flags, in particular regarding how Perl was compiled (with and without threads, etc). At the moment the CPAN Testers Statistics database doesn't include that information, but the Metabase that is behind CPAN Testers 2.0 will. In addition the Metabase will be able to be queried to glean the reports that contain a specific set of flags, etc. At the moment there are quite a few different setups testing on the top few operating systems being tested. While some authors see these as just repeated results, in some cases they provide slight differences in the test results. This is particularly what Tim was interested in for <a href="http://search.cpan.org/dist/Devel-NYTProf">Devel-NYTProf</a>. Hopefully we'll be closer to getting more of that information more readily available soon. In the meantime, if you do want to get involved with CPAN Testers, and only have a traditional operating system available, take a look at some of the reports posted by current testers for the same platform, and see what different setups you could provide.</p><p>In the CPAN Testers namespace, CPAN has seen a new upload, <a href="http://search.cpan.org/dist/CPAN-Testers-Data-Addresses">CPAN-Testers-Data-Addresses</a>. This release will be the new way for me to manage the tester address mappings. To begin with the testing is being run stand-alone, but it will be shortly be integrated to the <a href="http://stats.cpantesters.org/">CPAN Testers Statistics</a> website. From there it will also be integrated into the new site that is hopefully being launched early next year, which will allow testers to register their testing addresses (among other things). More uploads to the CPAN Testers namespace are being worked on, in particular ones to provide a more programmatic access to the CPAN Testers APIs. More news on those hopefully next month.</p><p>This weekend sees the annual <a href="http://conferences.yapceurope.org/lpw2009/">London Perl Workshop</a>. Featured in the schedule is <a href="http://search.cpan.org/~bingos">Chris 'BinGOs' Williams</a>' talk "<a href="http://conferences.yapceurope.org/lpw2009/talk/2433">Rough Guide to CPAN Testing</a>". If you are a CPAN Tester and are planning to attend the event, please come along and say hello<nobr> <wbr></nobr>:)</p><p>Last month we had a total of 164 tester addresses submitting reports. The mappings this month included 17 total addresses mapped, of which 7 were for newly identified testers. A bit of a low mapping month, mostly due to my attention being elsewhere. With the new mapping system hopefully this will become a little more streamlined for next year.</p><p>Until next time, happy Christmas testing<nobr> <wbr></nobr>:)</p> barbie 2009-12-04T10:52:07+00:00 journal Six Million Reports! http://use.perl.org/~barbie/journal/39934?from=rss <a href="http://blog.cpantesters.org/diary/58">CPAN Testers tops 6 million reports</a> barbie 2009-11-23T13:42:09+00:00 journal Birmingham.pm Sponsor CPAN Testers (again) http://use.perl.org/~barbie/journal/39907?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/diary/56">CPAN Testers Blog</a>.</p><p>Back in 2008, it was obvious that the fragmentation of CPAN Testers sites was a problem. The system was slow, usually getting updated just once a day, and the presentation was a little disjointed. At that point a dedicated server was suggested, as this would bring a number of the key sites together and potentially provide a base with which to improve the updates of the sites. In addition it was seen a first step towards <a href="http://wiki.cpantesters.org/wiki/Roadmap">CPAN Testers 2.0</a>.</p><p>In late September 2008 a proposal was put forward to the members of <a href="http://birmingham.pm.org/">Birmingham Perl Mongers</a>, to donate funds to aquiring a dedicate server to host a range of sites and databases. Unanimously they approved the proposal and a server was paid for at <a href="http://www.hetzner.de/">Hetzner Online AG</a>. Based in Germany on a high bandwidth line, the server has enabled CPAN Testers to grow and now supports a dynamic set of sites and databases that are a consistent benefit to authors and users.</p><p>The server was covered for 1 year, with the intention of looking for a corporate sponsor to continue the funding for further years. However, due to the recent economic climate, the opportunities for funding appear to be limited. As such, recently another proposal was put to the members of Birmingham Perl Mongers, and once again they unanimously approved it. The server and hosting is now paid up for another year, and plans are afoot to further increase the family of sites and provide more resources to authors, testers and users.</p><p>Many thanks to all of <a href="http://birmingham.pm.org/">Birmingham Perl Mongers</a> for their continued support of the CPAN Testers project.</p> barbie 2009-11-17T08:45:40+00:00 cpan CPAN Testers Summary - October 2009 - Live ... In The Raw http://use.perl.org/~barbie/journal/39851?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a>.</p><p>If you've not been following the <a href="http://blog.cpantesters.org/">CPAN Testers</a> in the last month, you will likely have missed the updates to the <a href="http://stats.cpantesters.org/">CPAN Testers Statistics</a> site. I would like to thanks <strong>MW487</strong>, <strong>JJ</strong> and <strong>Colin Newell</strong> for their thoughts and suggestions. The biggest changes have been around the matrices. The old matrices have been thrown away and a completely new set have been created, merging much of the data that was previously across the two old style matrices. The site now also looks at the OS system, rather than the specific version installed, which now gives a better general overview. In turn a new <a href="http://stats.cpantesters.org/mosname.html">OS table</a> is also available highlighting the number of tests per month are attributed to a particular OS. Unsurprisingly Linux is currently streets ahead of any other OS.</p><p>The graphs have always been of interest to those wishing to use them to promote Perl and CPAN, however, the way they are currently presented, doesn't always suit everyone, especially if they wish to change the style or take a different snapshot of the data. As such, you can now download the raw data files used to generate the graphs. All the files are in CSV format, so are easily loaded into you spreadsheet application of choice. Speaking of spreadsheets, in addition to changing the look of the matrices, you can now also download an XLS version of each matrix, as well as now having the ability to view each table in a widescreen format.</p><p>A new graph available is the <a href="http://stats.cpantesters.org/perform.html">Performance Graph</a>, which shows how the CPAN Testers Reports Page Builder is performing each day, against the volume of reports submitted per day. While the majority of the time the Builder does perform well, every so often it slows due to the load on the web server, meaning it has to occasionally catch up, which can take several days. Now you can see whether any issues have caused your page to take a little longer to build, as well get a better idea of how many reports are getting submitted every day.</p><p>The most recent update has been the new dashboard on the homepage. Every so often I get asked how many CPAN distributions are on CPAN. Although the <a href="http://stats.cpantesters.org/statscpan.html">CPAN Statistics</a> have had their own page for a while now, some have mentioned that it would be really cool to have a ticker that flips as a new upload gets added to CPAN. Although I can't do that just yet in true realtime, the new dashboard does try an emulate the rate at which reports and uploads have been submitted over the previous 24 hours.</p><p>In other CPAN related news the proposals and discussions for <a href="http://www.dagolden.com/index.php/519/list-of-cpan-meta-spec-proposals/">Meta-Spec 2.0</a> have now come to a close. David Golden is currently <a href="http://github.com/dagolden/cpan-meta-spec">accepting patches</a> to the approved proposals and hopefully we'll have a new draft specification available soon. It's been an interesting discussion in some cases, while others have been agreed or rejected almost without question. Some require a bit more thought, so it's likely there will be a further refinement of the spec in the future. If you want to read all the threads, visit the <a href="http://www.mail-archive.com/cpan-workers@perl.org/index.html#00197">mailing list archives</a>.</p><p>Last month we had a total of 171 tester addresses submitting reports. The mappings this month included 27 total addresses mapped, of which 14 were for newly identified testers.</p><p>Until next time, happy testing<nobr> <wbr></nobr>:)</p> barbie 2009-11-06T20:23:45+00:00 cpan YAPC Conference Surveys - Future Plans http://use.perl.org/~barbie/journal/39723?from=rss <p>Now that I've got the <a href="http://yapc-surveys.org/yn2009-surveys.html">YAPC::NA</a> and <a href="http://yapc-surveys.org/ye2009-surveys.html">YAPC::Europe</a> surveys done and dusted, I can now start looking at some of the changes I want to make to the survey system. Some I have already been planning, while others have been suggested by people who have taken the surveys. </p><p> <b>Paint By Number Heart</b> </p><p>The YAML file used to drive the surveys was originally designed to not reference any question numbers, with the code creating these on the fly to ensure no duplicates. To begin with this worked fine, but there are now such a variety of questions, including some that relate back to others, that knowing the question number ahead of time is useful. This is a fairly minor change, so shouldn't be too tricky to implement.</p><p> <b>Take It To The Limit</b> </p><p>I've had a few comments from people regarding the number of times they have attended a particular conference or workshop. Seeing as some have now been running for many years, if you've been to all of them, and can't remember how many that is, some have entered big numbers to imply they've been to all of them. As such, in future I'm going to add the feature that allows you to write 'ALL', as well as including the max number of occurances in the question label. Then if anyone does enter a rather large number or 'ALL' it'll get reset to the correct maximum.</p><p> <b>Who Can It Be Now?</b> </p><p>With the talk evaluations, a few speakers have asked whether I can tell them who wrote what in their evaluation results. Due to the nature of submitting everything anonymously I currently don't include any contact details with the responses. Having said that, there is no reason why anyone submitting an evaluation can't give their conscent to being identified, so that speakers can respond to the individual if they wish. The speakers concerned have had genuine reasons for wishing to contact individuals, so from next year there will be an extra tick box if you would like to be identified to the speaker. The default will still be anonymous submission, so it will be entirely the individual's choice as to whether they identify themselves or not.</p><p> <b>Just Like You Imagined</b> </p><p>I started writing a specification for the Survey YAML file during the Birmingham YAPC::Europe in 2006, but haven't kept it up to date with the changes that have been made in more recent times. I plan to complete this so that anyone reading the YAML files can make sense of them. It also might be useful for others to suggest new features.</p><p> <b>P Machinery</b> </p><p>I have the raw data from all the surveys since 2006 in SQL form. However, I want to make it more freely available and accessible, so others can analyse the data in different ways. Together with the raw data itself, this then needs the YAML file, the Survey specification and an example translation program to help understand how the data maps to the questions and what questions relate to each other. Essentially this just means me cleaning up the program I use to prepare the results and documenting everything.</p><p> <b>Speak My Language</b> </p><p>At the moment the survey site is written and presented in English. However, it's been long over due for the questions and text to be in a variety of languages. So although I don't speak other languages (at least not well enough to be competent in them), I'm hoping others will be willing to help out with some translations. Before I get to that stage though, I need to add support to allow for detecting and changing between languages. </p><p>Part of the reason for doing this is to allow YAPCs (and workshops) in other countries to take advantage of the surveys, if they wish to. At the moment I've only been working with the YAPC::Europe and YAPC::NA teams, as they have been the conferences I've attended, and are predominantly English language based. But these days I don't need to be there to run the surveys, and the Perl community pretty much covers the whole world, so why not accumulate knowledge from other events. At the very least, regular organising teams for workshops should then have the opportunity to get feedback to improve their events too.</p><p> <b>Time Stand Still</b> </p><p>It doesn't, but I often wish it would! A couple of people have commented that it takes such a long time between the end of the conference and the close of the surveys. This year I reduced it to 6 weeks, but it can then take an additional couple of weeks to process all the results, create graphs, prepare the website pages, write the documents for organisers and generate the emails for all the speakers. Much of it is automated now, but there are several tests and tweaks that need to happen to get it looking right. This will be improved further with the question numbering change, as I can fine tune the templates to particular questions.</p><p>As for improving the response rate, it often takes people some time to collect their thoughts. Typically within the first 2 weeks after the conference the bulk of responses are received. The last week sees another big increase in responses, when I send out the '1 week left' reminder. The weeks inbetween still have a steady trickle of responses, but it is slow. By way of example below are the number of responses received each day for the YAPC::Europe survey. The last 2 columns indicate the number of survey responses and the number of talk evaluation responses respectively:</p><blockquote><div><p> <tt>Mon&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&lt;-- evaluations opened<br>Tue,&nbsp; 4th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;-&nbsp; 27<br>Wed,&nbsp; 5th August&nbsp; &nbsp; 2009&nbsp; &nbsp; 36 234&nbsp; &lt;-- last day of conference<br>Thu,&nbsp; 6th August&nbsp; &nbsp; 2009&nbsp; &nbsp; 34 119<br>Fri,&nbsp; 7th August&nbsp; &nbsp; 2009&nbsp; &nbsp; 31 101&nbsp; &lt;-- end of tutorials<br>Sat,&nbsp; 8th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;6&nbsp; 55<br>Sun,&nbsp; 9th August&nbsp; &nbsp; 2009&nbsp; &nbsp; 14&nbsp; 47<br>Mon, 10th August&nbsp; &nbsp; 2009&nbsp; &nbsp; 17&nbsp; 28<br>Tue, 11th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;9&nbsp; 58<br>Wed, 12th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;4&nbsp; 34<br>Thu, 13th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;3&nbsp; 13<br>Fri, 14th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;3&nbsp; 18&nbsp; &lt;-- end of week 1<br>Sat, 15th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;3<br>Sun, 16th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;2&nbsp; 11<br>Mon, 17th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;4&nbsp; &nbsp;2<br>Tue, 18th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;2&nbsp; &nbsp;-<br>Wed, 19th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;2&nbsp; 33<br>Thu, 20th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;2&nbsp; &nbsp;-<br>Fri&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&lt;-- end of week 2<br>Sat<br>Sun<br>Mon, 24th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;7<br>Tue, 25th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;-<br>Wed<br>Thu<br>Fri&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&lt;-- end of week 3<br>Sat<br>Sun, 30th August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;6<br>Mon, 31st August&nbsp; &nbsp; 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;-<br>Tue<br>Wed,&nbsp; 2nd September 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;-<br>Thu,&nbsp; 3rd September 2009&nbsp; &nbsp; &nbsp;-&nbsp; 13<br>Fri&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&lt;-- end of week 4<br>Sat<br>Sun<br>Mon<br>Tue<br>Wed<br>Thu, 10th September 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;-<br>Fri, 11th September 2009&nbsp; &nbsp; &nbsp;1&nbsp; &nbsp;-&nbsp; &lt;-- end of week 5<br>Sat<br>Sun<br>Mon<br>Tue, 15th September 2009&nbsp; &nbsp; 23&nbsp; 48&nbsp; &lt;-- reminder sent out<br>Wed, 16th September 2009&nbsp; &nbsp; &nbsp;2&nbsp; &nbsp;1<br>Thu, 17th September 2009&nbsp; &nbsp; &nbsp;3<br>Fri, 18th September 2009&nbsp; &nbsp; &nbsp;2&nbsp; 46&nbsp; &lt;-- end of week 6<br>Sat, 19th September 2009&nbsp; &nbsp; &nbsp;2&nbsp; &nbsp;8<br>Totals:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 209 912</tt></p></div> </blockquote><p>There is indeed a lull in the middle 3 weeks, so next year I do plan to try and reduce the availability of the surveys to at most 4 weeks. With the improvements to the evaluation and preparation code, it should then take about a week at most to review the data and then publish the results.</p><p>However, although my aim is to improve the turnaround of conference to results, there are a few other factors involved. All of my time taken to administer the surveys (before, during and after each conference) needs to accommodate my employer and family too. The conferences are typically around holiday season, so juggling everything to get the results done in a timely manner can be fun! Please bear this in mind if I don't fit into your personal schedule.</p><p> <b>Release The Bats</b> </p><p>It's always been my intention to release the code that runs the survey system as Open Source, however it means releasing two projects as Open Source, not one. The survey code is built on top of Labyrinth, a web management system I started developing in 2002 (2001 if you include the prototype). and is now used to run several CPAN Testers websites, as well as Birmingham.pm, GlousLUG and many other sites I run. Having a developer of 1 has meant it has taken a while for the code base to reach a stable state, which it has been now for a couple of years. It's not Catalyst or Jifty, although the core could potentially be abstracted and released as such. However, that's not what I'm going to release. Essentially I'll be releasing Labyrinth in two parts; the Perl library code and the supporting data files. On top of that I can then release the YAPC Survey code with the appropriate templates, plugins and additional data files.</p><p>Whether anyone then collaborates or not is another matter, but at least they'll have the chance to. People running smaller events will also have the ability to download, install and run the code themselves to administer their own surveys. </p><p> <b>Pretending To See The Future</b> </p><p>If you're an event organiser and think that the YAPC Survey system would be something you could use, please get in touch with any ideas you'd like to see featured. Also if you took any of the surveys this year, and think that they could be improved, whether it's simply rewording questions, adding questions, or some functionality that could be improved, please let me know.</p><p>I am very grateful for the support of all the conferences organisers, Eric Cholet for writing a new ACT API for me, and several speakers who have sent me personal thanks for supplying them with feedback to their talks, so it's obviously something that is being appreciated by the majority (though not all sadly). The split between the main survey and the talk/tutorial surveys seems to have worked well, and the majority of improvements so far have gone down well. There is still room for improvement, and hopefully the changes above will make for more feedback next year.</p> barbie 2009-10-06T13:02:52+00:00 journal CPAN Testers Summary - September 2009 - Wind And Wuthering http://use.perl.org/~barbie/journal/39715?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a>.</p><p>Last month was a fairly quiet month in terms of development, as projects such as the call for <a href="http://www.dagolden.com/index.php/458/call-for-proposals-cpan-meta-spec/">CPAN Meta Spec change proposals</a> were opened on the <a href="http://perl-qa.hexten.net/wiki/index.php/CPAN_Meta_Spec_Proposals">QA Wiki</a>, and the <a href="http://yapc-surveys.org/html/yn2009-survey.html">NA</a> and <a href="http://yapc-surveys.org/html/ye2009-survey.html">Europe</a> <a href="http://yapc-surveys.org/">YAPC Conference Surveys</a> were unveiled. However, there are some statistics that are planning to see the light of day soon, thanks to <a href="http://blog.timbunce.org/">Tim Bunce</a> putting together an updated <a href="http://go2.wordpress.com/?id=725X1342&amp;site=timbunce.wordpress.com&amp;url=http%3A%2F%2Fwww.slideshare.net%2FTim.Bunce%2Fperl-myths-200909">Perl Myths</a> talk for <a href="http://www.ossbarcamp.com/">OSSBarCamp</a> in Dublin last month. Expect to see them on the <a href="http://stats.cpantesters.org/">CPAN Statistics site</a> some time during the month.</p><p>The CPAN Testers have been continuing to make headway through the uploaded modules, and I'm also pleased to say that the builder keeping the <a href="http://www.cpantesters.org/">Reports</a> <a href="http://static.cpantesters.org/">sites</a> up to date, has been managing the page requests very well this month, despite such a large volume of reports being submitted and continued interest in the site.</p><p>After all the news featured in the <a href="http://blog.cpantesters.org/diary/45">August Summary</a>, it's not too surprising we've not had too much to report for September.</p><p>Last month we had a total of 161 testers submitting reports. The mappings this month included 22 total addresses mapped, of which 11 were for newly identified testers.</p><p>As I've mentioned previously, if you're planning to present a testing related talk at a forthcoming workshop or technical event, please let me know and I'll get it posted on here too.</p><p>That's all for this month's summary, so until the next post, happy testing<nobr> <wbr></nobr>:)</p> barbie 2009-10-05T19:10:25+00:00 journal ImageMagick / convert / --no-globbing! http://use.perl.org/~barbie/journal/39698?from=rss <p>I've recently reworked an image creation script at work to use <a href="http://www.imagemagick.org/">ImageMagick</a>'s own <a href="http://www.imagemagick.org/script/convert.php"> <code>convert</code> </a> script. The main reason being that text support is much better using <code>convert</code> than calling ImageMagick through its <a href="http://search.cpan.org/dist/PerlMagick/Magick.pm">Perl API</a>. However, it threw up a rather confusing issue, that has taken a while to track down and resolve.</p><p>The command issued for a number of images is along the lines of the following:</p><blockquote><div><p> <tt>convert -background "#ffffff" -fill "#000000" -font Helvetica -pointsize 14 -size 400x caption:"*" 9780596001735.png</tt></p></div> </blockquote><p>The 'caption' is the piece of text we wish to have in the image. In most cases this is 1 or 2 short sentences, but in some cases it can include a single asterisk, as above. This has the confusing result of creating many files with the text in each a different file name as found in the current directory. It's perhaps not too confusing to realise that filename expansion has occurred. However, the asterisk is quoted, so shouldn't be expanded by bash. After a bit of investigation and various attempts to check quoting in the shell, I discovered that ImageMagick's <a href="http://www.imagemagick.org/script/command-line-processing.php">own documentation</a> has this to say regarding <code>convert</code>...</p><blockquote><div><p> <i>In Unix shells, certain characters such as the asterisk (*) and question mark (?) automagically cause lists of filenames to be generated based on pattern matches. This feature is known as globbing. ImageMagick supports filename globbing for systems, such as Windows, that does not natively support it. For example, suppose you want to convert 1.jpg, 2.jpg, 3.jpg, 4.jpg, and 5.jpg in your current directory to a GIF animation. You can conveniently refer to all of the JPEG files with this command: </i> </p><blockquote><div><p> <tt>$magick&gt; convert *.jpg images.gif</tt></p></div> </blockquote></div> </blockquote><p>In the above command this would actually be expanded by the shell on Unix like systems, not <code>convert</code>, as there is no quoting around the asterisk. However, ImageMagick in its desire to be "helpful" gleefully ignores any quoting and does the filename expansion (or globbing as they call it) regardless. Hence why in my original command several hundred files could be generated.</p><p>Understanding this, I then set about trying to pass the character in hex form, escaping with '\' and quoting in a variety of ways to just get the single asterisk written as the text, all to no avail. I hunted through the online documentation (both on ImageMagick's site and in other forums) to find a solution, but drew a blank. I had expected to find a command-line option such as '--no-globbing' or similar, that would suppress the filename expansion feature, but alas no.</p><p>Through a bit of trial and error I finally discovered the solution:</p><blockquote><div><p> <tt>convert -background "#ffffff" -fill "#000000" -font Helvetica -pointsize 14 -size 400x caption:"* " 9780596001735.png</tt></p></div> </blockquote><p>Can you see the difference? All it took was a magic(k?) space character after the asterisk for <code>convert</code> to suppress the filename expansion! </p><p>Nowhere in the manual, docs and online help is there any mention of this. Perhaps I'm the first to encounter this, but I doubt it. As a way to help others who might also come across this frustration, I'm posting it here. I've submitted a bug report to the ImageMagick Wizards, so hopefully it may get considered for a future release. However, for now this looks to be the only way to get it working as intended.</p> barbie 2009-09-30T12:19:59+00:00 journal YAPC::Europe 2009 Survey - Results Now Online http://use.perl.org/~barbie/journal/39684?from=rss <p>At the beginning of August 2009, the 10th Annual YAPC::Europe took place. In the following weeks attendees were asked to complete surveys for the talks, tutorials and the conference as a whole. I'm pleased to announce that the results of the <a href="http://yapc-surveys.org/">YAPC Conference Survey</a> for the <a href="http://www.yapceurope2009.org/">YAPC::Europe 2009</a> event are <a href="http://yapc-surveys.org/html/ye2009-survey.html">now available online</a>.</p><p>The additional comments and suggestions given via the feedback forms have been sent to the organisers, as well as to next year's organisers, hopefully giving them the opportunity to refine their ideas to improve the conference experience for everyone in the future.</p><p>In addition, the results of the talk and tutorial evaluation forms have also been sent out to the respective speakers. If you were a speaker in Lisbon and haven't received an email from me by the end of today, check your spam box first, then contact me if you still haven't found anything.</p><p>My thanks to the organisers for letting me run the survey for YAPC::Europe this year, and many thanks to everyone who responded to the main survey, as well as all the evaluation surveys.</p><p>If you have suggestions for improving the surveys, please let me know.</p> barbie 2009-09-28T12:22:56+00:00 journal Birmingham.pm Technical Meeting - Wed 23rd September http://use.perl.org/~barbie/journal/39653?from=rss <code> Event:&nbsp;&nbsp;&nbsp;Technical Meeting<br> Date:&nbsp;&nbsp;&nbsp;&nbsp;Wednesday 23rd September 2009<br> Times:&nbsp;&nbsp;&nbsp;from 7pm onwards (see below)<br> Venue:&nbsp;&nbsp;&nbsp;The Victoria, 48 John Bright Street, Birmingham, B1 1BN<br> Details:&nbsp;<a href="http://birmingham.pm.org/tech/next">http://birmingham.pm.org/tech/next</a> <br> </code> <p> <b>Talks:</b> </p><ul> <li>Python Web Trends [Stuart Langridge]</li><li>Silverlight and Expression Studio [Nick Harewood]</li><li>Web Mashups with Catalyst [Jon Allen (JJ)]</li></ul><p> <b>Details</b> </p><p>This month we're lucky to have two special guest speakers; <b>Nick Harewood</b> and <b>Stuart Langridge</b>.</p><p>They'll be taking us beyond the realms of Perl, with Stuart talking about the latest web trends in the Python community, and Nick taking us through Microsoft Silverlight and Expression Studio. This will be a great opportunity to find out what's happening in other languages. In addition, JJ will be flying the Perl flag, with a talk about building web mashups with Perl and Catalyst.</p><p>As per usual, this month's technical meeting will be upstairs at The Victoria. The pub is on the corner of John Bright Street and Beak Street, between the old entrance to the Alexandra Theatre and the backstage entrance. If in doubt, the main entrance to the Theatre is on the inner ring road, near the Pagoda roundabout. The pub is on the road immediately behind the main entrance. See the map link on the website if you're stuck.</p><p>As always entry is free, with no knowledge of Perl required. We'd be delighted to have you along, so feel free to invite family, friends and colleagues<nobr> <wbr></nobr>;)</p><p>Some of us should be at the venue from about 6.00pm, usually in the backroom downstairs. Order food as you get there, and we'll aim to begin talks at about 8pm. Talks usually finish by 9.30pm, with plenty of time for discussion in the bar downstairs afterwards.</p><p> <b>Venue &amp; Directions:</b> </p><p> The Victoria, 48 John Bright Street, Birmingham, B1 1BN<br> - <a href="http://www.pubs247.co.uk/pubPages/pub.asp?id=24358&amp;from=&amp;promotion=">Pub Details</a> <br> - <a href="http://birmingham.pm.org/images/upload/victoria.jpg">Picture</a> <br> - <a href="http://www.google.co.uk/maps?hl=en&amp;client=firefox-a&amp;q=the+victoria+pub&amp;near=Birmingham&amp;radius=0.0&amp;cd=1&amp;cid=52482921,-1893619,7492755984503563963&amp;li=lmd&amp;z=14&amp;t=m">Google Map</a></p><p>The venue is approximately 5-10 minutes walk from New Street station, and about the same from the city centre. On street car parking is available see full details and directions on the <a href="http://birmingham.pm.org/tech/next">website</a>.</p><p> <b>Times:</b> </p><p>These are the rough times for the evening:</p><ul> <li>food available until 9.00pm</li><li>talks: 8.00-10.00pm</li><li>pub closes: 11.00pm</li></ul><p>Please note that beer will be consumed during all the above sessions<nobr> <wbr></nobr>;)</p> barbie 2009-09-21T08:09:31+00:00 journal YAPC::NA 2009 Survey - Results Online http://use.perl.org/~barbie/journal/39624?from=rss <p>For those interested, the results of the <a href="http://yapc-surveys.org/">YAPC Conference Survey</a> for the <a href="http://yapc10.org/">YAPC::NA 2009</a> event are <a href="http://yapc-surveys.org/html/yn2009-survey.html">now online</a>.</p><p>The additional comments and suggestions given via the feedback forms have been sent to the organisers, as well as to next year's organisers, to hopefully given them some ideas to improve the conference experience for everyone in the future.</p><p>In addition the results of the talk and tutorial evaluation forms have also been sent out to the respective speakers. If you were a speaker in Pittsburgh and haven't received an email from me by the end of today, check your spam box first, then contact me if you still haven't found anything.</p><p>My thanks to the organisers for letting me run the survey for YAPC::NA this year, and many thanks to everyone who responded to the main survey, as well as all the evaluation surveys.</p><p>If you have suggestions for improving the surveys, please let me know.</p> barbie 2009-09-14T12:58:09+00:00 journal Celebrating Our Ninth Birthday http://use.perl.org/~barbie/journal/39601?from=rss <p>Back in July 2000, Mike Bissett and I went to OSCON in Monterey. While there we attended a BOF about Perl Monger groups. When we got back, we had a look around and discovered the nearest PM group to Brum was a now defunct group based in Stafford. So that August we applied to run <a href="http://birmingham.pm.org/">Birmingham Perl Mongers</a>.</p><p>We held our first meeting in The Hogshead in the city centre, with Mike, myself and Mike's girlfriend. Humble beginnings, but we have since grown to a regular(ish) crowd of about 10 (more on a good day). We now hold regular technical meetings, as well as the traditional social meetings. We've had guest speakers and visitors from foreign parts (including Milton Keynes!), all of whom have been very welcome to eat, drink and chat with us.</p><p>Today, on the <b>9th September 2009</b>, we will be celebrating our <b>9th birthday</b>. For the geeks that's 09 on 09/09/09. We didn't plan it honest, and in fact I only noticed it when checking the date for the next meeting after last month's<nobr> <wbr></nobr>:)</p><p>If you're in or near Brum tonight and want to help us celebrate, please <a href="http://birmingham.pm.org/meet/next">feel free to drop by</a>, we'll likely be there all night<nobr> <wbr></nobr>;). We'll be at <a href="http://www.jdwetherspoon.co.uk/pubs/pub-details.php?PubNumber=5609">The Dragon Inn</a> on <a href="http://maps.google.co.uk/maps?q=B5%204TD">Hurst Street</a> (about 5mins walk from New Street station) from about 5.30pm onwards. Hope to see you there if you can make it.</p> barbie 2009-09-09T12:43:00+00:00 journal CPAN Testers Summary - August 2009 - Ten http://use.perl.org/~barbie/journal/39570?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a>.</p><p>This post is a bit of a celebration (you'll have to imagine the fireworks and associated oohs and ahhs), as last month we hit a few very notable milestones.</p><p> <b>The 5 Millionth Post</b> </p><p>Firstly, <em>Chris Williams</em> posted the <a href="http://nntp.x.perl.org/group/perl.cpan.testers/50000000">5 millionth post</a> on 13th August. While very notable, many posts to the cpan-testers mailing list are CPAN Upload messages and in the early years included discussions when the submission of reports was fairly low. So it doesn't equate to the 5 millionth report. However,<nobr> <wbr></nobr>....</p><p> <b>The 5 Millionth Report</b> </p><p>For our second milestone, <em>Oliver Paukstadt</em> posted the <a href="http://nntp.x.perl.org/group/perl.cpan.testers/5121936">5 millionth report</a> on 25th August. Considering we only had our 4 millionth post in June, it's been an incredibly short period of time to post 1 million reports. Over 300,000 of those reports came from <em>Dan Collins</em> and <em>Chris Williams</em> who both topped over 196,000 reports and 107,000 report respectively for July and August.</p><p>The sheer volume of processing was noted on the CPAN Testers server, as the page builder was put under load to consume the reports as quickly as possible. Thankfully it has been coping rather well and while some page requests have been delayed by up to 3 days at times (it takes over 5 days to rebuild the site from scratch!), the number of pages outstanding has been very manageable. This has largely been a good thing as I was on holiday last week, without any internet access, and I wouldn't have been able to do anything had something gone wrong. A testament to stability of the server these days.</p><p> <b>CPAN Testers is Ten Years Old</b> </p><p>Thirdly, yes we hit another milestone last month, and perhaps the biggest cause for celebration. <strong>CPAN Testers has been running now for TEN YEARS!</strong> On August 28th 1999, at 7:08am Pacific Standard Time, the <a href="http://nntp.x.perl.org/group/perl.cpan.testers/1">very first post</a> to the cpan-testers mailing was recorded, it was a CPAN Upload mail announcing Bundle-ABH had been released to CPAN. A few hours later, at 12:14pm Pacific Standard Time, the <a href="http://nntp.x.perl.org/group/perl.cpan.testers/6">very first report</a> was posted to the mailing list by Paul Schinder, an UNKNOWN report for FCGI-0.48 running on a Solaris box under Perl 5.5.3.</p><p>The idea of CPAN Testers was started by <em>Chris Nandor</em> and <em>Graham Barr</em> all those years ago, and has had many contributors to the eco-system since. Having said that notable mentions go to <em>Robert Spier</em>, <em>Ask Bj&oslash;rn Hansen</em>, <em>Andreas K&ouml;nig</em>, <em>Jos Bormans</em>, <em>L&eacute;on Brocard</em>, <em>Audrey Tang</em>, <em>Adam Foxson</em>, <em>David Golden</em>, <em>Chris Williams</em>, <em>David Cantrell</em>, <em>Slaven Rezi&#263;</em>, <em>Adam Kennedy</em> and I suppose it doesn't to hurt to mention myself (<em>Barbie</em>) too<nobr> <wbr></nobr>;)</p><p>My thanks to <a href="http://stats.cpantesters.org/testers.html">all the CPAN Testers</a> who have contributed reports over the years, whether they have submitted several thousand or just the one, it has all helped.</p><p> <b>YAPC::Europe 2009</b> </p><p>At the beginning of last month several CPAN testers attended <a href="http://yapceurope2009.org/">YAPC::Europe</a> in Lisbon. We had a BOF and discussed ideas, and hopefully recruited some new testers to our humble community. One potential tester may have access to some very under tested platforms, so here's hoping we can add those to our regularly tested platforms each month.</p><p>We also discussed statistics, following on from <a href="http://birmingham.pm.org/talks/barbie/stats-of-cpan-lt">my lightning talk</a> about the state of CPAN. The talk had been previously given in Pittsburgh for <a href="http://yapc10.org/">YAPC::NA 2009</a>, but had largely been overlooked. It seems giving it as a lightning talk provided a much greater impact and left a much longer lastring impression for European attendees.</p><p>Many of the statistics mentioned are now available on the <a href="http://static.cpantesters.org/">CPAN Testers Statistics</a> site, and I'm always happy to hear of more suggestions to include on there. The one overwhelm message taken from the talk is that CPAN contributions, both in terms of distributions and authors have been increasing consistently ever since CPAN was instigated back in 1995. New authors are signing up and contributing every month, and the existing contributors are not letting up either. Perl and CPAN have probably never been healthier. Which all goes to show just how important CPAN testers can be in helping authors maintain good quality and reliable code for the benefit of all.</p><p> <b>Promoting CPAN Testers</b> </p><p>It was an incredible month last month for CPAN Testers, with all the potential for announcements. Sadly, I was without an internet connection last week to be able to make announcements for the last two events at the time, as it would have been nice to have spread a bit of good karma around to everyone who has been involved.</p><p>Which raises a point about CPAN Testers that I have been asked several times over the last year. Why don't we shout out more about CPAN Testers, as according to some, it is one of the Perl Community's successful community projects. Sadly I'm only one person, and I don't shout very loud. I used to give talks about becoming a CPAN Testers regularly at YAPCs in North America and Europe, but these days look more to the BOF sessions to encourage interest. <em>Chris Williams </em>has started to continue the CPAN Tester howto style talks in Europe, but we could do with others around the world.</p><p>So if you're thinking of what talk to do at a forthcoming YAPC, OSDC, Workshop or local technical meeting, or even at a technical conference, workshop or local user group meeting that isn't focused on Perl, please consider whether you could help promote CPAN Testers. I'll gladly feature links to you, your slides and the event in a dedicated post on the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a> to help out. Likewise if you have written an article for a printed or online magazine or even in your own blog, please let me know and I'll mention it in a future post.</p><p> <b>And Finally</b> </p><p>Last month we had a total of 166 testers submitting reports. The mappings this month included 23 total addresses mapped, of which 13 were for newly identified testers.</p><p>Once again many thanks to all the several thousand CPAN Testers who have contributed reports, and here's to the next ten years<nobr> <wbr></nobr>:)</p> barbie 2009-09-02T18:30:55+00:00 journal The YAPC Surveys 2009 http://use.perl.org/~barbie/journal/39443?from=rss <p>This year, both <a href="http://yapc10.org/">YAPC::NA</a> and <a href="http://yapceurope2009.org/">YAPC::Europe</a> are using my survey software to administer the talk and tutorial feedback, as well as the regular Conference Survey. This is the first time I'll have run it for YAPC::NA, so the results are going to be interesting, particularly comparing them against views of both conferences.</p><p>The YAPC::NA surveys are currently due to close at the end of this week, but as I haven't had the time to send out reminders, I may well extend that by a week. So if you attended YAPC::NA and haven't submitted a response to the survey, please consider doing so. If you've lost your keycode email, please contact me and I'll sort you out a new one.</p><p>The attendees for YAPC::Europe needed no encouragement whatsoever, and the rate of submissions was staggering. Less than a day after the survey was opened in Lisbon, we had nearly as many responses as the YAPC::NA survey in the previous 5 weeks! I get that the Americans might be a little skeptical, but I was hoping for more than the 38% we currently have. YAPC::Europe responses, even after less than a week, are already over 42%! The first YAPC::Europe survey in 2006 received just under 54%, while the 2008 survey received just over 67% response. I'm hoping we can get over 70% this year, which will be fantastic.</p><p>The survey software itself is being cleaned up, and I'm hoping to be able to properly Open Source it within the next few months, as there are a lot of smaller workshops and conferences that I think may wish to take advantage of the codebase. While I'm happy to run the YAPC surveys, and collate them on the <a href="http://yapc-surveys.org/">YAPC Conference Surveys</a> website, trying to co-ordinate them all for everyone who wanted to operate a similar survey, would be a little too much work. Plus, it might help to get some feedback and patches to improve the surveys for the future.</p><p>Thanks to the organisers of YAPC::NA and YAPC::Europe for letting me run the surveys for them, and thanks to everyone who has contributed to the feedback. It all helps to improve YAPCs for everyone for the future.</p><p> <b>Update:</b> Just to give you some hard numbers...</p><blockquote><div><p> <tt>---------------------------------------------------<br>|&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| YAPC::NA | YAPC::Europe |<br>|-------------------------------------------------|<br>| Total Attendees&nbsp; &nbsp; &nbsp; &nbsp;|&nbsp; &nbsp; &nbsp; 284 |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 331 |<br>| Conference Survey&nbsp; &nbsp; &nbsp;|&nbsp; &nbsp; &nbsp; 108 |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 141 |<br>| Talk/Tutorial Surveys |&nbsp; &nbsp; &nbsp; 373 |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 640 |<br>| Open for (in weeks)&nbsp; &nbsp;|&nbsp; &nbsp; &nbsp; &nbsp; 5 |&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1 |<br>---------------------------------------------------</tt></p></div> </blockquote> barbie 2009-08-11T12:53:31+00:00 journal Another Data::FormValidator Filter http://use.perl.org/~barbie/journal/39438?from=rss <p>At <a href="http://yapceurope2009.org/">YAPC::Europe 2009</a> last week, I launched the Conference Survey during the final keynote, and almost immediately people began submitting their responses. I'll be posting more about the surveys later in the week, but this post concerns itself with a specific technical aspect.</p><p> <a href="http://search.cpan.org/~smylers/">Smylers</a>, being a rather clever fellow, likes to find the edge cases. He found one such edge case in the survey submissions, and although it wasn't a vulnerability, it was potential providing a misleading error to users. The problem arose due to the use of what are usually refered to as Microsoft "smart" characters. These are the characters that don't conform to standard Unicode character sets, as they use a range that is supposed to be reserved for control characters (see <a href="http://en.wikipedia.org/wiki/Windows-1252">Wikipedia</a> for more details).</p><p>Smylers had entered an en-dash character and some double quote characters from a Windows machine, and had attempted to submit one of the talk feedback forms. The result was a rather confusing error. The reason being that the backend of the survey system had deleted the field with the smart characters, because they were part of a range not accepted as string characters by the validation code, and flagged as an input error. The solution was to add a filter to the Data::FormValidator profile and translate the characters into something more sensible, before validating the input string. Which is what I did.</p><p>As a result <a href="http://search.cpan.org/dist/Data-FormValidator-Filters-Demoroniser">Data-FormValidator-Filters-Demoroniser</a> is now winging its way to CPAN. The code has been in the backend system for sometime, just not in the right place to pre-validate input strings. As it turned out it was much easier to abstract it and create a new module than rewrite some of the internal code.</p><p>My thanks to Smylers for initially spotting and reporting the bug, the guys behind <a href="http://search.cpan.org/dist/Data-FormValidator">Data::FormValidator</a> for making it so easy to add the filter, and <a href="http://www.justatheory.com/">Dave Wheeler</a> for already implementing many of the translations via his <a href="http://search.cpan.org/dist/Encode-ZapCP1252">Encode::ZapCP1252</a> module.</p> barbie 2009-08-10T13:23:21+00:00 journal CPAN Testers Summary - July 2009 - The Dark Side Of The Moon http://use.perl.org/~barbie/journal/39401?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a>.</p><p>July has seen several major changes to the backend code to running the <a href="http://www.cpantesters.org/">CPAN Testers Reports</a> website. The builder got some rules added to help manage the updates better and the front page now gets a status table. Tim Bunce suggested having some form of status indicator, and at the time some of the data needed to evaluate any indicators wasn't available. However, with some changes to the database, I was able to get the right information. As the site takes up to 5 days to get through all the changes, I had to wait a week before making the status table visible. At the moment the builder seems to have settled on running 2-3 days behind at most, and high profile pages often get built within a few minutes. The database now gets updated hourly, so there is now a much faster turn around of some of the pages. In addition to the status table, every author and distribution page now includes a timestamp of when the page was last built. If it's waiting in the queue to be rebuilt, at least you can now see how old the data available is. In many cases it will only be a few hours.</p><p>There are currently 25k+ pages on the site (double if you include the static site), with roughly 6 files being generated for each request. This is now streamlined as much as possoble, but I'm still looking at better ways to improve the process of creating them. With so much data now available it'll become more important to figure that one out.</p><p>Another change has been to the <a href="http://stats.cpantesters.org/">CPAN Testers Statistics</a> website. For the last year or so, I've been asked various questions about the data, and how different sets of criteria stack up. Some I already provide regarding the CPAN Testers, but there is also a lot of data regarding CPAN too. As such, there are now some extra pages included on the Statistics website, that give a current snapshop of the state of CPAN. I've been meaning to provide these CPAN graphs for some time, and was a suggestion given to me after a London Perl Workshop, in 2007! If there are any other trend graphs or stats tables that you think might be useful or interesting, please let me know. I have another couple of tables I'm planning to add, but I'm always open to adding more<nobr> <wbr></nobr>:)</p><p>This week I'll be at <a href="http://yapceurope2009.org/">YAPC::Europe 2009</a> in Lisbon. If you're a CPAN Tester and will be in Lisbon too, please come and say hello. I was disappointed to not meet a couple of people in Pittsburgh, though I did get to meet our 4 millionth poster to the cpan-testers mailing, <a href="http://stats.cpantesters.org/images/photos/yn2009-george-greer.jpg">George Greer</a>. So please don't be shy, just come and find me and introduce yourself<nobr> <wbr></nobr>:)</p><p>Last month we had a total of 171 testers submitting reports. The mappings this month included 22 total addresses mapped, of which 10 were for newly identified testers.</p><p>Much the same as last month, though congrats to Dan Collins, who has been outstanding in submitting reports. There was a time I thought no-one could compete with Chris, but it seems he does indeed have a competitor. However, for anyone thinking that we have all scenarios covered, please take a look at the monthly test coverage. We are still looking for a variety of platforms, and indeed some of the older perls too. Unfortunately we don't know what is out there in the wild, and concentrating on the latest and greatest is not really a true representation of what companies are actually still using. So if you want to get involved with CPAN Testers, and don't have an unusual platform to test on (if you do, *please* get involved), you can still help out by ensuring that new CPAN releases still work on older version of perl. You might be surprised to learn that deployment is still happening on 5.003! While the author might not be willing to support older versions, many users still need that information.</p><p>We're fast approaching the 5 millionth post to CPAN Testers, and expect to see it hit sometime during August month. With so many reports being posted at the moment, I'm also expecting the 5 millionth report to also hit slightly later in August. We shall see by next month's summary<nobr> <wbr></nobr>:)</p> barbie 2009-08-04T09:21:31+00:00 cpan Current Status of the CPAN Testers Reports Builder http://use.perl.org/~barbie/journal/39348?from=rss <p>Cross-posted from the <a href="http://blog.cpantesters.org/diary/42">CPAN Testers Blog</a> </p><p>If you don't normally hit the front page of the <a href="http://www.cpantesters.org/">CPAN Testers Reports</a> site (the dynamic one), you might not have noticed the visual updates made this weekend to the site. The change to the front page now allows you to see the current status of the builder, working away on the backend. The status update reports the number of requests in the queue and the request time of the oldest request. This should hopefully reduce the number of emails I get asking whether the server is working, and whether its coping well with the volume of reports.</p><p>At 19:10 on Saturday 25th July, the numbers looked like this:</p><blockquote><div><p> <tt>+---------------------+-------+-------+<br>| oldest&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | total | count |<br>+---------------------+-------+-------+<br>| 2009-07-23 14:01:39 | 11668 |&nbsp; 7900 |<br>+---------------------+-------+-------+</tt></p></div> </blockquote><p>From this you can see that the oldest request is just over 2 days old, there are a total of 11,668 requests waiting, of which 7,900 are unique. That's a low number, which is good. If the total goes over 20k or the oldest request is over 5 days old, then something has gone wrong. If that happens then feel free to send me an email point it out.</p><p>On the backend, some of the changes now ensure that the oldest requests are at least processed periodically, so that they don't stay in the queue too long. Also requests which usually have only 1 request in the total, are often quick hits, and every couple of hours, these get looked at too. It all means that the pages get manage quite nicely and the updates keep a nice balance across the whole site.</p> barbie 2009-07-25T17:25:27+00:00 journal YAPC::NA 2009 Survey - Update http://use.perl.org/~barbie/journal/39314?from=rss <p>After just a few weeks of launching the Conference Survey for the <a href="http://yapc10.org/">YAPC|10</a> event in Pittsburgh, we've had 107 people respond to the survey. With 284 registered attendees, that works out at roughly 37.6% response. While on the whole that isn't a bad return, our first survey at YAPC::Europe in 2006 got over 50%, and last years got nearly 70%, so the NA attendees need to do a bit of catching up.</p><p>In addition to the Conference Survey, this year each talk and tutorial has the ability to record feedback from attendees. I'm delighted to see we've had 355 responses to those.</p><p>The YAPC::NA survey closes on <b>14th August 2009</b>, so if you haven't completed the Conference Survey, please consider doing so. Also if you can still recall the talks and/or tutorials you attended, please consider completing those too.</p><p>If you attended YAPC|10, and have lost or didn't receive your email containing your keycode login, please contact me and I'll get it sent out to you.</p> barbie 2009-07-19T17:36:47+00:00 journal Test::JSON::Meta http://use.perl.org/~barbie/journal/39295?from=rss <p>A few days ago, Ricardo did <a href="http://rjbs.manxome.org/rubric/entry/1779">a braindump of his current projects</a>. One interested me regarding the META.json to compliment META.yml in CPAN package distributions.</p><p>As a consequence I started porting my META.yml test modules to create a META.json one. A few moments ago it got sent to PAUSE, so expect a new distro to my roster to hit a CPAN mirror near you shortly.</p><p> <a href="http://search.cpan.org/dist/Test-JSON-Meta">Test-JSON-Meta</a> </p> barbie 2009-07-15T18:39:01+00:00 journal No PHP here, mate http://use.perl.org/~barbie/journal/39284?from=rss <p>I'm not sure whether this is amusing or embarrassing:</p><p><div class="quote"><p>80.93.48.103 - - [13/Jul/2009:16:30:21 +0200] "GET<nobr> <wbr></nobr>/show//components/com_simpleboard/file_upload.php?sbp=http://quangpham.info/wp-<nobr>i<wbr></nobr> ncludes/images/blank.gif?? HTTP/1.1" 404 365 "-" "libwww-perl/5.803" 189 www.cpantesters.org</p></div><p>In case you're wondering, the above is an entry from the access logs on the CPAN Testers server. The script they are trying to access doesn't exist, and from what I can tell it's a poor attempt at crashing a server. The bit that amused me is that they're using LWP to run a PHP app. The bit that's embarrassing is that Perl is being used for undesirable purposes<nobr> <wbr></nobr>:(</p> barbie 2009-07-13T14:42:59+00:00 journal CPAN Testers Summary - June 2009 - The Nylon Curtain http://use.perl.org/~barbie/journal/39226?from=rss <p>Cross posted from the <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a>.</p><p>June saw a lot of work behind the scenes for CPAN Testers. At the end of the month <strong>David</strong> and <strong>Ricardo</strong> finally got to <a href="http://www.dagolden.com/index.php/261/metabase-opinions-from-anyone-about-anything/">release Metabase</a> to <a href="http://search.cpan.org/search?query=metabase&amp;mode=dist">CPAN</a>, the project key to moving towards CPAN Testers 2.0. If you're interested in helping out or finding out more, <a href="http://www.listbox.com/subscribe/?list_id=137267">join the mailing list</a>, or take a look at the current <a href="http://github.com/rjbs/metabase/tree/master">Github repo</a>. David has identified some of the areas still to be worked on, so if you have some tuits to help out, it would be very much appreciated.</p><p>The end of June also enjoyed the sun in Pittsburgh as part of YAPC::NA 2009, aka <a href="http://yapc10.org/yn2009/">YAPC|10</a>. While there were some testing related talks, there wasn't a specific CPAN Testers talk this year, or BOF. So much has been going into the work of getting the websites upgraded I never got the time to prepare a talk about it all. Next year hopefully we'll have a lot more to say about Metabase and the CPAN Testers 2.0 infrastructure. The talk I did do in Pittsburgh, <a href="http://birmingham.pm.org/talks/barbie/stats-of-cpan/index.html">The Statistics of CPAN</a>, did however highlight some very positive numbers about the state of CPAN. If nothing else it highlights that CPAN Testers has a lot of work to continue with for a long time to come. I'm looking at putting a number of the tables and graphs into the <a href="http://stats.cpantesters.org/">CPAN Testers Statistics website</a>, and if you have any suggestions for more, please let me know.</p><p>Following the changes in the <a href="http://www.cpantesters.org/">CPAN Testers Reports website</a>, the old domains now point to the static pages. Thanks to Ask, Robert and Jos for helping out with that. In doing so, a number of issues were pointed out that caused others problems. Specifically with the YAML files that are produced. Due to the vast number of reports now available, processing them is extremely time consuming. As a consequence to reduce the overhead, I ended up streamlining the data recorded in the YAML and JSON files, as several fields were either repeated or complete redundant. Unfortunately this has meant that some consumers of these files now are not able to process them correctly. As such there is now a new distribution on CPAN, <a href="http://search.cpan.org/dist/CPAN-Testers-WWW-Reports-Parser">CPAN-Testers-WWW-Reports-Parser</a>, which can be used to correctly parse a CPAN Testers YAML or JSON file or data block, and return the fields you want. It supports all the fields previously used and knows how to construct them all from the current data set. If you plan on using the CPAN Testers data for a future project, please consider using this to ensure any future changes are instantly picked with a simple upgrade.</p><p>Last month we had a total of 165 testers submitting reports. The mappings this month included 34 total addresses mapped, of which 17 were for newly identified testers.</p><p>Congratulations to <strong>Dan Collins</strong>, who managed to post over 89,000 test reports in a single month, the highest we've ever had. Unsurprisingly Chris wasn't too far behind<nobr> <wbr></nobr>:) I was also delighted to meet up with <strong>George Greer</strong> at YAPC|10, as for those that weren't aware, George took the honour of the 4 millionth post to the CPAN Testers mailing list at the end of May. A few days later, on June 7th, <strong>Serguei Trouchelle</strong> posted the 4 millionth accepted test report. Hopefully I'll get to meet Serguei at some point too. On average we have previously being seeing just over 200,000 reports posted each month, however, June saw 358,107 reports posted, a staggering amount of effort from all the testers.</p><p>The next summary will hopefully be posted during <a href="http://yapceurope2009.org/ye2009/">YAPC::Europe 2009</a> in Lisbon. If you're a tester and will be there too, please come and say hello</p> barbie 2009-07-04T13:45:28+00:00 journal CPAN Testers Stats - May Summary - Seven Stories Into Eight http://use.perl.org/~barbie/journal/39066?from=rss <p>Cross posted from the <a href="http://blog.cpantesters.org/diary/33">CPAN Testers Blog</a> </p><p>Quite a bit of activity has been happening in the last month, as you are no doubt aware if you are already reading this<nobr> <wbr></nobr>:) It marks a significant leap forward for CPAN Testers, which I hope continues.</p><p>First off David Golden managed to <a href="http://www.dagolden.com/index.php/164/kitty-hawk-moment-for-cpan-testers-20/">successfully input a report into the Metabase</a>, following the full process of testing, reporting, and submitting the report. The Metabase is the centre piece to the whole move to CPAN Testers 2.0, so this is a major step forward. There is still plenty to do, before CT2.0 is fully implemented, but it a great step towards the end goal. Well done to David and Ricardo.</p><p>The biggest visual change that happen last month was the <a href="http://blog.cpantesters.org/diary/32">facelift given to several of the CPAN Testers websites</a>. The new look had been on the cards to do for quite sometime, but it wasn't a high priority, as functionality changes took most of my time. However, with some enforced CFT, I finally got a round tuit (although sadly not one of those wooden ones I see every so often at conferences<nobr> <wbr></nobr>:( ). Prompted by a post by Adam Kennedy, I spent some time after doing the biggest functionality to look at ways to improve the look and feel. I found a design that looked suitable, and began to adapt it for the look and feel that was released. Despite some disappointed comments, the majority of feedback as been very favourable.</p><p>Part of the functionality changes include with the release of the new designs, was to now have a static site that reflects all the recent fixes to the underlying codebase, but without all the javascript extras available in the dynamic site. A number of people have been asking for a site that would enable them to switch from the old site, including some with issues with accessibility. As the old sites had not been updating since the end of March, the need for the static site grew. After a week bedding the new site in, the old URLs have now been moved to the new sites, so you will at least get redirected now. However, I would like to ask that if you have any reference to the old domains in code or documentation, or can update and wikis or online articles, please change to the new domains. If you don't have access or cannot update any changes, please let me know and I will try and contact the right author. The domains to change are:</p><ul> <li>testers.cpan.org =&gt; <a href="http://static.cpantesters.org/">static.cpantesters.org</a> </li><li>cpantesters.perl.org =&gt; <a href="http://static.cpantesters.org/">static.cpantesters.org</a> </li><li>perl.grango.org =&gt; <a href="http://stats.cpantesters.org/">stats.cpantesters.org</a> </li><li>cpantest.grango.org =&gt; <a href="http://wiki.cpantesters.org/">wiki.cpantesters.org</a> </li></ul><p>On the 30th March 2009, at around 2.15AM, the 4 millionth post to the cpan-testers mailing list was made. While we still have a little way to go for the 4 millionth report, it still marks an impressive milestone in the history of CPAN Testers. With just another 80,000 or so reports to go, we should reach the 4 millionth report sometime this month.</p><p>On the CPAN Testers Discussion mailing list, I happened to point out that on every page of the new site design, across all the sites, the footer now includes a reference to the Perl programming language. David Golden has now done the same for the reports from CPAN-Reporter, but he <a href="http://www.dagolden.com/index.php/221/cpan-testers-supports-perl-programming/">lamented that if only we'd been doing this from the start</a>. Thankfully I have a solution for that, as another site I'm planning on implementing is one to replace the NNTP web interface that is currently used. So that'll be 4 million web pages to add to the engines<nobr> <wbr></nobr>:)</p><p>We topped 144 testers submitting reports last month, so thank you again to everyone involved. The mappings this month included 36 total addresses mapped, of which 22 were for newly identified testers.</p> barbie 2009-06-02T12:37:36+00:00 journal Dig The New Breed http://use.perl.org/~barbie/journal/39031?from=rss <p> <b>Launch Day</b> </p><p>After much anticipation, I am delighted to finally announce the launch of all the new designs for the CPAN Testers family of websites. Or at least the ones I look after. This is a major step forward for the websites, as they have long been in need of a facelift to bring them into line with each other. While my design skills are now l33t by any stretch, hopefully they have served me well enough to give the sites a more professional and polished look.</p><p> <b>The New Designs</b> </p><p>If you haven't seen the sites yet, follow any of the links below. You can then click one of the options from the family navigation bar to see all the others<nobr> <wbr></nobr>:)</p><ul> <li> <a href="http://www.cpantesters.org/">CPAN Testers Dynamic Reports</a> </li><li> <a href="http://static.cpantesters.org/">CPAN Testers Static Reports</a> </li><li> <a href="http://wiki.cpantesters.org/">CPAN Testers Wiki</a> </li><li> <a href="http://stats.cpantesters.org/">CPAN Testers Statistics</a> </li><li> <a href="http://pass.cpantesters.org/">CPAN Testers Pass Matrix</a> </li><li> <a href="https://prefs.cpantesters.org/">CPAN Testers Preferences</a> </li><li> <a href="http://devel.cpantesters.org/">CPAN Testers Development</a> </li><li> <a href="http://blog.cpantesters.org/">CPAN Testers Blog</a> </li></ul><p> <b>The Static Reports</b> </p><p>As mentioned on several occasions now, there is now a static site version of CPAN Testers Reports. However, please bear in mind that some of the pages are huge and will take your browser several minutes to render. The static site is provide as a companion to the dynamic site for those who either wish to not use Javascript, or their method of viewing does not support it. All the same code is used to generate the pages of both sites, so rest assured that any future fixes will be replicated into both sites at the same time.</p><p>Once any potential issues are ironed out, I hope to get Jos to switch the DNS for the old testers.cpan.org to point to the new static site. As such, if you currently use the old site and feel there is anything missing from the new site, please let me know as soon as possible.</p><p> <b>The CPAN Testers Blog</b> </p><p>Another new addition to the CPAN Testers family of websites is the new CPAN Testers Blog. Initially this will just contain the summaries and some of the announcements (as ported from the blog on the old CPAN Testers Statistics site) that have been made over the past couple of years, but in time I hope to encourage others involved with CPAN Testers to contribute news and articles for inclusion.</p><p> <b>Feedback</b> </p><p>It seems like it's taken forever to these changes done, though in reality it has only been about 2-3 months. It has been an intense working regime, having spent most of my waking, non-employer working hours on the sites, and pretty much full-time during April. I'd personally like to thank Nicole, partly as an art advisor critiquing some of the colour choices, but mostly for putting up with me sitting on the sofa in the evenings for the past few months, while I've been getting all the changes done</p><p>As per usual, if there are any problems or (helpful) suggestions you have regarding the new design or a specific site, please let me know.</p><p>Now it's time for a rest!</p> barbie 2009-05-25T18:11:17+00:00 journal Cleaning up the Meta http://use.perl.org/~barbie/journal/39026?from=rss <p>Taking a break from waiting for the CPAN Testers updates (nearly done), I took some time to review some updates to the META.yml test distributions, which have had a couple of outstanding bug reports that should have been applied months ago. My thanks to David Golden and Jonathan Yu for their bug reports and patches.</p><p>I've also now added the repos to GitHub, so feel free to follow the repos and send patches if you have any suggestions for improvement.</p><ul> <li> <a href="http://search.cpan.org/dist/Test-CPAN-Meta">Test-CPAN-Meta</a> (<a href="https://github.com/barbie/Test-CPAN-Meta/tree">GitHub</a>)</li><li> <a href="http://search.cpan.org/dist/Test-YAML-Meta">Test-YAML-Meta</a> (<a href="https://github.com/barbie/Test-YAML-Meta/tree">GitHub</a>)</li></ul> barbie 2009-05-24T19:43:24+00:00 journal Waiting For The Big One http://use.perl.org/~barbie/journal/39017?from=rss <p>The new family is now sitting waiting patiently, all but the big one are up to speed and ready to say hello to the world. Unfortunately the big one is HUGE and despite ripping through data like a hot knife through butter, still has a large amount to get through.</p><p>The old reports site currently holds over 5.2GB of data files, and the new site is likely to be over 10GB. The builder behind the scenes is getting through roughly 200 data points an hour (producing 6 files for every data point). There are over 25,000 data points in total and after a day of processing is now down to 21,000 data points still to go. Understandably the database usage is going through the roof, but amazingly the live site doesn't seem to be suffering.</p><p>I'm hoping that over the weekend the builder will get through a large enough chunk of the remaining data points, so that I can finally do the official launch next week. Check back here after the weekend for an update<nobr> <wbr></nobr>:)</p> barbie 2009-05-21T21:41:42+00:00 journal Dynamic CPAN Testers Reports - Phases Two, Three &amp; Four! http://use.perl.org/~barbie/journal/38989?from=rss <p>After much talk about the ideas for improving the <a href="http://www.cpantesters.org/">CPAN Testers Reports</a>, I'm finally please to announce the completion of the work so far. There have been 3 significant changes to the site behind the scenes, all of which will hopefully improve the way everyone interacts with the site.</p><p>Phase Two was to finish off a number changes to enable a dynamic site. As it's turned out a full dynamic site hasn't been possible, as some of the pages still take several minutes to retrieve all the records from the database and render the pages. The 'ADAMK test' took over 30 minutes! As such a redesign of the caching was implemented that now means the pages are updated in the background. It means that the most frequently visited pages are more likely to be up to date now.</p><p>Phase Three was to implement a flat HTML site, that was similar to the old site, before the Javascript was included, but also benefits from all the fixes that have been implemented into the current site. As a result of the caching changes in Phase Two, this was actually extremely easy to implement, as it only meant adding templates.</p><p>With all that done, I was virtually ready to implement the sites on the live server. However, following a post by Adam Kennedy about the state of some of the Perl websites, I decided to hold back and concentrate on Phase Four. This last phase wasn't just planned for the reports sites, but aimed at covering all the CPAN Testers websites I'm responsible for. When I first created the Statistics website, I was more interested in making the the data and information available, and always figured I'd get around to designing a proper layout at some point. With me taking a month out during April, the time was ripe to completely redesign all the sites. So taking an inital design from the OSWD site, I amended it appropriately for my uses, and I have now converted all the CPAN Testers sites I look after across to the new design.</p><p>Hopefully the new design will meet with everyone's approval (Leon, fear not there is still an orange colour scheme in there<nobr> <wbr></nobr>:)), and the changes to the functionality improve the way people are able to use and access the site. For the time being the old path mappings should remain working, but I would advise moving to the new path structure when the sites go live.</p><p>So when are you going to get to see all these changes? Well very very soon. I'm currently setting up the new designs on the server and once everything is in place I'll be doing a symlink switch. Please be patient<nobr> <wbr></nobr>.. just a little while longer<nobr> <wbr></nobr>;)</p> barbie 2009-05-17T22:01:06+00:00 journal CPAN Testers Stats - April Summary - Hau Ruck http://use.perl.org/~barbie/journal/38925?from=rss <p> <a href="http://stats.cpantesters.org/">CPAN Testers Statistics</a> </p><p>So this month has mostly featured a lot of work after the QA Hackathon, without too many announcements. While there has been a lot of changes behind the scenes, we've mostly been getting on with stuff. David Golden and Ricardo Signes have been continuing with the Metabase, to the point David can now submit reports locally. There is still a bit of work needed to get the rest of the pieces all synced and working, but we are getting closer to CT2.0.</p><p>The mailers managed to highlight a fault recently, that has now been fixed, so if you've been wondering why the Summaries haven't been appearing, they should start filtering through again soon. <a href="http://stats.cpantesters.org/updates.html#April2009c">As announced during April</a>, there are now Weekly and Monthly Summary reports available, as well as the ability to receive individual mails again. Check the appropriate pages on the <a href="http://prefs.cpantesters.org/">CPAN Testers Preferences</a> website and update as you require. </p><p>As mentioned in <a href="http://use.perl.org/~barbie/journal/38904">my blog last week</a>, my time over the last month has featured work on the dual dynamic and static sites for the Reports. I'm pleased to say the underlying code is now complete. It will take a little while to carefully change the live system, as there are some significant database changes required, so I want to make sure the changes don't have any adverse affects. In addition, prompted by <a href="http://use.perl.org/~Alias/journal/38842">Adam Kennedy's blog post</a> about the state of many Perl websites, I took some to look at some designs and found one that looked perfect for the job. I've now reskinned several of the CPAN Testers sites and just have 1 publicly visible site left to do. There is another that has been waiting in the wings for some time, but I may wait until CT2.0 is available before unleashing it. However, with all the changes going on, there is one site that will be new (sort of), although it really is just a fork one of the existing sites. Seeing as I don't want to spoil the surprises, you're just going to have to wait for a little while longer to see all the results<nobr> <wbr></nobr>:)</p><p>We passed 3.5 million test reports last month, and although there were quite a number of reports posted last month, considering that last month CPAN had the most distributions submitted in a month ever (<b>1897</b>), it wasn't quite as many as I would have expected. Unfortunately the graphs on CPAN Testers Statistics have reached their Google Chart limit, so I'll be altering the graphs slightly for next month. </p><p>We topped 149 testers submitting reports last month, so thank you again to everyone involved. The mappings this month included 24 total addresses mapped, of which 10 were for newly identified testers.</p> barbie 2009-05-05T22:00:00+00:00 journal What Happened to April? http://use.perl.org/~barbie/journal/38904?from=rss <p>For those that might not be aware, I got made redundant on 31st March (the day after the QA Hackathon had finished). Thankfully, I start a new job next week, so I've managed to land on my feet. However, this has meant that I've ended up having the whole of April off to do stuff. My plan was to work on some of the Open Source projects that I'm involved with to move them further along to where I wanted them to be. As it turned out two specific projects got my attention over the last 4 weeks, and I thought it worth giving a summary of what has been going on.</p><p> <b>YAPC Conference Surveys</b> </p><p>Since 2006, I've been running the conference surveys for YAPC::Europe. The results have been quite interesting and hopefully have help organisers improve the conferences each year. For 2009 I had already planned to run the survey for <a href="http://yapceurope2009.org/">YAPC::Europe</a> in Lisbon, but this year will also see <a href="http://yapc10.org/">YAPC::NA</a> in Pittsburgh having a survey of their own.</p><p>The survey site for Copenhagen in 2008 added the ability to give feedback to Master Classes and talks. The Master Classes feedback was a little more involved, as I was able to get the attendee list, but the talks feedback was quite brief. As such, I wanted to try and expand on this aspect and generally improve the process of running the surveys. Part of this involved contacting Eric and BooK to see if ACT had an API I could use to automate some of the information. I was delighted to get an email back from Eric, who very quickly incorporated an API that I could use, to retrieve the necessary data to keep the survey site for a particular conference up to date, even during the conference.</p><p>With the API and updates done, it was time to focus on expanding the surveys and skinning the websites to match that of the now live conference sites. The latter was relatively easy, and only required a few minor edits to the CSS to get them to work with the survey site. The survey site now has 3 types of survey available, though only 2 are visible to anyone not taking a Master Class. Those that have taken one of the YAPC::Europe surveys will be aware I don't use logins, but a key code to access the survey. This has been extended so that it can now be used to access your portion of the survey website. This can now be automatically emailed to attendees before the conference, and during if they pay on the door, and will allow everyone to feedback on talks during the conference. On the last day of the conference the main survey will be put live, so you can then answer questions relating to your conference experience. </p><p>I'm hoping the slight change won't be too confusing, and that we'll see some ever greater returns for the main survey. Once it does go live, I'd be delighted to receive feedback on the survey site, so I can improve it for the future.</p><p> <b>CPAN Testers Reports</b> </p><p>Since taking over the <a href="http://www.cpantesters.org/">CPAN Testers Reports</a> site in June 2008, I have spent a great deal of time improving it's usability for users. However, it's come at a price. By using more and more Javascript to dynamically change the contents of the core pages, it's meant that I have received a number of complaints that the site doesn't work for those with Javascript disabled or who use a browser that doesn't implement Javascript. For this reason I had decided that I should create a dynamic site and static site. The problem with this is that the current system to create all the files takes several hours for each set of updates (currently about 16 hours per day). I needed a way to drive the site without worrying about how long everything was taking, but also add some form of prioritisation so that the more frequently requested pages would get updated more quickly than those rarely seen.</p><p>During April, JJ and I went along to the <a href="http://miltonkeynes.pm.org/">Milton Keynes Perl Mongers</a> technical meeting. One of the talks was about memcached and it got me thinking as to whether I could use it for the Reports site. Discussing this with JJ on the way home, we threw a few ideas around and settled on a queuing system to decide what needed updating, and to better managed the current databases to add indexes to speed up some of the complex lookups. I was still planning to use caching, but as it turned out memcached wasn't really the right way forward.</p><p>The problem with caching is that when there is too much stuff in the cache, the older stuff gets dumped. But what if the oldest item to get dumped is extremely costly on the database, and although it might not get hit very often, it's frequent enough to be worth keeping in the cache permanently. It's possible this could be engineered with memcached if this was for a handful of pages, but for the Reports site it's true for quite a few pages. So I hit on a slightly different concept of caching. As the backend builder process is creating all these static files, part of the process involves grabbing the necessary data to display the basic page, with the reports then being read in via the now static Javascript file for that page. Before dropping all the information and going on to the next in the list, the backend can simply write the data to the database. The dynamic site can then simply grab that data and display the page pretty quickly, saving ALOT of database lookups. Add to the fact that the database tables have been made more accessible to each other, the connection overhead has also been reduced considerably.</p><p>The queuing system I've implemented is extremely simple. On grabbing the data from the cache, the dynamic site checks quickly to see if there is a more recent report in existence. If there is, then a entry is added to the queue, with a high weighting to indicate that a website user is actually interested in that data. Behind the scenes the regular update system simply adds an entry in the queue to indicate that a new entry is available, but at a low weighting. The backend builder process then looks to build the entries with the most and highest weightings and builds all the static files, both for the dynamic site and the static site, including all the RSS, YAML and JSON files. It seems to work well on the test system, but the live site will be where it really gets put through its paces.</p><p>So you could be forgiven in thinking that's it, the new site is ready to go. Well not quite. Another part of the plan had always been to redesign the website. Leon had designed the site based on the YUI layouts, and while it works for the most part, there are some pages which don't fit well in that style. It also has been pretty much the same kind of style since it was first launched, and I had been feeling for a while that it needed a lick of paint. Following Adam's blog post recently about the state of Perl websites, I decided that following the functional changes, the site would get a redesign. It's not perhaps as revolutionary as some would want, judging from some of the ideas for skins I've seen, but then the site just needs to look professional, not state of the art. I think I've managed that.</p><p>The work to fit all the pieces together and ensure all the templates are correct is still ongoing, but I'm hopeful that at some point during May, I'll be able to launch the new look websites on the world.</p><p>So that's what I've been up to. I had hoped to work on Maisha, my other CPAN distributions, the YAPC Conference Survey data, the videos from the QA Hackathon among several other things, but alas I've not been able to stop time. These two projects perhaps have the highest importance to the Perl community, so I'm glad I've been able to get on with them and get done what I have. It's unlikely I'll have this kind of time again to concentrate solely on Open Source/Perl for several years, which in some respects is a shame, as it would be so nice to be paid to do this as a day job<nobr> <wbr></nobr>:) So for now, sit tight, it's coming soon...</p> barbie 2009-05-01T14:15:11+00:00 journal