use Perl

CPAN Testers Statistics

This month we hit the 2 millionth post to cpan-testers, thanks to Chris Williams. I congratulated him after his talk in Copenhagen, but he was more relieved to find out that the report itself was a PASS :) I still need to check which was the 2 millionth verified report, but I suspect Chris will claim that too. With such a high volume of reports, it's getting more urgent to move to something other than SMTP. There have been further discussions about updating the transport method and hopefully there should be some movement on that during the latter part of the year. In the meantime, if you're interested in getting involved in the way CPAN Testers gets reported in the future, feel free to join cpan-testers-discuss@perl.org, and add your thoughts.

Having taken over the code that runs the CPAN Testers Reports website, I have been hard at work behind the scenes updating all the code that generates the database and websites. As the CPAN Testers Statistics site worked from an enhanced version of the database, the first hurdle has been to merge the two and ensure both websites can be rebuilt from the same database. Despite a few hiccups with the NNTP server, that part of the process has gone very well. As a consequence several code releases of CPAN-WWW-Testers-Generator have already hit CPAN. The code for the Reports and Statistics site is currently being packaged and will be released soon too. The two sites meanwhile are being heavily tested with the new database, and again all seems to be running smoothly. For some thoughts into my plans for the future of the Reports site, see my use.perl posting.

Last month featured an excursion by many testers to Copenhagen, for the annual YAPC::Europe Perl Conference. Chris Williams presented 'Rough Guide to CPAN Testing', and we held a CPAN Testers BOF at the end of the first day. We had several people along to discuss several aspects of CPAN testing, and Adam Kennedy's PITA idea also got a lot of interest. After the BOF we managed to get a group photo of some of the testers. Next year I hope it will be an even bigger collection of testers :)

Both at the end of Chris' talk and during the first set of Lightning Talks, JJ and I announced that with the support of Birmingham.pm, we would be hosting the second QA Hackathon in Birmingham. We have already spoken to several people, who we hope will be able to attend, and we're hoping a few others will also be able to get involved too. More details will be announced in due course, but expect more CPAN Testers work to be involved in the hackathon.

Again we had over 100 testers submitting reports last month, with 49 new addresses mapped, of which we had 16 new testers identified. Once I've got the current work out of the way, my plan is to graph the number of new and core testers we see each month. I'm interested to see what effect the promotion of CPAN Testers at major events, or significant online promotions, have on the uptake of new testers. After that I hope to look at the platforms tested in a little more depth.

While trawling the back catalogue of articles on the NNTP server, in order to verify the database was building correctly, I uncovered a couple of posts from Brian King; FAQ First Draft Thread and FAQ Second Draft Thread. Interesting to see we still have the same basic principals of CPAN Testing now :)

Finally got through all my photos from Copenhagen. Unfortunately I'd only just bought my new camera, so many of the photos aren't as good as I would have liked. Still at least I have the memories :)

Enjoy!

So a couple of weeks ago, I was over in Copenhagen for YAPC::Europe. Putting on a YAPC::Europe event, or any YAPC event for that matter, takes up a lot of thought and preparation time before the event. The organisers want to put on the best conference they can and the previous organisers and the supporting team from the YEF Venue Committee try and help with documents, advice and the like every year, but we are all still learning. So it's understandable that sometimes we don't get everything right, or we miss something. Having said that, each year the organisers do try something new or build on previous ideas with the aim of making the whole event a better experience for everyone. This year the Copenhagen guys tried some new things, some worked some didn't, and built on ideas from previous conferences, again some worked some didn't. In between all that there were also a few areas that lapsed. Overall it was a good conference, but it also highlighted some areas we can improve on for the future. In the survey feedback, several have picked on the successes and pitfalls, and most so far have offered some very good feedback and offered suggestions and ideas where they can, which is great, and is part of the purpose of the survey.

However, I would say it's worth bearing in mind that this is a completely volunteer effort (i.e. we don't get paid), none of us are professional conference organisers and we largely do it because we're passionate about Perl and love to bring the European community together to share ideas and meet each other in person. If things go wrong, think about the positive ways to suggest improvements. We're all human, and don't necessarily remember some of the obvious things. I'm sure if the Copenhagen guys were to do it all again, there would be several alterations they would have made to their planning.

This year's event was slightly more subdued than normal, partly I think due to the growing contingent for whom this is the first or second YAPC::Europe. In some respects that is good, as it means we are encouraging new people to the event each year, however, it also potentially means we are losing those who only make it to their local YAPC::Europe. The latter happens at all the YAPC::Europe events, and I can understand several reasons why people don't return for a second or more year. However, some of the reasons are perhaps that the event wasn't quite what they were expecting. I'd like to help that and perhaps make it something more of what people are expecting. What follows are some observations that I took away from Copenhagen and are meant to be food for thought, partly for future YAPC::Europe conferences, but also for any conference. Some are small niggles, while others will need to be addressed more suitably.

Communication before the event wasn't the best and caused a number of people, particularly regulars, to not attend as it was far too late to arrange anything once they had confirmation of costs and the like. This was unfortunate and has emphasised how important a PR type person is needed in the organising team. The Copenhagen team were focused on getting many other aspects of the conference organisation tasks done, that promotion of the event fell behind. Plans are afoot to make the communication much easier in the future, to the point that Twitter accounts and the like have been created to keep the information flow going. But I would say that if anyone has questions for the organisers, please email them. If they've forgotten something, more often than not they'll be glad of the reminder.

The one thing that I'm really glad Copenhagen did, was to provide lunchtime bags of food. It worked very well in the main, and saved attendees the hassle of going out to the nearby shops to find food. However, it wasn't perfect, as unfortunately the bags for the vegetarian options were mixed with the regular bags, and meant a lot of searching through the bags was required. But it was something that could have been easily resolved, had the organisers been made aware of it. In a similar situation, a large number of us are tea drinkers and abhor coffee. Now while I'm sure some were delighted to see Earl Grey and some fruit infusions, many were a bit disappointed to not find any regular tea (it turns out they call it "Classic Tea" in Denmark). As such I made a request for more such tea bags were made available, and they were for the rest of the conference breaks. My suggestion is to try and resolve similar situations at the time, rather than wait until a survey comes out to criticise the organisers.

The venue itself was brand new, and as such made it a little difficult to find. One comment from the survey suggested doing as Birmingham had done (which in turn, we built on from the Braga guys), with pictorial references of how to get from the airport to hotels and the venue. It meant the attendees could find their way with little problem of taking the wrong route. Despite some confusion, everyone pretty much did find the right building eventually, but it would have been better to have provided some clearer signage from the Metro station to the venue itself. As a note to future organisers of any conference, pretend you've just landed in the country, have no guide book and cannot speak the language. This applies equally to English speaking countries too. Could you get to your hotel or the venue with the instructions you've provided?

As with Birmingham and Vienna, Copenhagen utilised the "plasma screen" code, that Birmingham had in turn pinched from the LUGRadio guys. Projected on the main wall outside the main auditorium, the hourly schedule, together with a regular drop of Flickr photos, were highlighted for all to see. Even if you didn't have the schedule to hand, you could easily see what was coming up. I hope future organisers take this further, as it was a useful aid to remember what the time was, and what was coming up next. Plus it was nice to see the photos that people had posted without having to check on Flickr yourself. As the organisers chose not have any printed proceedings, some have mentioned that having something to hand that expanded beyond the title of a talk, such as a talk abstract, would have given them a much better idea of what to see. In future it might be interesting to see whether ACT can provide a simple PDF of all the talks and abstracts, which attendees can print out for themselves. Although, it does depend whether the author actually does include a decent abstract of course.

Although the main auditorium and first two rooms were more than adequate for their size, the 4th room, which proved rather popular for several talks, was more of a classroom size. Thankfully none of the talks I saw there were over subscribed, thanks partly due to the spread of attendees across 4 rather than 3 rooms, but it did highlight that getting the popular talks out of smaller rooms is important. In Birmingham we suffered with this, despite trying to rearrange some of the talks at short notice. This year, just before the conference, Éric Cholet managed to add a feature to ACT that enabled attendees to select the talks they planned to see. While a fun feature, it's usefulness for future organisers is likely to be a life-saver. Having the ability to see which talks are likely to generate the most interest, will mean they have more options before the event to juggle the schedule to get all the talks into the right rooms.

Regards the talks themselves, there was a good selection of talks, covering all sorts of interests and levels of ability. However, it isn't always obvious from the title or abstract what level of knowledge is required for the talk, and whether it is an introduction, in-depth or discussion type talk. As a consequence it makes it a little difficult when there are several talks to choose from, which one is most likely to be most appropriate for you. Hopefully speakers can be prompted better in future to describe their talks a little better, so beginners don't attend expert talks and visa versa.

The 30 minute length for talks also didn't work for me. I was fortunate, as my Understanding Malware talk was able to span two sessions, but I was the only one to do so, and the talk needed that time slot to cover the subject matter. There were several other talks that I felt would have benefited from more time and equally several talks that really only merited 20 minutes, as the extra 10 minutes felt like padding. We had a cross-section of experienced speakers and new speakers this year, and while some of the newer speakers handled their presentations well, some didn't. At more corporate conferences, I've seen Mark Jason Dominus and Damian Conway give excellent presentations on how to give presentations. There are some techniques suggested by both that I now use in my own presentations. Now while those presentations are on the web somewhere, it would be wonderful to get those guys to do a special session, particularly for new speakers, so they can help all speakers improve both their presentation materials and the way they present the subject matter. Some of the new guys just need that little bit of guidance, as their subject matter is interesting.

The four tracks idea can work, but it can also spread talks too thin. This year, I didn't feel there was a lot of benefit to having four tracks, apart from making sure everyone who submitted a talk got accepted. Although I would encourage new speakers to propose talks, I do think the acceptance of talks from new speakers needs a bit of a review. Is the speaker qualified to speak on the subject? Have other similar talks already been submitted by more experienced speakers? We should also get a stronger weighting of experienced speakers rather than new speakers, as in the main most want to be taught something. General interest talks or discussion talks are worth having, but they're usually better handled by experienced speakers. As mentioned above, some of the new speakers did have interesting subject matter to present, so there are definitely some that would still get through the selection criteria, but I feel it's more important to have a strong line-up of talks, than necessarily having a slot for everyone who submitted a talk.

Regarding this last aspect, one post on use.perl, has annoyed me perhaps rather more than it should have. I have no idea who was supposedly promoting themselves rather a lot, as it wasn't obvious to me (or maybe it was me!). But what offended me is the inference that once you've presented a talk, that you should never speak again! Perhaps that wasn't the intention of the original poster, but it does bother me that people assume that they can see all 70+ talks throughout the whole 3 days. In previous years the survey has been useful to see who attendees would like to have seen, both in terms of talks they missed, or speakers that were unable to attend the event. Usually it highlights a few talks that would be worth repeating, following several requests to see them. Surely knowing you've seen a few talks before, gives you more opportunity to see the other talks! A good portion of the attendees have never seen those talks, and as far as they are concerned everyone is fresh blood. Anyway why shouldn't experienced speakers have the chance to give others the benefit of their knowledge? If they're good presenters, get them back as much as possible, both to present new and interesting ideas, but to also help new presenters of the future see how well it can be done. I also wonder whether the OP has ever submitted a talk? If you want fresh blood, put yourself on the spot, and don't expect others to give you a comfortable life.

There are some talks I've given, which having gone down well I've considered repeating every so often, but hearing such comments from people that they've seen it all before puts me off. However, probably over half of the attendance have never seen some of the talks I've previously presented. Do we really have to have completely fresh talks every year? Personally I don't think so and I think there is some value in repeating talks, especially if they cover a worthwhile subject for new attendees. I was actually pleased to have Chris Williams speaking about CPAN Testers, as it gives others a chance to carry on the promotion, and not just me. Had he not have done so, I would have done it, as I think reminding people about CPAN Testers and encouraging people to get involved is important. There are other projects that are equally deserving of promotion and involvement, and talks at YAPCs are an ideal opportunity to repeatedly promote them.

Something that is becoming more and more obvious regarding the scheduling these days, is that we are trying to cram far too much into the day. The early starts were a trial for many, as they were fighting late nights (including hangovers for some), jet-lag and the general intensive mental focus of the day. Remember that most of us don't get away from our computers that much during the general work-day, so focused training, discussion and socialising can be extremely exhausting. While it may well mean that you have to accept less talks, I would rather see more time given to discussion periods. One such casulty were The Birds of a Feather sessions this year as due to over-running it left little time after the main event to hold them. On Wednesday the CPAN Testers BOF went ahead, and was reasonably well attended, but after an hour of discussion, at 7pm it made for a very long conference day. It also meant we were far too late for the Vertical Metre Of Beer Party that Adam Kennedy and Jos Boumans were hosting. On Thursday, again due to over-running, there was no time left for any BOFs, as we were supposed to be at the Conference Dinner venue by 6.30pm. I was disappointed that the Perl Mongers BOF didn't happen, as from previous versions of this BOF, it has been a good sounding board to get new groups started, and energize groups into thinking about organising a YAPC::Europe themselves. This year we didn't get that chance.

Speaking of the socialising aspect, the pub that was chosen for the pre-conference meeting point, and subsequent free evenings, wasn't really suitable for the large number of people that came along. Understandably finding one pub or bar that can accommodate everyone is going to be difficult, but if there is anywhere that has a selection, so that attendees can hop from one to the other is often much better. This worked well in Braga, as in the town centre, even if people disappeared off for food, they all came back later and there was plenty of room outside for us to chat and drink. Of course it helps that Portugal mostly has decent weather! The pub suggested by the Copenhagen guys, The Globe, was an Irish pub, and wasn't too bad. Certainly several of the UK contingent liked the food and beer served up. But it didn't allow for everyone to easily mingle and chat. As a consequence, after meeting there for a first pint, people organised eating parties and headed off to other restaurants. It meant there wasn't really a place where people could meet up together, drink, chat and socialise into the evening and night. There were several people I never quite felt I got the chance to really chat to over the week, and would have loved to have spent more time just relaxing in good company.

Although it might sound like I didn't have a great time in Copenhagen, I did. These are just observations that I thought were worth sharing, rather than just in the feedback form of the survey. There was a lot of things that I really liked about the conference, but there were several things that I do hope future organisers, whether a YAPC or any other conference, might bare in mind. I am now planning to completely overhaul the conference documents that have been on Google Docs, as although a lot of the content is still relevant, there is more to add, and I want to better present it for organisers. At the moment it's a long read and I'm not convinced it gets digested as it currently is.

I would like to thank all the Copenhagen crew for their efforts. While there were some problems and lessons to be learnt, the event itself largely did go well. The crew were very friendly and helpful, there were some great talks, it was a good venue and I'm glad I finally got to visit the city of Copenhagen. I had a great time, and would love to go back to Copenhagen for a longer sight seeing visit.

My biggest disappointment of the event though, is completely and utterly my own fault. As with Vienna I had plans to make a note of people attending who I would like to meet in person and find them and say hello. Once again this year I failed miserably. So next time around, in case I forget, can Aristotle, Diego, George and Steffen come and say hello :) On the plus side there were a few people I bumped into, that I was rather glad to have a chat to, including Marcel and Charles. It's one aspect of a YAPC that I like, being able to put a person to a name.

Next year I'm sure that they'll be other things that I'll be thinking didn't go as planned, but I'm pretty sure I'll still have a great time regardless, as I did this year. I guess I just worry too much :)

Following on from the release of the YAPC::Europe 2008 Survey, I'm pleased to have just launched a companion set of surveys for the Master Class tutorials. Although the Course Assessment Form is perhaps not as verbose as the main survey, I've collated several ideas from other assessment forms to get a brief idea of what attendees thought of the course, to allow them to make suggestions for improvement, and to highlight what they thought was good about the course.

With currently over 60% of responses in from those polled on the main survey, I'm hoping that a good proportion, if not all, will respond to the feedback for the individual courses. The results will be collated and sent to the presenters once all their attendees have responded, or just after October 30th (the closing date). This is essentially a trial, as suggested by Dave Cross, to enable the tutorial presenters to get some form of feedback on the courses they presented before and after YAPC::Europe in Copenhagen.

It has also been suggested that each regular talk should have a similar feedback mechanism. However, whereas the tutorials required a signup, so we have their exact attendees, the regular talk sessions are a little more ad-hoc. It could be opened up as a free-for-all to enter what they like about a particular talk, but there is no guarantee that the respondent saw the talk. Having said that, several of the respondents have chosen not to rate the talks in the main survey, so it's likely to improve feedback for the presenter at the very least.

My intention was to roll this into ACT, but for the YAPC::Europe surveys at least, I'm not so sure that it needs to be. Having the survey independent of the ACT system (although I do use the attendee lists from ACT), and the fact that one person (me) is currently controlling access to that data to ensure anonymity, may actually be providing a degree of confidence in the privacy of comments, etc. I'll have to see what the Lisbon guys want to do, but I'll be happy to provide the same setup (if not more extensive) for the surveys next year.

For the past month I've been working on the code upgrades to CPAN Testers Reports, and since taking it over one of the aspects that has been requested several times has been to restrict the content that is seen in the browser. In some cases this is based on the author only wanting to see the FAIL reports, or a user only wishing to see reports that relate to their OS and/or version of Perl, etc. There are essentially 405 different ways of displaying the page for the combinations I'm currently working with, and I haven't yet worked out the OS and Perl version yet!

Because the reports site is currently static and rebuilt periodically, this method of refining results relies on the browser being Javascript enabled, changing the CSS settings as appropriate. However, with the sheer volume of data (a complete site rebuild now takes several hours), this isn't perhaps the most efficient way to actually produce the site anymore. With that in mind, I've been looking at ways to improve this, and the most obvious one is to change the reports site to a dynamic site. Only processing pages that are requested means there are less pages being rendered, which can be cached to reduce processing, and thus reduces disc space. Part of the problem at the moment is that the site takes up 3.7GB with 85,120 pages. Some may claim disk space is cheap, but with the current VMs I have access to that's a notable portion of my allocation, especially when you factor in the 17GB data store of all the NNTP posts since CPAN Testers started.

So, the next release of the code will likely be the last one for a static site. It will be released with all the requested fixes that I've been able to implement, and will hopefully clear out most of the current RT queue. There are still some fixes that need to be investigated, but in the main I think most people will be pleased with the changes so far.

After the next code release, the site generation code will be reworked to run as a dynamic site. The user will have the ability to set preferences to what they see, which will then be stored in a cookie, with the immediate benefit being that users returning to the site will repeatedly see only the content they wish to see. However, there is also another benefit. I had a discussion with JJ the other day and he was of the very strong opinion that the numbers that appear on each distribution page of search.cpan.org, regarding the number of reports it has, are at worst erronous and at best misleading. I agree. Currently I would rather just see a link to the appropriate page on the reports site. Being able to use the cookie preferences and an API to retrieve the content for that line of text, search.cpan.org, Kobes search, or any other CPAN search site of the future, need only send a request for that distribution, and a line can be returned that is specifically tailored to that user.

With the volume of reports having increased from a few thousand a month to hundreds of thousands a month in the last year or so, the daily rebuild gets out of date very quickly. Having a dynamic site would make it much easier to keep the content current. It does mean that the current implementation for the data store, using a SQLite database, isn't suitable for this kind on interaction. As such the knock on effect is that the generation code will now need to write to a database that is more agreeable to web delivery. This then leads on to the next major change that is planned for CPAN Testers.

Changing to a dynamic site will also mean that once we finally move to using the HTTP transport method for reports, and using the CPAN::Metabase API, I should be able to change the backend relatively simply to interface to whatever database we end up using to store the reports. Or rather the reports metadata. It also means the site can actually be used virtually realtime, rather than the current lag due to the SMTP and NNTP delays we currently experience. SMTP and NNTP have their own failings, which have been a constant source of frustration to Ask and Robert, who have been very kindly looking after all this all this time. Moving to a more reliable data store (and therefore recovery mechanism) will also mean that we shouldn't lose reports and really REALLY long reports don't get truncated, meaning all that really useful 'perl -V' information isn't lost either.

It all basically means that a lot of work on the backend is expected to happen over the next year. My plan was to get a lot of this working for the reports site by the end of the month, but that is now unlikely to happen. The next release of the static site will be done by then, but the dynamic site will take longer to develop.

If you have ideas for further improvements to the site, please post them to the RT Queue. If you have general ideas for improving CPAN Testers, including the CPAN::Metabase API, please consider joining the cpan-testers-discuss mailing list and sharing them with most of the CPAN Testers developers.

Yesterday I sent out the invite mails for the YAPC::Europe 2008 Survey. In roughly 24 hours we already have 85 out of 284. Outstanding! Seeing as people are still in the process of returning home and unwinding, I expect there will be plenty more submissions this coming week.

Following a comment from one speaker, I wanted to follow up a point about the survey. Towards the end of the survey is some feedback regarding each talk. Although I was planning to share this feedack with the organisers (2008 & 2009), and the speakers of the respective talks, I wasn't intending that part of the survey to be public. Also keynotes were purposefully missed out.

I have also had some suggestions to improve the survey for next year, which I'm planning on doing, if next year's organisers would like me to.

This year's survey will close at midnight 30th October 2008, and I'll try and get the data to the appropriate people as soon as possible after that. Hopefully the followup reports will be published soon after.

CPAN Testers Statistics

First off an apology. It seems that I used a legitimate email address in a rant back last year, about the use of tester email addresses. At the time, the address didn't appear to be real and had given the impression that the address was being used as a spamtrap. The owner of the address has since contacted me to inform me that it is real, and that it doesn't get used as a spamtrap. So my sincere apologies to Rob for casting doubt on his email address. However, the address in question had caused confusion, so Rob has kindly altered the address he now sends from, so that authors can feel more comfortable about sending any queries to him. Many thanks to Rob for this.

At the beginning of the month, I launched a new domain for CPAN Testers, cpantesters.org. The new domain will hopefully prove more helpful in remembering all the sites used in the CPAN Testers family of sites, and maybe even start including a whole bunch of new ones. In order to make it even more easier to find other sites, linking between the sites with a standard dropdown is now planned, thanks to the suggestions of a few people. The reports site is currently receiving an update and will be rsync'ing to the public site soon.

August sees many of us here in Europe heading to Copenhagen, for the annual YAPC::Europe Perl Conference. On the first day of the event, on Wednesday 13th August, Chris Williams will be presenting 'Rough Guide to CPAN Testing', with a CPAN Testers BOF planned at the end of the day, so be sure to come along to find out more. I'm also hoping to add to my collection of photos of CPAN Testers, so please make sure you come and say hello to me during the conference if you are a CPAN Tester.

As mentioned last month, Leon has given me the go ahead to update the code behind the scenes for the data collection and creation of the reports site. The first part of the switch over was to merge the code that does the data collection, with the previously forked code that is currently used by the Statistics site and CPAN Dependencies site. When I forked, there were a few fields that weren't previously being recorded that were needed (e.g. the tester email address) and a few that weren't used by the stats site that were dropped. Rather than having two separate systems it makes sense to merge these changes. Plus in the last month or so, I've picked up on reports that were sent in Base64 or Quoted-Printable that were previously not being record properly. There are some further changes, such as the fact that the stats database also parses the UPLOAD articles, and a whole bunch of new tests, which have resulted in quite a nicely packaged little distribution. Before launching it on the world, I'm running it in by generating the database from scratch for all articles. Originally I was going to drop the creation of the articles database, which is held locally, but decided to keep it, as it makes look ups a lot quicker. However, it's currently over 7GB and is still not half of the way through the rebuild! The lastest complete stats database is just over 400MB, and is much more manageable. Thankfully disk space isn't an issue yet, but as a consequence I've made the creation of the articles database optional. Assuming anyone else is mad enough to try and generate this stuff ;)

This time next month, I hope to having all the CPAN Testers sites running off the same database, and will hopefully have the updated reports site online for all. Thanks again to everyone for the patches, ideas and support. It's much appreciated.

Again we had over 100 testers submitting reports last month, with 43 new addresses mapped, of which we had 7 new testers identified. After David Golden's magnificent effort last month, his bots have settled down now to a more sedate pace. As a consequence Chris has once again taken the top spot this month, and has even posted his highest number of reports in a single month. No doubt he's been getting his daughter to help out while he's been at work ;)

Seeing as there several posts here telling everyone that they're off to OSCON, I thought I'd mention that I've just been to LUGRadio Live :)

For more of a run down of the event see my personal blog ... with some teaser photos.

As I've recently taken over a distribution relating to the CPAN Testers, I've been looking to increase it's test coverage. I have several more samples and tests to add to the suite, and wanted to see what the results were. The following are all using the lastest version, 0.64, from either CPAN or a PPM repository.

The first extract is on Windows, from a PPM installation, as I don't have a compiler on the machine. The results are annoying and misleading. Annoying as it's choosen to include everything bar the kitchen sink. According to the docs, you can use -ignore and -inc to get rid of these, only -ignore doesn't work and I get a 'non-existant option' error for -inc. It's misleading as the two items at the bottom of the table are the bits I'm interested in, however, the results are useless as there is nothing to indicate what the real coverage is for either. The second to last line isn't the coverage for that module.

C:\wip\distros\CPAN-Testers\CPAN-WWW-Testers-Generator-0.23>cover
Reading database from C:/wip/distros/CPAN-Testers/CPAN-WWW-Testers-Generator-0.2
3/cover_db
Devel::C over: ignoring extra subroutine
Devel::Cover: ignoring extra subroutine
Devel::Cover: ignoring extra statement
Devel::Cover: ignoring extra statement
Devel::Cover: ignoring extra branch
Devel::Cover: ignoring extra branch
Devel::Cover: merging data for lib/CPAN/WWW/Testers/Generator/Article.pm into bl
ib/lib/CPAN/WWW/Testers/Generator/Article.pm

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
.../lib/ActivePerl/Config.pm    0.0    0.0    0.0    0.0    0.0    n/a    0.0
...l/lib/ActiveState/Path.pm    0.0    0.0    0.0    0.0  100.0    n/a    4.8
C:/Perl/lib/AutoLoader.pm       0.0    0.0    0.0    0.0    0.0    n/a    0.0
C:/Perl/lib/B.pm               14.6   16.7   16.7   12.5    n/a   93.6   15.0
C:/Perl/lib/B/Debug.pm          0.0    0.0    0.0    0.0    n/a    n/a    0.0
C:/Perl/lib/B/Deparse.pm        0.1    0.0    0.3    0.2    0.7    1.8    0.2
C:/Perl/lib/Carp.pm            10.5    0.0    n/a   22.2    0.0    0.0    9.8
C:/Perl/lib/Carp/Heavy.pm       6.6    0.0    0.0   16.7    0.0    0.0    4.7
C:/Perl/lib/Config.pm          52.6   40.0    n/a   40.0    n/a    0.0   47.1
C:/Perl/lib/Config_heavy.pl     0.0    0.0    0.0    0.0   66.7    n/a    4.1
C:/Perl/lib/Cwd.pm              3.3    0.8    0.0   11.1   25.0    0.0    2.6
C:/Perl/lib/Data/Dumper.pm      0.0    0.0    0.0    0.0   25.0    n/a    1.1
C:/Perl/lib/Digest/base.pm      0.0    0.0    n/a    0.0    0.0    n/a    0.0
C:/Perl/lib/DynaLoader.pm      70.0   50.0   66.7   50.0   33.3    1.0   61.4
C:/Perl/lib/Errno.pm           39.4    0.0    0.0   50.0    n/a    0.0   29.0
C:/Perl/lib/Exporter.pm        52.1   71.4   56.8   44.4    0.0    0.3   52.2
...erl/lib/Exporter/Heavy.pm   14.8   16.3   31.3   33.3    0.0    0.2   16.7
C:/Perl/lib/Fcntl.pm            0.0    0.0    n/a    0.0    0.0    n/a    0.0
C:/Perl/lib/File/Basename.pm   28.9   18.1    9.1   60.0  100.0    0.0   24.9
C:/Perl/lib/File/Find.pm        0.0    0.0    0.0    0.0   25.0    n/a    0.2
C:/Perl/lib/File/Glob.pm        0.0    0.0    n/a    0.0   20.0    n/a    1.1
C:/Perl/lib/File/Path.pm       70.2   48.8   29.4  100.0    0.0    0.0   53.5
C:/Perl/lib/FileHandle.pm       0.0    0.0    n/a    0.0    0.0    n/a    0.0
C:/Perl/lib/IO.pm               0.0    0.0    n/a    0.0    n/a    n/a    0.0
C:/Perl/lib/IO/File.pm         59.1   36.4   28.6   66.7  100.0    0.0   46.9
C:/Perl/lib/IO/Handle.pm        7.2    2.6   11.1   10.3   21.6    0.3    8.0
C:/Perl/lib/IO/Seekable.pm      0.0    0.0    n/a    0.0  100.0    n/a   16.7
C:/Perl/lib/IO/Socket.pm       14.1    1.0    1.9   25.8   41.7    0.0   11.5
...erl/lib/IO/Socket/INET.pm   12.4    0.0    0.0   27.3   70.0    0.0    8.4
...erl/lib/IO/Socket/UNIX.pm   30.0    0.0    0.0   42.9   75.0    0.0   20.0
C:/Perl/lib/MIME/Base64.pm    100.0    n/a    n/a  100.0    n/a    0.0  100.0
...l/lib/MIME/QuotedPrint.pm  100.0    n/a    n/a  100.0    n/a    0.0  100.0
C:/Perl/lib/Net/Cmd.pm          4.5    0.6    0.0   15.4   66.7    0.0    6.4
C:/Perl/lib/Net/Config.pm      42.9    0.0    n/a   80.0  100.0    0.0   37.0
C:/Perl/lib/Net/NNTP.pm         6.9    0.0    0.0    8.4   90.2    0.0    8.1
C:/Perl/lib/Pod/Find.pm         0.0    0.0    0.0    0.0  100.0    n/a    1.3
...l/lib/Pod/InputObjects.pm    0.0    0.0    0.0    0.0    n/a    n/a    0.0
C:/Perl/lib/Pod/Parser.pm       0.0    0.0    0.0    0.0    0.0    n/a    0.0
C:/Perl/lib/SelectSaver.pm      0.0    0.0    0.0    0.0    0.0    n/a    0.0
C:/Perl/lib/Socket.pm          56.3   10.0    0.0   75.0  100.0    0.0   43.1
C:/Perl/lib/Storable.pm         0.0    n/a    n/a    0.0    0.0    n/a    0.0
C:/Perl/lib/Symbol.pm          10.8    0.0    0.0   16.7    0.0    0.0    6.3
C:/Perl/lib/Win32.pm           26.9    0.0    0.0   16.7    6.7    0.0   15.8
C:/Perl/lib/XSLoader.pm        66.7   50.0   22.2   50.0   50.0    0.2   56.5
C:/Perl/lib/base.pm            39.5   35.7   36.8   57.1    0.0    0.2   37.6
C:/Perl/lib/blib.pm            72.2   37.5   33.3  100.0    n/a    0.0   60.0
C:/Perl/lib/bytes.pm            0.0    0.0    n/a    0.0    0.0    n/a    0.0
C:/Perl/lib/constant.pm        38.0   22.2   25.0   25.0    n/a    0.0   30.4
C:/Perl/lib/integer.pm         50.0    n/a    n/a   50.0    0.0    0.0   40.0
C:/Perl/lib/lib.pm             57.9   26.9   33.3   83.3    0.0    0.0   48.5
C:/Perl/lib/overload.pm         0.0    0.0    0.0    0.0   18.2    n/a    1.3
C:/Perl/lib/re.pm               0.0    0.0    0.0    0.0    0.0    n/a    0.0
C:/Perl/lib/strict.pm          84.6   62.5   50.0  100.0    0.0    0.0   71.4
C:/Perl/lib/utf8.pm             0.0    0.0    n/a    0.0    0.0    n/a    0.0
C:/Perl/lib/vars.pm            44.4   45.5    0.0  100.0    n/a    0.2   40.4
C:/Perl/lib/warnings.pm        10.5    7.1    0.0   12.5   50.0    0.0    9.1
.../lib/warnings/register.pm  100.0   50.0    n/a  100.0    0.0    0.0   89.5
.../lib/CPAN/DistnameInfo.pm   68.3   57.7   11.1   41.7   81.8    0.0   54.6
...ite/lib/Class/Accessor.pm   35.3   17.6   15.2   30.4   85.7    0.2   32.0
...ib/Class/Accessor/Fast.pm   43.5   30.0    n/a   50.0  100.0    0.0   45.5
...rl/site/lib/DBD/SQLite.pm   34.8   11.9    0.0   53.8    0.0    0.2   21.6
C:/Perl/site/lib/DBI.pm        19.3    8.3    7.6   27.4    n/a    0.2   14.4
...site/lib/Devel/Symdump.pm    0.0    0.0    0.0    0.0  100.0    n/a    2.2
.../site/lib/Email/Simple.pm   91.4   50.0   40.0   94.4  100.0    0.0   85.0
...ib/Email/Simple/Header.pm   35.2   23.7   16.7   38.5  100.0    0.3   33.5
...ite/lib/File/Spec/Unix.pm    0.0    0.0    0.0    0.0  100.0    n/a    6.6
...te/lib/File/Spec/Win32.pm   18.7   11.4    0.0   23.1  100.0    0.0   20.1
.../site/lib/Pod/Coverage.pm    0.0    0.0    0.0    0.0  100.0    n/a    2.8
.../Coverage/CountParents.pm    0.0    0.0    n/a    0.0    n/a    n/a    0.0
...l/site/lib/Scalar/Util.pm    0.0    0.0    0.0    0.0   50.0    n/a    2.2
.../site/lib/Test/Builder.pm   49.6   26.8   26.3   65.2  100.0    0.5   44.4
...ib/Test/Builder/Module.pm   90.6   50.0   33.3  100.0  100.0    0.0   86.7
...te/lib/Test/MockObject.pm   42.0   14.3   26.8   36.5   84.6    0.0   39.1
...erl/site/lib/Test/More.pm   24.7    7.5    4.8   36.6   95.5    0.0   22.6
...rl/site/lib/Time/Local.pm   38.4    0.0    2.8   60.0  100.0    0.0   25.0
.../WWW/Testers/Generator.pm   90.5   58.3   69.2  100.0  100.0    1.1   81.7
...ters/Generator/Article.pm    n/a    n/a    n/a    n/a    n/a    n/a    n/a
Total                          15.5    7.2    5.8   19.2   30.3  100.0   13.0
---------------------------- ------ ------ ------ ------ ------ ------ ------

So okay that was Windows. Just to see what would happen, I copied the code across to a Linux box and reran the coverage tests. The following is what I get. All the lines about core or installed modules have disappeared, but I still get the misleading lines. In fact despite the same code being run, notice the numbers are different for the Generator.pm line.

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
.../WWW/Testers/Generator.pm   89.4   57.8   75.0  100.0  100.0  100.0   81.1
...ters/Generator/Article.pm    n/a    n/a    n/a    n/a    n/a    n/a    n/a
Total                          89.4   57.8   75.0  100.0  100.0  100.0   81.1
---------------------------- ------ ------ ------ ------ ------ ------ ------

After playing with the test files a bit, I discovered that removing one specific file, produces the following results:

---------------------------- ------ ------ ------ ------ ------ ------ ------
File                           stmt   bran   cond    sub    pod   time  total
---------------------------- ------ ------ ------ ------ ------ ------ ------
.../WWW/Testers/Generator.pm   28.9    4.7    8.3   58.8  100.0    7.7   24.4
...ters/Generator/Article.pm   96.1   58.7   42.9  100.0  100.0   92.3   81.9
Total                          60.6   27.3   32.5   78.1  100.0  100.0   52.1
---------------------------- ------ ------ ------ ------ ------ ------ ------

Now that's more like it. However, the one file that I removed contains a large part of the tests for the first module. The only thing I can think that might be causing this is that I'm using Test::MockObject to avoid network usage. There is nothing special about the test file, it runs some samples through the system and creates a couple of files. I would consider submitting this as a bug, both for the spurious inclusion of installed modules and the dodgy coverage lines, but is this a bug, or something about the way I'm using Devel::Cover? Does it have a problem with Test::MockObject? Is there a way to suppress any recording of installed modules?

I've now edited the offending test script, editing out any reference to Test::MockObject and I still get the misleading lines, so it isn't that. I'll carry on writing tests, but it's annoying that I can no longer use Devel::Cover to see what I'm missing :(

I've tried searching Google and Perlmonks for posts about similar issues and haven't found anything. Have I really found an issue (or rather issues) that no-one else has? If you've come across these issues, please let me know, preferably with links to solutions, otherwise if they are new, I'll try and figure how better to submit them to RT.

For the last two years we've run a survey after YAPC::Europe, which has had a very positive response. It's also helped to generate new ideas about what people want from the conferences and provides good feedback to past and future organisers, of what worked and what didn't. As mentioned in a previous post, I'm trying to think of questions that would be suitable to understand how newcomers felt about the conference. I've been pinching ideas from other surveys and think I've got a decent set of survey question ready this time around, so I'm looking forward to seeing the results.

The new survey backend has been rewritten, as I originally wrote it purely in HTML and all the form field names where handcrafted, requiring a lot of effort in the backend. The rewrite was much easier in the end than I expected. The survey file is now written in YAML, with a fairly straightforward schema, which I'll get round to documenting at some point, and the backend to create the HTML and save the form field entries is mostly centred on parsing the YAML and the CGI parameters. The rest hangs off all the previous work I've already done with Labyrinth. My long term goal was to plug this into ACT, so that organisers would only need to flip the switch and have all the survey codes generated and mails sent to attendees automatically. It probably will eventually get into ACT, but for YAPC::Europe 2008 I'll be running from my system.

However, this year I plan to flip the switch on the last day of the conference and not a couple of weeks later, as a common complaint from attendees is that by the time they receive the notification, they've forgotten a lot of what they wanted to say. Not that that has stopped many attendees giving us feedback :) As in previous years, I'll generate all the nice graphs and provide the templates for the documents for the Copenhagen guys, so they can publish their own report afterwards, but I'm hoping these tools can also be automated in the future, so that future conferences and workshops can have a survey as just a standard part of their attendee experience.