TAP Archives are nice little packages of your test run that contain all the different TAP streams as well as test suite metadata. It's an easy way to store the results of a test run and then analyze them later. This is what Smolder uses but it's not specific to Smolder.
While it's easy to use TAP::Harness::Archive yourself to create your own harness it's kind of overblown for small projects. prove -a works, but sometimes you really want it as part of your standard perl Build.PL &&
Enter Module::Build::TAPArchive. Just use it instead of Module::Build in your Build.PL and then you can do
perl Build.PL &&
It's not doing anything really difficult behind the scenes, it should just make life a little easier for anyone who wants to play with TAP archives.
perl -e scenario. My new solution (with some help from rjbs) is actually even simpler than what I'm doing now. The downside is it's not really portable. Should work for most *nixes and I only really need it to work on Linux, so I'm good.
$ENV{LID_LIBRARY_PATH} = '/some/computed/path';
exec(split("\0", `cat/proc/$$/cmdline`));
$ENV{LD_LIBRARY_PATH} from your shell to point to the right place. But this is a bother, and not to mention really hard to do if you need some dynamic bits to figure out what the right path is.
LD_LIBRARY_PATH and have it be respected by the system's loader. It has to be done by the calling process (in most cases the shell).
LD_LIBRARY_PATH by relaunching the same program with the same arguments as a new sub-process. Simply put a BEGIN block like this in your script:BEGIN {
if(! $ENV{LD_LIBRARY_PATH} ) {
$ENV{LD_LIBRARY_PATH} = join(':', @some_paths_I_need);
my $self_cmd = "$^X $0 " . join(' ', @ARGV);
system($self_cmd);
exit;
}
}
Smolder 1.21 has just been released thanks to the Oslo Perl QA Workshop and my
trip sponsored by Oslo.pm and Linpro (Linpro was an amazing host for this event. They gave us free reign of their offices for the whole weekend, free food, drinks and travel). This releases fixes several installation
bugs on various platforms and other small bugs. It also adds several new
features. The highlights are:
+ Project categories have now been replaced by the more flexible smoke report tags
+ TAP archives can now be downloaded through the UI for a given report
+ Graphs can now be done on test run duration if that info was provided in the
archive. This was inspired by Ovid's work on the BBC's test suite's run time.
+ The default DB-Platform is now SQLite instead of MySQL
See the changle log for full details.
http://sourceforge.net/projects/smolder
I was also pretty heavily involved in the group working on the next version of TAP. I have to say that this gave me a new appreciation for standards committees. It's really hard to balance extensibility, backwards compatability and ease of use. Especially with TAP since one of it's goals is being extremely human readable. We definitely made some progress and all of the new, slightly less readable features are completely optional so if authors don't want to use them, they don't have to. But Smolder is going to have fun with all this new information coming it's way!
I found some time this weekend to do some more work on my TPF grant to improve Smolder. I'll be doing a presentation to the Raliegh Perl Mongers on Smolder the first week of June, so I'll need to find some more time pretty soon too. Plus I'll need to definitely finish this round of work before YAPC and OSCON.
The first milestone (ditching the old XML format for plain TAP) is now feature complete (with tests of course). The accepted format is now a TAR archive containing TAP files with one YAML file with META information about the test run. To see more documentation see http://smolder.svn.sourceforge.net/viewvc/smolder/trunk/docs/upload_file_format
The next 2 milestones have to do with setting up a system to easily automate CPAN style distributions for automatic testing from svn, etc. I've put this off for a few days but will get to it soon.
The 4th milestone was to add project specific RSS feeds. I was originally planning on using XML::RSS, but I've decided to go a simpler route. I don't need to parse RSS or do anything fancy other than emit XML using RSS tags. Using a template would be much easier. It also means I don't need to include the dependencies of XML::RSS inside of Smolder either, so it's that much smaller.
This milestone is now feature complete as well. That just leaves 2 left.
I was a little delayed in my work this week ($work and fun stuff) because my SVK repository got really screwy and wouldn't let me push changes back for certain branches. So I took a whole lot of time troubleshooting that and then finally just deleting everything and starting my svk mirrors all over again.
After that I was able to jump right back into the gutting of the XML stuff from smolder in favor or plain ol' TAP. This went pretty smoothly and I was able to get a nice an intial implementation working, so I committed it back to the svn repo. I was a little pressed for time, so the test suite is still broken, but that shouldn't take too much time to get into order.
While working with the code that I haven't touched in a while, I noticed a lot of other areas (not related to my grant work) that could use some sprucing up too. Plus there's lot of new tricks and tips I've picked up from my work on other stuff (mostly the AJAXification of Krang) that would be useful for smolder to. But I guess that's the way it goes. There's always something to learn and never enough time to backport those learnings to earlier projects.
After battling some sickness and a little bit of yak shaving, I've finally started work on my grant for smolder.
The first deliverable is to replace the custom XML schema that I used (which was based on the internal representation of a test run used by Test::TAP::Model. I know, I know... this was a really bad thing to do, but so it goes (and yes, Test::TAP::Model has changed their internal representation). I want to replace the XML with straight TAP, but there are a few complications. TAP doesn't currently support multiple files. Each TAP "stream" correlates to one test file and the harness keeps track of the results to show the viewer at the end. Since I want smolder users to upload 1 file per test run I need something to contain them. I could wrap all those TAP streams in some container format (YAML, JSON) but then I'm no better of than the XML route I currently have.
So I've decided that smolder will take a simple archive file (tar/tar.gz initially) which contains the various TAP files. Since we sometimes want metrics like "How long did this test run take" or "What order where these files run in" (since it doesn't have to be alphabetical) then there will also be an optional YAML (or JSON I haven't decided) file in there with this meta information.
So the first step was ripping out all the XML stuff currently in Smolder and it's reliance on Test::TAP::Model and friends. Then to replace it with a TAP parser (using TAP::Parser). This is currently about half way done, and I hope to find the time to finish it this week.
I'm pretty excited that my TPF Grant for improving Smolder has been accepted. Some of the improvements are minor, but others are pretty big. I've been watching the progress of TAP::Parser with enthusiasm since it means I can get rid of the YAML/XML transport hack that Smolder currently uses to get test results into the server. Hopefully this will also give some evangelism for TAP as a nice, simple and really human readable format for testing.
The grant also comes at a good time because my proposal for OSCon this year was accepted and I'll be talking about an updated testing toolkit for web applications. So Smolder will be all nice and shiny from the grant work before the talk. Which will be a nice chance to show it off and hopefully get more people using it.
I'll be updating this journal pretty regularly as I work on this grant, so stay tuned if you're interested.
I gave the first presentation on smolder at the first Pittsburgh Perl Workshop on Saturday. It was a nice little conference that was well worth the money (and not just because it was free to speakers
In case others are interested and missed it, my slides are found here.