dagolden's Journal http://use.perl.org/~dagolden/journal/ dagolden's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:14:25+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 dagolden's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~dagolden/journal/ Call for proposals: CPAN META Spec http://use.perl.org/~dagolden/journal/39539?from=rss <p> The CPAN META Spec is the basis for the META.yml metadata files included in most modern CPAN distributions. Since the spec was last updated two years ago, there have been suggestions in many forums for how it could be improved in clarity or functionality, but nothing has been done. </p><p> To move from ideas to implementation, I have convened a working group of Perl/CPAN toolchain developers, maintainers, packagers and indexers -- the people responsible for the tools you use to configure, build, install or search for CPAN modules -- to review proposals, reach consensus and ultimately implement a new CPAN META Spec. </p><p> To ensure that we benefit from the best ideas in the Perl community, on behalf of this working group, I am issuing a public call for proposals. </p><p> <b>Process</b> </p><p> Here is the process and timeline: </p><ol> <li>Add proposals to the <a href="http://perl-qa.hexten.net/wiki/index.php/CPAN_Meta_Spec_Proposals">CPAN Meta Spec Proposals</a> page on the Perl QA Wiki. Discussion and refinement is encouraged through the discussion page on the wiki or on the cpan-workers mailing list (see below for details) </li><li>The Call for Proposals will close on Oct. 1, 2009. </li><li>The working group will post proposals on the cpan-workers mailing list for public discussion </li><li>Public discussion will close on Nov. 1, 2009. </li><li>The working group will reach consensus on proposals to adopt or reject and release a new CPAN META Spec no later than Dec. 1, 2009. </li></ol><p> <b>Criteria</b> </p><p> Since the work of implementation requires volunteer effort, the working group is looking for incremental improvements rather than complete rewrites of the spec. Proposal are more likely to be adopted if they: </p><ul> <li>Improve clarity or resolve ambiguity</li><li>Are narrow in scope</li><li>Address specific, current deficiencies</li><li>Are consistent with the general style of the existing spec</li><li>Require minimal implementation effort</li></ul><p> <b>Resources</b> </p><p> <strong>The current spec</strong>: Browse the current spec, as converted to HTML, here: <a href="http://module-build.sourceforge.net/META-spec.html">http://module-build.sourceforge.net/META-spec.html</a>. </p><p> <strong>The repository</strong>: The official spec draft file, META-spec.pod, has been moved from the Module-Build repository into its own repository on github. Please feel free to include proposed patches as part of proposals: </p><ul> <li>Repository: git://github.com/dagolden/cpan-meta-spec.git</li><li>Browser: <a href="http://github.com/dagolden/cpan-meta-spec/">http://github.com/dagolden/cpan-meta-spec/</a> </li></ul><p> <strong>The wiki</strong>: Proposals will be kept on the Perl QA Wiki: <a href="http://perl-qa.hexten.net/wiki/index.php/CPAN_Meta_Spec_Proposals">http://perl-qa.hexten.net/wiki/index.php/CPAN_Meta_Spec_Proposals</a> </p><p> <strong>The mailing list</strong>: Subscribe to cpan-workers by emailing <em>cpan-workers-subscribe@perl.org</em>. List archives are available at <a href="http://www.mail-archive.com/cpan-workers@perl.org/">http://www.mail-archive.com/cpan-workers@perl.org/</a> </p> dagolden 2009-08-26T23:29:01+00:00 cpan Leaving use.perl.org for dagolden.com http://use.perl.org/~dagolden/journal/38891?from=rss <p>Like many others, I'm giving up on use.perl.org for keeping a journal and setting up my own blog. If you follow my post here, I hope you will consider subscribing to <a href="http://www.dagolden.com/">dagolden.com</a> to get my updates in the future.</p><p>I've <a href="http://www.dagolden.com/index.php/28/iron-man-challenge-accepted/">accepted the Iron Man challenge</a>, so I hope to be posting a bit more frequently.</p><p>The newest post is my <a href="http://www.dagolden.com/index.php/35/belated-perl-qa-hackathon-progress-report/">summary of the 2009 QA Hackathon</a>.</p><p>-- dagolden</p> dagolden 2009-04-29T13:54:37+00:00 journal Module::Build/CPANPLUS yak shaving http://use.perl.org/~dagolden/journal/38432?from=rss <p> Lately, I've decided to take on my long-standing frustration with getting CPANPLUS and CPANPLUS::Dist::Build to upgrade Module::Build. Thanks to some help from Jos Boumans, Chris "BinGOs" Williams and Eric Wilhelm, I think these hairy yaks are finally getting shaved. </p><ul> <li> <p> <b>Step 1:</b> Module::Build 0.31_03 has just been released. It contains some workarounds that should allow existing versions of CPANPLUS::Dist::Build to upgrade Module::Build.</p></li><li> <p> <b>Step 2:</b> BinGOs is working on an alpha of CPANPLUS::Dist::Build that will no longer try to use the Module::Build API directly, but will interact with Build.PL and Build through subprocesses using IPC::Cmd.<b>[1]</b> This will also mean bumping up the Module::Build version prerequisite, which will actually work, thanks to Step 1.</p></li><li> <p> <b>Step 3:</b> After a period of testing, I hope to bump these from alphas to full releases and also roll both into bleadperl in time for 5.10.1.</p></li></ul><p>If all goes well, we should finally be at a point where modules can just specify 'configure_requires' on a recent version of Module::Build and -- assuming a recent enough CPAN.pm or CPANPLUS -- <b>things should just work.</b> </p><p> I'm also looking into porting code from TAP::Parser::Iterator::Process into IPC::Cmd that will enable it to capture output buffers from interactive programs on Win32. This -- along with Step 2 -- will make the output from Build.PL and 'Build test' available for CPAN Testers reports on all major platforms, which is a common complaint today about CPANPLUS-based reports for modules with Build.PL </p><p>-- dagolden </p><p> <b>[1]</b> Why avoid the Module::Build API? The issue is that Build.PL expects to be able to create a Module::Build object from an arbitrary class. In this case of Module::Build itself, it wants to use the new version in 'lib' to build and install itself (just like ExtUtils::MakeMaker does). Or any Build.PL could use a Module::Build subclass, either on the fly or bundled in the 'inc' directory. Or it could use a replacement for a Module::Build<nobr> <wbr></nobr>.pm file bundled into the 'inc' directory. </p><p>In a more extreme case, some future tool could use the Module::Install trick of taking over Build.PL and generating a Build program and providing a new implementation for 'Build', 'Build test', 'Build install' and so on. </p><p>In short -- a program calling Build.PL can't make any assumptions about what happens <b>except</b> the creation of a 'Build' program that can be executed with various actions. </p><p>Therefore, the only <b>safe</b> external API that installers should use is running 'perl Build.PL' and 'Build *' actions in a separate process. The Module::Build API itself should be a private one that is only used within a Build.PL or within Module::Build subclasses.</p> dagolden 2009-02-09T00:55:30+00:00 journal How many parsers does it take to break META.yml? http://use.perl.org/~dagolden/journal/38243?from=rss <p>It looks like Perl 5.10.1 may include <a href="http://search.cpan.org/perldoc?Parse::CPAN::Meta">Parse::CPAN::Meta</a>, so that CPAN and CPANPLUS will support 'configure_requires' out-of-the-box. This is a major step in improving forward compatibility of the CPAN installation toolchain.</p><p>However, it sparked a discussion on perl5-porters whether Parse::CPAN::Meta is ready to take on any META.yml on CPAN and so I used the <a href="http://search.cpan.org/perldoc?visitcpan">visitcpan</a> tool I wrote about in my <a href="http://use.perl.org/~dagolden/journal/38081">last journal post</a> to investigate.</p><p>I checked not only how Parse::CPAN::Meta handled META.yml, but also YAML::XS, YAML::Syck, YAML::Tiny and YAML.pm. Plus, I cross-checked each pair to see whether they produced the same output.</p><p>The <a href="http://echo.dagolden.com/~xdg/meta-yaml-testing/">results</a> show that very few of the 16,000 or so distributions on CPAN have META.yml problems (most probably don't even have META.yml). But of those that do, the different YAML parsers can give subtly different output. (Click any red "-" box to see details.)</p><p>Is Parse::CPAN::Meta any worse than the others? Hard to say. Personally, I think it's "good enough" for 5.10.1 and hope any corner cases are fixed over time.</p><p>-- dagolden</p> dagolden 2009-01-08T22:35:21+00:00 journal How to look inside every CPAN distribution http://use.perl.org/~dagolden/journal/38081?from=rss <p>Ever wonder how many CPAN distributions use Module::Build, or Module::Install? Or how common the different ways Changes, ChangeLog, Changelog, etc. show up in a distribution? Or how many distributions use a particular expression in code?</p><p>I wrote <a href="http://search.cpan.org/dist/App-CPAN-Mini-Visit/">App-CPAN-Mini-Visit</a> and the <a href="http://search.cpan.org/perldoc?visitcpan">visitcpan</a> tool to make answering such questions a lot easier.</p><p>It provides a command line tool that unpacks each distribution tarball in a <a href="http://search.cpan.org/dist/CPAN-Mini/">CPAN::Mini </a> repository, visits the resulting directory, and executes another program. For example, listing all distributions that contain Build.PL:</p><blockquote><div><p> <tt>$ visitcpan --append=dist -- perl -E 'say shift if -f "Build.PL"'<br> <br>AASSAD/Catalyst-Helper-Controller-Scaffold-HTML-Template-0.03.tar<nobr> <wbr></nobr>.gz<br>ABERGMAN/Alien-0.91.tar.gz<br>ABERGMAN/File-Find-Random-0.5.tar.gz<br>ABERGMAN<nobr>/<wbr></nobr> SVL-0.29.tar.gz<br>...</tt></p></div> </blockquote><p>Clearly, if I were running frequent analyses, I might just use <a href="http://search.cpan.org/dist/CPAN-Mini-Extract/">CPAN::Mini::Extract</a> and keep all the tarballs extracted, but since I only occasionally have questions like this, I'm willing to make the space/time tradeoff.</p><p>Oh, and to answer that first question, here's how many distributions use each style of installer:</p><blockquote><div><p> <tt>$ visitcpan -- perl -E 'if (-f "inc/Module/Install.pm") { say "MI" } elsif (-f "Build.PL") { say "MB" } elsif (-f "Makefile.PL") { say "EUMM" }' | perl -E 'while (&lt;&gt;) { chomp; $cnt{$_}++ } say "$_: $cnt{$_}" for keys %cnt'<br> <br>MI: 2306<br>MB: 2675<br>EUMM: 11781</tt></p></div> </blockquote><p>Looks like Module::Install has almost caught up to Module::Build. Impressive!</p><p>--dagolden</p> dagolden 2008-12-12T21:03:55+00:00 journal Putting CPAN modules on github http://use.perl.org/~dagolden/journal/38053?from=rss <p>I saw in Ovid's <a href="http://use.perl.org/~Ovid/journal/38051">post about App::Prove::History</a> that he's hosting it on <a href="http://github.com/">Github</a>.</p><p>I've recently started <a href="http://github.com/dagolden/">mirroring my git repositories to Github</a> as well. It has some nice eye candy and tracking tools.</p><p>One thing that made it easy to migrate was brian d foy's <a href="http://search.cpan.org/~bdfoy/github_creator-0.13/">github-creator</a> package, which is a command-line tool to create a new repository on Github. You just run it from a distribution directory and it figures out a name and so on.</p><p>It's not a module (yet), so installing with CPAN requires an explicit distribution name:</p><blockquote><div><p> <tt>cpan&gt; install BDFOY/github_creator-0.13.tar.gz</tt></p></div> </blockquote><p>If you're thinking of putting your CPAN modules on github, definitely try it out.</p><p>-- dagolden</p> dagolden 2008-12-09T14:37:45+00:00 journal Hey Perl Advent, ToolSet gets even better! http://use.perl.org/~dagolden/journal/38028?from=rss <p>I was tickled to find my <a href="http://search.cpan.org/dist/ToolSet">ToolSet</a> module <a href="http://www.perladvent.org/2008/1/">featured </a> in the <a href="http://www.perladvent.org/">2008 Perl Advent Calendar</a>.</p><p>However, in response to the comment that ToolSet didn't support pragmas like "no indirect", I decided it was time for a feature upgrade. Now that Perl 5.10 allows for <a href="http://perldoc.perl.org/perlpragma.html">user pragmas</a>, ToolSet needed more general pragma support.</p><p>So <a href="http://search.cpan.org/dist/ToolSet">ToolSet</a> 0.99 has a new API for pragmas:</p><blockquote><div><p> <tt># My/Tools.pm<br>package My::Tools;<br>use base 'ToolSet';<br> <br>ToolSet-&gt;use_pragma( 'strict' );<br>ToolSet-&gt;use_pragma( 'warnings' );<br>ToolSet-&gt;no_pragma( 'indirect' );<br> <br>1;</tt></p></div> </blockquote><p>This should work with any simple pragma that uses <tt>$^H</tt> or <tt>%^H</tt> without other side-effects during <tt>import()</tt>.</p><p>Happy holidays!</p><p>-- dagolden</p> dagolden 2008-12-05T03:36:37+00:00 journal Fun with Perl: rdesktop launcher http://use.perl.org/~dagolden/journal/37916?from=rss <p>I've started experimenting with using rdesktop to remotely access my (Windows) laptop from my Ubuntu desktop. The annoying part was figuring out what DHCP address was assigned to the laptop each time.</p><p>Then, of course, I realized this was a job for Perl! (Plus some system tools)</p><p>So I wrote a "laptop-launcher" program and here's what it does:</p><ul> <li>run 'nmap' to scan for hosts with open remote desktop ports</li><li>parse the nmap output to map MAC addresses to IP addresses (thank you Regexp::Common qw/net/)</li><li>run 'rdesktop' with the IP address corresponding to my laptop's MAC address</li></ul><p>Just a couple minutes work and now launching a remote desktop session to my laptop is a snap.</p><p>Thank you, Perl!</p><p>-- dagolden</p> dagolden 2008-11-21T03:46:29+00:00 journal ToolSet yak shaving adds Perl 5.10 features http://use.perl.org/~dagolden/journal/37821?from=rss <p>As often happens, I can't remember the exact original need that started the latest round of yak shaving. I think I was working on a one-liner to prototype a test program for some small program I was writing.</p><p>In any case, I was annoyed that my XDG <a href="http://search.cpan.org/dist/ToolSet/">ToolSet</a> module I use for one-liners didn't give me Perl 5.10 features, but adding them required a new <a href="http://search.cpan.org/dist/ToolSet/">ToolSet</a> release.</p><p>So now "perl -MXDG -e<nobr> <wbr></nobr>... " gives me strict, warnings, some of Carp, DDS, Path::Class, File::Spec, Scalar::Util <b>and</b> either all of the Perl 5.10 features or else at least Perl6::Say.</p><p>If anyone else finds this useful, here's the code, or <a href="http://echo.dagolden.com/git/?p=ToolSet-XDG">clone it from git</a> and adapt as you see fit.</p><blockquote><div><p> <tt>package XDG;<br> <br>our $VERSION = '0.05';<br> <br>use base 'ToolSet';<br> <br>ToolSet-&gt;use_pragma( 'strict' );<br>ToolSet-&gt;use_pragma( 'warnings' );<br> <br>ToolSet-&gt;export(<br>&nbsp; &nbsp; 'Carp' =&gt; 'carp croak confess',<br>&nbsp; &nbsp; 'Data::Dump::Streamer' =&gt; 'Dump',<br>&nbsp; &nbsp; 'File::Spec' =&gt; undef,<br>&nbsp; &nbsp; 'Path::Class' =&gt; 'file dir',<br>&nbsp; &nbsp; 'Scalar::Util' =&gt; 'refaddr reftype blessed',<br>);<br> <br>if ( $] &gt;= 5.010 ) {<br>&nbsp; ToolSet-&gt;use_pragma( 'feature', ':5.10' );<br>}<br>else {<br>&nbsp; ToolSet-&gt;export( 'Perl6::Say' =&gt; 'say' );<br>}<br> <br>1;</tt></p></div> </blockquote> dagolden 2008-11-07T03:05:03+00:00 journal Hello World -- family style http://use.perl.org/~dagolden/journal/37616?from=rss <blockquote><div><p> <tt>$self-&gt;wife-&gt;spawn("Jeremy William Golden");</tt></p></div> </blockquote><p>Born 1:16 AM on Oct 7, 2008. 9 pounds, 10.5 ounces. <a href="http://echo.dagolden.com/~xdg/jeremy.jpg">Photo</a><nobr> <wbr></nobr>.</p><p>-- dagolden</p> dagolden 2008-10-07T16:21:48+00:00 journal Calling CPAN Testers: time for an upgrade http://use.perl.org/~dagolden/journal/37377?from=rss <p>Dear <a href="http://cpantesters.org/">CPAN Testers</a>,</p><p>As per several long discussion threads on the <a href="http://www.mail-archive.com/perl-qa@perl.org/">perl-qa</a> and <a href="http://www.mail-archive.com/cpan-testers-discuss@perl.org/">cpan-testers-discuss</a> mailing lists, and per the <a href="http://www.mail-archive.com/cpan-testers-discuss@perl.org/msg00583.html">plan I posted</a> the other day, today I took steps to upgrade CPAN Testers tools to be less annoying to the CPAN author community: </p><ul> <li> <p>Released <a href="http://search.cpan.org/dist/Test-Reporter/">Test::Reporter</a> 1.50 -- authors will no longer be copied on reports (the send() method ignores any arguments) -- this is a key step to move us to a centrally administered "opt-in" notification approach</p></li><li> <p>Released <a href="http://search.cpan.org/dist/CPAN-Reporter/">CPAN::Reporter</a> 1.17 -- failures during PL/make stages will be graded UNKNOWN; removed "cc_author" option and related code; added checks to discard failures due to "Build -j3" and "Makefile out of date" errors.</p></li><li> <p>Created a CPANPLUS repository branch -- PL/make failures will be graded UNKNOWN; "dontcc" option has been removed; output buffer will be included with NA reports. Jos Boumans committed to review and incorporate the CPANPLUS changes in the next release.</p></li></ul><p>All testers should upgrade at least Test::Reporter. Users of CPAN::Reporter should upgrade to 1.17. </p><p>Please test these out manually before turning them loose with automated smoke testers. </p><p>If anyone would like to explore or test the CPANPLUS branch, it can be found here: </p><p> <a href="http://oss.dwim.org/branches/dagolden/cpanplus-devel/">http://oss.dwim.org/branches/dagolden/cpanplus-devel/</a> </p><p>As always, let me know what bugs you find and I'll try to address them as soon as I can. </p><p>I know Barbie is hard at work on a notification program based on the CPAN Testers database so author notification should resume soon.</p><p>-- dagolden</p> dagolden 2008-09-07T02:30:08+00:00 journal Why CPAN Testers sucks if you don't need it http://use.perl.org/~dagolden/journal/37342?from=rss <p>There have been some mega-email threads about CPAN Testers on the perl-qa mailing list that started with a <a href="http://www.nntp.perl.org/group/perl.qa/2008/08/msg11235.html">question about the use of 'exit 0'</a> in Makefile.PL. </p><p>I want to sum up a few things that I took away from the conversations and propose a series of major changes to CPAN Testers. Special thanks to an off-list (and very civil) conversation with chromatic for triggering some of these thoughts. </p><p> <b>Type I and Type II errors</b> </p><p>In statistics, a Type I error means a "false positive" or "false alarm". For CPAN Testers, that's a bogus FAIL report. A Type II error means a "false negative", e.g. a bogus PASS report. Often, there is a trade-off between these. If you think about spam filtering as an example, reducing the chance of spam getting through the filter (false negatives) tends to increase the odds that legitimate mail gets flagged as spam (false positives). </p><p>Generally, those involved in CPAN Testers have taken the view that it's better to have a false positives (false alarms) than false negatives (a bogus PASS report). Moreover, we've tended to believe -- without any real analysis -- that the false positive *ratio* (false FAILs divided by all FAILs) is low. </p><p>But I've never heard a single complaint about a bogus PASS report and I hear a lot of complaints about bogus FAILS, so it's reasonable to think that we've got the tradeoff wrong. Moreover, I think the downside to false positives is actually higher than for false negatives if we believe that CPAN Testers is primarily a tool to help authors improve quality rather than a tool to give users a guarantee about how distributions work on any given platform. </p><p> <b>False positive ratios by author</b> </p><p>Even if the aggregate false positive ratio is low, individual CPAN authors can experience extraordinarily high false positive ratios. What I suddenly realized is that the higher the quality of an author's distributions, the higher the false positive ratio. </p><p>Consider a "low quality" author -- one who is prone to portability errors, missing dependencies and so on. Most of the FAIL reports are legitimate problems with the distribution. </p><p>Now consider a "high quality" author -- one who is careful to write portable code, well-specified dependencies and so on. For this author, most of the FAIL reports only come when a tester has a broken or misconfigured toolchain The false positive ratio will approach 100%. </p><p>In other words, the *reward* that CPAN Testers has for high quality is increased annoyance from false FAIL reports with little benefit. </p><p> <b>Repetition is desensitizing</b> </p><p>From a statistical perspective, having lots of CPAN Testers reports for a distribution even on a common platform helps improve confidence in the aggregate result. Put differently, it helps weed out "outlier" reports from a tester who happens to have a broken toolchain. </p><p>However, from author's perspective, if a report is legitimate (and assuming they care), they really only need to hear it once. Having more and more testers sending the same FAIL report on platform X is overkill and gives yet more encouragement for authors to tune out. </p><p>So the more successful CPAN Testers is in attracting new testers, the more duplicate FAIL reports authors are likely to receive, which makes them less likely to pay attention to them.</p><p> <b>When is a FAIL not a FAIL?</b> </p><p>There are legitimate reasons that distributions could be broken such that they fail during PL or make in ways that are not the fault of the tester's toolchain, so it still seems like valuable information to know when distributions can't build as well as when they don't pass tests. So we should report on this and not just skip reporting. On the other hand, most of the false positives that provoke complaint are toolchain issues during PL or make/Build. </p><p>Right now there is no easy way to distinguish the phase of a FAIL report from the subject of an email. Removing PL and make/Build failures from the FAIL category would immediately eliminate a major source of false positives in the FAIL category and decrease the aggregate false positive ratio in the FAIL category. Though, as I've shown, while this may decrease the incidence of false positives for high quality authors, the false positive ratio is likely to remain high. </p><p>It almost doesn't matter whether we reclassify these as UNKNOWN or invent new grades. Either way partitions the FAIL space in a way that makes it easier for authors to focus on which ever part of the PL/make/test cycle they care about. </p><p> <b>What we can fix now and what we can't</b> </p><p>Some of these issues can be addressed fairly quickly. </p><p>First, we can lower our collective tolerance of false positives -- for example, stop telling authors to just ignore bogus reports if they don't like it and find ways to filter them. We have several places to do this -- just in the last day we've confirmed that the latest CPANPLUS dev version doesn't generate Makefile.PL's and some testers have upgraded. BinGOs has just put out CPANPLUS::YACSmoke 0.04 that filters out these cases anyway if testers aren't on the bleeding edge of CPANPLUS. We now need to push testers to upgrade. As we find new false positives, we need to find new ways to detect and suppress them. </p><p>Second, we can reclassify PL/make/Build fails to UNKNOWN. This won't break any of the existing reporting infrastructure the way that adding new grades would. I can make this change in CPAN::Reporter in a matter of minutes and it probably wouldn't be hard to do the same in CPANPLUS. Then we need another round of pushing testers to upgrade their tools. We could also take a decision as to whether UNKNOWN reports should be copied to authors by default or just sent to the mailing list. </p><p>However, as long as the CPAN Testers system has individual testers emailing authors, there is little we can do to address the problem of repetition. One option is to remove that feature from Test::Reporter and reports will only go to the central list. With the introduction of an RSS feed (even if not yet optimal), authors will have a way to monitor reports. And from that central source, work can be done to identify duplicative reports and start screening them out of notifications. </p><p>Once that is more or less reliable, we could restart email notifications from that central source if people felt that nagging is critical to improve quality. Personally, I'm coming around to the idea that it's not the right way to go culturally for the community. We should encourage people to use these tools, sign up for RSS or email alerts, whatever, because they think that quality is important. If the current nagging approach is alienating significant numbers of perl-qa members, how can we possibly expect that it's having a positive influence on everyone else? </p><p>Some of these proposal would be easier in CPAN Testers 2.0, which will provide reports as structured data instead of email text, but if "exit 0" is a straw that is breaking the Perl camel's back now, then we can't ignore 1.0 to work on 2.0 as I'm not sure anyone will care anymore by the time it's done. </p><p>What we can't do easily is get the testers community to upgrade to newer versions of the tools. That is still going to be a matter of announcements and proselytizing and so on. But I think we can make a good case for it, and if we can get the top 10 or so testers to upgrade across all their testing machines then I think we'll make a huge dent in the false positives that are undermining support for CPAN Testers as a tool for Perl software quality. </p><p>I'm interested in feedback on these ideas. In particular, I'm now convinced that the "success" of CPAN Testers now prompts the need to move PL/make fails to UNKNOWN and to discontinue copying authors by individual testers. I'm open to counter-arguments, but they'll need to convince me of a better long-run solution to the problems I identified. </p><p>-- dagolden</p> dagolden 2008-09-03T21:22:41+00:00 journal Oslo Hackathon 2008 -- wrap-up writeup http://use.perl.org/~dagolden/journal/36113?from=rss <p> First, I'd like to thank The Perl Foundation for sponsoring my trip to the hackathon. I'd also like to thank <a href="http://www.linpro.no/">Linpro</a> for providing a wonderful venue as well as on-site food and refreshments throughout the hackathon. This is my wrap-up summary so they have a sense of what their contributions helped make possible. </p><p> <b>Project 1 -- Test::Reporter transport plugins</b> </p><p>I worked with rjbs on adding "transport" plugins to Test::Reporter to provide better options for how reports are sent to CPAN Testres. The existing transports (Mail::Send and Net::SMTP) were extracted into separate modules (Test::Reporter::Transport::Mail::Send, etc.). Plus we created three new transports: </p><ul> <li> <p>Test::Reporter::Transport::HTTPGateway -- provides HTTP to email transport via a the new Test::Reporter::HTTPGateway module</p></li> <li> <p>Test::Reporter::Transport::Net::SMTP::TLS -- provides TLS and authenticated SMTP (a long-standing wishlist item)</p></li> <li> <p>Test::Reporter::Transport::File -- reports saved as files in a directory for offline CPAN Testing and manual submission</p></li> </ul><p>Unfortuately, the HTTPGateway transport is only a temporary step towards a better CPAN Testers as the gateway still just relays to email. But people can set up their own gateways for now or perhaps a central server will be deployed somewhere. </p><p>Using transport plugins for all of these with a consistent API makes it easy for clients like CPAN::Reporter to switch between them based on configuration settings. E.g., in<nobr> <wbr></nobr>.cpanreporter/config.ini</p><blockquote><div><p> <tt>transport=HTTPGateway http://gateway.example.com/http2email/</tt></p></div> </blockquote><p> <b>Project 2 -- CPAN Metabase for CPAN Testers 2.0</b> </p><p>The big issue with CPAN Testers "1.0" (what we currently have) is that it uses email to cpan-testers@perl.org for submissions and the only archive for reports is effectively the perl.cpan.testers newsgroup. For a while, I and others were talking about creating CPAN Testers 2.0 to move to HTTP submission and a centralized, indexed, searchable database. </p><p>I'd hoped to interest people in Oslo in thinking through what it might require or even working on some pieces of it. What I found was that there are a <i>lot</i> of areas where people are looking to collect 'meta' information about CPAN distributions, each with their own approach and APIs for gathering, storing or publishing in a very application-specific way. </p><p>That got me and rjbs started designing a more general solution, which we've called a CPAN 'metabase'. This would work for CPAN Testers 2.0 or for other projects that want to store/publish distribution-level information. </p><p>After a few days of work, we achieved: </p><ul> <li> <p>General design of a system (class hierarchy, architecture, etc.) to meet at least initial envisioned needs with the flexibility to expand over time</p></li> <li> <p>A simple, working proof-of-concept -- with filesystem-based storage and indexing and the ability to store and retrieve a simple "fact" (a text string) via a web client.</p></li> </ul><p>Next steps are to refine, standardize and document the classes and APIs; to add additional capabilities to the proof-of-concept; and to get a 0.01 release to CPAN for community feedback. </p><p> <b>Other contributions</b> </p><ul> <li> <p>Moral support for Adam Kennedy's time-travelling efforts to build the April Strawberry Perl</p></li><li> <p>Feedback on TAP diagnostic semantics and the implication for downstream consumers like CPAN::Reporter</p></li><li> <p>Figured out a hack with the Berkeley DB libraries to help Tux build DB_File on Strawberry Perl</p></li><li> <p>Participated in a best practice discussion/argument on environment variables, 'xt' directories, etc.</p></li> </ul><p> <b>Things that came up that I didn't get around to working on</b> </p><ul> <li> <p>CPAN::Reporter::Smoker -- a couple people (Gabor and someone else?) said it would be helpful if C::R::S could take a specific list for a work queue instead of trying to smoke only the latest things on CPAN. Now that someone asked for it, that gets bumped up on my priority list</p></li> <li> <p>Salve brought up using boilerplating as a way of discussing and later disseminating best practices so I showed him a boilerplate tool in my repository (not yet released). He and rjbs thought it had promise given easy customization so I'm going to try to finish the documentation and get a 0.01 release out soon</p></li> <li> <p>Schwern brought up the CPAN::PERL5INC taint bug in recent devel versions of CPAN.pm</p></li> <li> <p>Various people were working on codifying and releasing modules for various parts of the PAUSE/CPAN tools. rjbs has a work project along these lines and brian d foy's work to index BackPAN is driving similar efforts. There was some discussion of making it a YAPC::NA hackathon project</p></li> </ul><p> <b>Other things I learned (not necessarily Perl)</b> </p><ul> <li> <p>git is awesome. I had been looking for a project to force me to live with it and get over the learning curve. Since Test::Reporter lives in git and rjbs knew it already, I got my chance. rjbs and I were routinely developing simultaneously on the same code base, merging back and forth several times a day and despite the huge flux in our early development code, it "just worked" </p><p>The best part was that during the flight home, we continued hacking, using a git repository on a usb drive to exchange and merge our branches. We'd just walk it between our seats about every hour or so</p></li> <li> <p>Developing Perl on a Ubuntu virtual machine on Windows is a vastly better, faster experience than native Perl on Windows. (It would be interesting for someone to figure out what's behind that)</p></li> <li> <p>rjbs showed me the wonders of screen (and screen + irssi). I don't know how I could have gone so long without learning how cool it is.</p></li> <li> <p>Norwegians are friendly. The organizers, Linpro staff and people on the streets were welcoming and helpful whenever we need it</p></li> </ul><p> <b>Other thanks</b> </p><ul> <li> <p>To <a href="http://pobox.com/">Pobox</a> (via rjbs) for a temporary mailbox for testing TSL and authenticated SMTP</p></li> <li> <p>To the BBC for <a href="http://backstage.bbc.co.uk/">backstage.bbc.co.uk swag</a></p></li> <li> <p>To jonasbn for being a sounding board as rjbs and I drew crazy metabase design diagrams</p></li><li> <p>To rjbs for putting in a tremendous effort in a short period of time on projects that he wasn't even focusing on before the hackathon and for teaching me a lot of new tools</p></li></ul><p>-- dagolden</p> dagolden 2008-04-09T19:03:53+00:00 journal Oslo - Day 1 - HTTP transport for CPAN::Testers http://use.perl.org/~dagolden/journal/36069?from=rss <p>If you haven't seen <a href="http://use.perl.org/~rjbs/journal/36068">rjbs's journal</a>, he posted about some of what he, <a href="http://use.perl.org/~jonasbn/">jonasbn</a> and I have been working on here in Oslo.</p><p>We came up with some good ideas for a general metabase for CPAN distributions, which we think can become the back-end not just for CPAN Testers 2.0 but for other tools as well that are all doing essentially the same thing in collecting distributed information about distributions.</p><p>However, we realized that it would probably be a lot of work, so we wanted to start with something tangible that could be successful quickly. So we added 'HTTP' as a valid transport() option for Test::Reporter and RJBS created Test::Reporter::HTTPGateway to be a remote server that can gateway HTTP transport to cpan-testers@perl.org. We've done some limited testing and it appears to be working well. (More testing tomorrow.)</p><p>The documentation could still be improved -- but essentially, if you install the latest dev releases of Test::Reporter (1.38_01) and CPAN::Reporter (1.14_01), then you can use this config option in your<nobr> <wbr></nobr>.cpanreporter/config.ini:</p><blockquote><div><p> <tt>&nbsp; &nbsp; transport=HTTP http://some.example.com/gateway/</tt></p></div> </blockquote><p>and assuming you're running T::R::HTTPGateway at that URL it should work.</p><p>In the process, I wound up using some code that I had started working on a couple months ago and never finished to support other transports like Net::SMTP::TLS, so while that hasn't been tested yet (maybe tomorrow), it's possible that Net::SMTP::TLS may actually be working as well, or could be made to work with little additional effort.</p><p>That's the news from Oslo.</p><p>As an additional note, I want to particularly thank TPF for sponsoring my travel and <a href="http://www.linpro.no/">Linpro</a> for providing excellent facilities for the hackathon and also some very friendly and very helpful hosts. Also, I want to send a big thank you to <a href="http://use.perl.org/~sjn/">Salve</a> for taking the lead in organizing the event in the first place!</p><p>-- dagolden</p> dagolden 2008-04-05T23:47:36+00:00 journal libwin32: can we divide and conquer? http://use.perl.org/~dagolden/journal/35683?from=rss <p>I recently installed Strawberry Perl on a virtual machine. Since Strawberry Perl doesn't (yet) include libwin32 from the start that was one of the first distributions I installed from CPAN.</p><p>I was annoyed to find that it aborted testing on Win32::FileSecurity. Why? Because my partion was FAT32, not NTFS. Ironically -- it warns that it will fail when not on NTFS, but does nothing to test for it. (c.f. Win32::FsType() -- though maybe that didn't exist when the test was first written) </p><p>While that's an easy fix, it highlighted a bigger issue that I have with libwin32. Because it bundles multiple modules as separate distributions with recursive Makefiles, that single failure stopped tests, so I couldn't even see if the <i>rest</i> of libwin32 worked correctly. (So, yes, hack-fix Win32::FileSecurity, continue testing, all tests pass, install, post bug and patched test.pl file to RT, whee...)</p><p>I wish all the individual modules in libwin32 could be broken out into separate CPAN distributions and all of them were just added to Bundle::libwin32 instead. That way, some strange failure of one module wouldn't stop the rest from installing.</p><p>Moreover, I wonder if having separate distributions would make it easier to distribute responsibility for tackling the <a href="http://rt.cpan.org/NoAuth/Bugs.html?Dist=libwin32">growing RT queue</a> and let individual modules be fixed and released on a faster cycle.</p><p>So that's my challenge to the maintainers and the lazyweb -- help us divide and conquer libwin32!</p><p>-- dagolden</p> dagolden 2008-02-18T13:50:39+00:00 journal CPAN 1.92_56 adds "trust_test_report_history" option http://use.perl.org/~dagolden/journal/35637?from=rss <p>I'm proud of this new config option I added to CPAN.pm: trust_test_report_history.</p><p>When this option is set and the latest version of CPAN::Reporter is installed, then CPAN.pm won't run tests on a distribution that has already passed tests on that platform and perl, but will use the last test result for that distribution instead.</p><p>Where this comes in handy is "build_requires" prerequisites. If a build_requires prereq is not installed, CPAN.pm will download, build and test it and then include it in PERL5LIB (if it passes its tests) for the rest of that session. But in any future session, the next time a distribution has that prerequisite, CPAN.pm does the exact same thing all over again. But with "trust_test_report_history", the test is only run once.</p><p>This may not have a big impact for day-to-day use, but it should save a lot of time and processor cycles for smoke testing.</p><p>-- dagolden</p> dagolden 2008-02-12T01:18:40+00:00 journal CPAN::Reporter::Smoker first alpha release http://use.perl.org/~dagolden/journal/35442?from=rss <p>For a long time, the only easy way to submit CPAN Testers reports was CPANPLUS. The first automated smoke testers for CPAN were built upon it, like <a href="http://search.cpan.org/dist/CPAN-YACSmoke/">CPAN::YACSmoke</a> and <a href="http://search.cpan.org/dist/POE-Component-CPAN-YACSmoke/"> POE::Component::CPAN::YACSmoke</a>.</p><p>About a year and a half ago, I wrote <a href="http://search.cpan.org/dist/CPAN-Reporter">CPAN::Reporter</a>, to bring CPAN Testers reporting to good-old CPAN.pm. Since then, some intrepid individuals have built automated smoke testers upon CPAN::Reporter, all writing their own programs to do so.</p><p>CPAN.pm itself added an experimental <i>smoke</i> command to test recent uploads, but it requires XML::LibXML and doesn't test distributions in isolation -- all tested distributions and their dependencies keep getting added to PERL5LIB until things explode.</p><p>Yesterday, I released the first, alpha version of <a href="http://search.cpan.org/dist/CPAN-Reporter-Smoker"> CPAN::Reporter::Smoker</a> to provide a better-behaved, more turn-key approach to smoke testing with CPAN::Reporter.</p><blockquote><div><p> <tt>(... configure CPAN and CPAN::Reporter as usual<nobr> <wbr></nobr>... )<br> <br>$ perl -MCPAN::Reporter::Smoker -e start</tt></p></div> </blockquote><p>This initial version still has some limitations (e.g. no developer versions smoked), but I'm ready for anyone interested to try it out and start sending feedback -- compliments, suggestions, or complaints all welcome.</p><p>-- dagolden</p> dagolden 2008-01-21T14:31:41+00:00 journal Can you run Inline::C? I need your help... http://use.perl.org/~dagolden/journal/34919?from=rss <p>If you have or can install <a href="http://search.cpan.org/perldoc?Inline::C">Inline::C</a>, I'd greatly appreciate your help testing IO-CaptureOutput-1.05_53.</p><p>I've recently adopted <a href="http://search.cpan.org/perldoc?IO::CaptureOutput">IO::CaptureOutput</a>, which is a wonderful tool for capturing program output to STDOUT or STDERR without ties and regardless of whether the output comes from perl, XS or programs in a subprocess.</p><p>However, the tests for XS use Inline::C and the C code was found to have portability problems (i.e. segfault on some Win32 platforms). At least one fix for Microsoft Visual C++ (MSVC) then broke on someone else's Linux platform.</p><p> <i>(Aside: the fact that it it this hard to portably print the equivalent of "Hello World" to STDOUT and STDERR just astonishes me.)</i> </p><p>My latest attempt at updating the C code now uses different code for MSVC and other compilers and now I want to test this as far and as wide as I can.</p><p>So if you have installed Inline::C and something that can send test reports (e.g. <a href="http://search.cpan.org/perldoc?CPAN::Reporter">CPAN::Reporter</a>), please test IO-CaptureOutput-1.05_53. For example, from the CPAN shell:</p><blockquote><div><p> <tt>cpan&gt; test DAGOLDEN/IO-CaptureOutput-1.05_53.tar.gz</tt></p></div> </blockquote><p>Thank you very much,</p><p>-- dagolden</p> dagolden 2007-11-18T16:21:09+00:00 journal I need MSVC help http://use.perl.org/~dagolden/journal/34704?from=rss <p>I'm helping David Cantrell with <a href="http://search.cpan.org/perldoc?Devel::AssertLib">Devel::AssertLib</a> -- particularly with Win32 portability.</p><p>I have a branch that passes tests on my Linux and Strawberry Perl platforms, and I've added code that I *think* should work with ActiveState and MSVC, but I don't have that platform set up anywhere.</p><p>I'd greatly appreciate if someone(s) would download the branch from my repository, try running tests and let me know what happens:</p><blockquote><div><p> <tt>http://dagolden.googlecode.com/svn/shadow/Devel-AssertLib/branches/dagolden/</tt></p></div> </blockquote><p>If it fails and you can patch it to work, that would be wonderful as well.</p><p>Thank you very much</p><p>-- dagolden</p> dagolden 2007-10-17T12:16:48+00:00 journal CPAN::Reporter 1.00 released http://use.perl.org/~dagolden/journal/34650?from=rss <p>I'm pleased at announce that <a href="http://search.cpan.org/perldoc?CPAN%3A%3AReporter">CPAN::Reporter</a> 1.00 has been released and should soon be appearing on a CPAN mirror near you. Now, anyone with CPAN and a relatively modern version of Perl can contribute to <a href="http://cpantesters.perl.org/">CPAN Testers</a>.</p><p>Version 1.00, combined with CPAN.pm 1.92 or better, provides several major enhancements over the 0.XX series. From the Changes file:</p><ul> <li> <p>Added support for reporting for *.PL and make/Build stages; bumped CPAN.pm prerequisite to 1.9203 to take advantage of this support</p></li><li> <p>Added support for the forthcoming Test::Harness 3.00</p></li><li> <p>Changed the name and format of the history file of sent reports to track history by PL/make/test phase. Old history.db will be automatically upgraded to new reports-sent.db.</p></li><li> <p>Moved 'cc_author' and 'send_duplicates' options from interactive to advanced (manual) configuration; defaults are strongly recommended</p></li><li> <p>Bumped Test::Reporter prereq to 1.34 for transport() method and set default transport to Net::SMTP on all platforms</p></li></ul><p>Anyone using an older version of CPAN::Reporter is <i>strongly</i> encouraged to upgrade.</p><p>Getting started with CPAN::Reporter is easy. From the CPAN shell:</p><blockquote><div><p> <tt>cpan&gt; install CPAN::Reporter<br>cpan&gt; reload cpan<br>cpan&gt; o conf init test_report</tt></p></div> </blockquote><p>A note to module authors -- CPAN::Reporter is getting smarter about handling prerequisite failures and unsupported Perl versions or platforms. But there are best practices you can follow to make this easier. See <a href="http://cpantest.grango.org/cgi-bin/pages.cgi?act=wiki-page&amp;pagename=CPANAuthorNotes">Author Notes</a> on the <a href="http://cpantest.grango.org/">CPAN Testers Wiki</a> for more details.</p><p>-- dagolden</p> dagolden 2007-10-11T01:23:04+00:00 journal CPAN::Reporter goes beta http://use.perl.org/~dagolden/journal/34528?from=rss <p>Since the initial release of CPAN::Reporter, one of the big missing features was sending CPAN Testers reports for *.PL and make/Build stages. A while ago, I added that capability in the CPAN::Reporter and CPAN development series.</p><p>With the new, stable release of CPAN 1.92, CPAN::Reporter is very close to its 1.00 release. I've just bumped the CPAN prerequisite to match and put out CPAN::Reporter 0.99_12. I consider this a release candidate for 1.00.</p><p>If you're already using a development version of CPAN::Reporter, please upgrade to the latest to help shake out any last blockers. If you aren't using a development version of CPAN::Reporter and would like to try it, it should install easily from the CPAN shell (and will upgrade CPAN for you if necessary).</p><blockquote><div><p> <tt>cpan&gt; install DAGOLDEN/CPAN-Reporter-0.99_12.tar.gz<br>...<br>cpan&gt; reload cpan</tt></p></div> </blockquote><p>Thank you very much to everyone that has contributed suggestions and bug reports along the way.</p><p>-- David</p><p>References:</p><ul> <li> <a href="http://search.cpan.org/dist/CPAN-Reporter-0.99_12/">CPAN::Reporter 0.99_12</a> (<a href="http://search.cpan.org/dist/CPAN-Reporter-0.99_12/Changes">Changes</a>)</li><li> <a href="http://search.cpan.org/dist/CPAN-1.92">CPAN 1.92</a> </li></ul> dagolden 2007-09-24T09:54:09+00:00 journal Please test dev versions of CPAN + CPAN::Reporter http://use.perl.org/~dagolden/journal/34062?from=rss <p>With CPAN version 1.91_53 and CPAN::Reporter version 0.99_02, CPAN will now report to CPAN::Testers any failures during Makefile.PL (Build.PL) or make (Build).</p><p>Ironically, this pair now needs some good-old-fashioned manual testing. Please consider installing them and then testing some modules that you've seen fail to build on your system.</p><blockquote><div><p> <tt>cpan&gt; install ANDK/CPAN-1.91_53.tar.gz<br> <br>cpan&gt; install DAGOLDEN/CPAN-Reporter-0.99_02.tar.gz<br> <br>cpan&gt; reload cpan</tt></p></div> </blockquote><p>Then test some module that fails to even make and see if the failure report made it to CPAN Testers.</p><p>Thank you!</p> dagolden 2007-08-09T12:04:58+00:00 journal Spreading the gospel of Perl, Technorati-style http://use.perl.org/~dagolden/journal/33931?from=rss <p>Soon, all the recent use.perl journal posts will just be links to someone's <a href="http://technorati.com/claim/8gme9zjxd7">Technorati Profile</a>.</p> dagolden 2007-07-31T15:35:36+00:00 journal Sub::Uplevel learns to play nice http://use.perl.org/~dagolden/journal/33714?from=rss <p>I've just released Sub::Uplevel 0.15_01. It includes Schwern's patch to localize the <code>CORE::GLOBAL::caller</code> override to just the <code>uplevel</code> call as well as my own additions to work with any existing <code>caller</code> override from within the <code>uplevel</code> call.</p><p>This <i>should</i> help it play nice with things like Contextual::Return or Hook::LexWrap that also play games with <code>CORE::GLOBAL::caller</code>.</p><p>In particular, since Test::Exception uses Sub::Uplevel, it should be safe to use Test::Exception with code that happens to use one of those other modules.</p><p>If you are in that situation, please give it a try and let me know how it works for you.</p> dagolden 2007-07-06T03:12:38+00:00 journal Become a CPAN Tester with CPAN::Reporter http://use.perl.org/~dagolden/journal/31534?from=rss <p> <a href="http://search.cpan.org/dist/CPAN-Reporter/">CPAN::Reporter</a> has had over 20 releases since I first <a href="http://use.perl.org/~dagolden/journal/30763">announced</a> it here on August 26. Now anyone with an up-to-date CPAN.pm can contribute to <a href="http://cpantesters.perl.org/">CPAN Testers</a> -- no smoke server needed.</p><p>Some of the major developments since August:</p><ul> <li>Full CPAN.pm support since version 1.88; CPAN::Reporter added to Bundle::CPANxxl</li><li>Added interactive configuration direct from CPAN.pm with: '<tt>o conf init test_report</tt>'</li><li>Added customization of action prompt defaults based on the test report grade. E.g., CC author on FAIL, but not otherwise</li><li>Distributions that fail tests but are missing prerequisites are marked as NA instead of FAIL</li><li>Test report includes prerequisite versions, environment variables, special Perl variables, and installed versions of the toolchain (e.g. ExtUtils::MakeMaker)</li><li>Added support for old style "test.pl" files</li></ul><p>Becoming a CPAN Tester with CPAN::Reporter is easy. From the CPAN shell prompt:</p><blockquote><div><p> <tt>&nbsp; cpan&gt; install CPAN::Reporter<br>&nbsp; cpan&gt; reload cpan<br>&nbsp; cpan&gt; o conf init test_report<br>&nbsp; cpan&gt; o conf commit</tt></p></div> </blockquote><p>Future development work on CPAN::Reporter will include better handling for interrupted tests and better integration with CPAN.pm to be able to report failures in Makefile.PL and "make" even before tests are run.</p><p>Come be a CPAN Tester today!</p> dagolden 2006-11-07T13:19:03+00:00 journal Are you using Vanilla/Strawberry Perl? Tell me about it! http://use.perl.org/~dagolden/journal/31193?from=rss <p>I'm putting together a presentation on Vanilla/Strawberry Perl for Perl Seminar NY</p><p>I'd love to include some real-world examples of how people are using it.</p><p>As of this morning, there have been 1500 downloads of various Vanilla/Strawbeery releases since my YAPC 2006 hackathon release in July. If you're one of those, please post a comment or send me an email with a couple lines about your experience.</p><p>Thanks!</p> dagolden 2006-10-02T13:20:01+00:00 journal RFC: Portable Alien Library System http://use.perl.org/~dagolden/journal/31107?from=rss <p> <i>This post was orginally made as <a href="http://win32.perl.org/wiki/index.php?title=Portable_Alien_Library_System"> Portable Alien Library System</a> on <a href="http://win32.perl.org/">win32.perl.org</a>. It is reposted here for wider commentary. Some additional design details, <a href="http://win32.perl.org/wiki/index.php?title=Talk:Portable_Alien_Library_System"> existing commentary</a> and references can be found on that page.</i> </p><p> <b>Synopsis</b> </p><ul> <li>Modules using "libfoo" list Alien::Foo as dependency </li><li>Alien::Foo namespace used to manage external libraries </li><li>Alien library files contained in the corresponding auto directory (auto/Alien/Foo) </li><li>Alien modules provide an API for CC flags in Makefile.PL/Build.PL </li><li>Alien modules verify library availability (e.g. in platform or Alien standard directories) </li><li>Alien modules responsible for installing binary library files when necessary </li><li>Alien modules support binary package download from a repository </li><li>Alien modules support source compilation or binary re-packaging to the binary package format used for installation </li></ul><p> <b>Introduction</b> </p><p>The Vanilla Perl Project provides a Win32 Perl and bundled compiler to build Perl modules from CPAN rather than relying on binary module package systems like PPM. Early feedback on the project demonstrate the many popular Perl modules that have dependencies on external libraries -- ones often assumed to exist on a Unix-based operating system.</p><p>This document describes the Portable Alien Library System (PALS) -- a framework for external library dependency specification and resolution. While its inspiration is drawn from Win32, its design is intended to provide benefits to module authors and users on all platforms.</p><p> <b>Requirements</b> </p><p>This list of functional objectives is adapted from the original <a href="http://search.cpan.org/dist/Alien/">Alien</a> module and commentary in <a href="http://win32.perl.org/wiki/index.php?title=External_Library_Handling">External Library Handling</a> at win32.perl.org.</p><p> <i>Dependency specification</i> </p><p>The solution must provide a way for Perl module authors to specify a dependency on an external library. An important consideration is how a solution integrates with or diverges from existing CPAN methods and tools.</p><p> <i>Library detection</i> </p><p>Libraries may be pre-installed in well known locations (e.g. "/lib") or may be installed ad hoc. The solution will need to detect if a library (and associated headers) are available and identify the associated paths.</p><p> <i>Library configuration</i> </p><p>Given the wide variety of locations for libraries, the solution needs to provide an abstraction layer to other tools that ensures proper configuration. Examples include:</p><ul> <li>Makefile.PL: compiler flags, including library and header paths </li><li>Shared/dynamic library loading: libraries in non-standard locations may need changes to environment settings such as PATH or LD_LIBRARY_PATH to ensure libraries are found at runtime </li><li>Other metadata: library metadata, such as version, may be required to manage dependencies </li></ul><p> <i>Binary library package generation</i> </p><p>External library compilation from source can be substantially more time-consuming than typical XS compilation for Perl modules. Providing binaries of external libraries (whenever possible) from an online repository will provide a more user-friendly experience.</p><p> <i>Binary library package installation</i> </p><p>Whether compiled from source or downloaded as a binary, external modules will need to be installed such that they may be located later for library detection and configuration. The solution will need to address whether this should be platform independent or platform specific.</p><p> <b>Design criteria</b> </p><p> <i>Portable</i> </p><p>External library support is a potential problem all platforms -- not just Win32 -- and developers will be more inclined to use the solution to specify external dependencies if it addresses dependency problems portably. The solution should provide a standard interface for module developers and module users that abstracts platform and compiler details.</p><p> <i>Modular</i> </p><p>Library packagers may not wish to take on responsibility for creating a solution for all platforms or all compilers or may lack the necessary know-how. The solution should modularize functionality wherever possible to allow individual library packagers to work on separate parts of the solution for platforms of interest.</p><p> <i>User-friendly</i> </p><p>The solution should degrade gently. If library support is not available for a particular platform or compiler, the solution should fail cleanly with an explanation of the dependency problem prior to generating XS compilation failures during linking.</p><p> <i>Maintainable</i> </p><p>Ongoing availability or upgrades of an external library should not depend on a single individual or on tacit, undocumented packaging know-how. As with Perl::Dist::*, the published solution should contain an automated process to convert an external library from source (or binary package) to the binary package format used by the solution.</p><p> <b>Design overview and commentary</b> </p><p>The following design description will make more sense if you examine the <a href="http://vanillaperl.com/images/alien_ecosystem.png">Alien Ecosystem</a> diagram.</p><p> <i>Dependency specification</i> </p><p>CPAN modules with external library dependencies will specify a module dependency on a corresponding Alien:: module, e.g. Alien::Foo for the "libfoo" module. Dependency management can thus be handled within the existing system using ExtUtils::MakeMaker, Module::Build or Module::Install prerequisite specifications.</p><p>Presence of an installed Alien::Foo should be considered evidence that libfoo is available. Alien::Foo must not install if libfoo cannot be found.</p><p> <i>Library detection</i> </p><p>Library searches will need to be done in platform specific ways. Perl Config values like "libpth" could be used as a starting point.</p><ul> <li>Win32: should also include the LIB environment variable and possibly the PATH environment variable for library searches </li><li>Unix: should include the LD_LIBRARY_PATH environment variable (or equivalents); could also use the output of "ldconfig -p" </li></ul><p> <i>Library configuration</i> </p><ul> <li>Makefile.PL: Alien::Base will provide an API for generating CC flags for both libraries and headers. </li><li>Shared/dynamic library loading: Will need to set the shared library search path before bootstrapping XS. This could be done during import of "use Alien::Foo" or could be done via an explicit class method call. </li><li>Other metadata: Alien::Foo could be extended with methods for library metadata. </li></ul><p> <i>Binary library package generation</i> </p><p>Alien::Foo::* should include the full procedure needed to generate a binary library package (a ".pal" file) for a particular platform. This should include downloading sources from a URL, any preparation work or patching, and generation of the binary.</p><p>This process could be done completely from the upstream library source, or could be a "repackaging" of another binary download, e.g. creating wrappers or definition files around a<nobr> <wbr></nobr>.dll on Win32.</p><p>At worst, this might just be a series of system() calls. The point is to ensure that the full process is available for future packagers to replicate, adapt or enhance.</p><p>Alien::Base should provide helper methods for URL downloading, PAL file generation, etc.</p><p>After the PAL file is generated, it can be uploaded to a repository for end-users to download for installation.</p><p> <i>Binary library package installation</i> </p><p>If not available already in platform-standard library directories, the library headers and binaries should be installed into the corresponding "auto" directory for the Alien module. E.g., "site/auto/Alien/Foo" for Alien::Foo. This location should be well-defined for all platforms, since it's the same as is used for XS modules.</p><p>Exact layout within this module is still open -- should there be "lib" and "include" subdirectories or should libraries and headers be combined in the top directory. Installation location of binary utilities included with libraries is still an open question.</p><p>The binary package may be either generated from source or downloaded from a URL (the default option). </p> dagolden 2006-09-24T12:41:08+00:00 journal Strawberry Perl 5.8.8 Alpha 2 released http://use.perl.org/~dagolden/journal/30785?from=rss <p> <a href="http://vanillaperl.com/">Strawberry Perl</a> 5.8.8 Alpha 2 for Win32 is out. Largely synchronizes with Vanilla Perl Build 7, but with updates to CPAN.pm and other toolchain modules.</p><ul> <li> <a href="http://vanillaperl.com/files/strawberry-perl-5.8.8-alpha-2.exe">Download strawbery-perl-5.8.8-alpha-2.exe</a> (24.0 MB) <a href="http://svn.phase-n.com/svn/cpan/tags/Perl-Dist-Strawberry/release-0.1.2/Changes">(Changelog)</a> </li></ul> dagolden 2006-08-29T04:35:50+00:00 journal CPAN.pm adds support for CPAN::Reporter http://use.perl.org/~dagolden/journal/30763?from=rss <p>As of version 1.87_57, <a href="http://search.cpan.org/dist/CPAN/">CPAN.pm</a> has support for <a href="http://search.cpan.org/dist/CPAN-Reporter/">CPAN::Reporter</a>. I wrote about this project in my last journal entry. At that time, one needed to install a subversion branch of CPAN.pm to use CPAN::Reporter. Now, it's easy to set up directly from the CPAN shell: </p><blockquote><div><p> <tt>cpan&gt; install ANDK/CPAN-1.87_57.tar.gz<br>cpan&gt; install CPAN::Reporter<br>cpan&gt; reload cpan</tt></p></div> </blockquote><p>If you're not prompted to adjust your CPAN configuration settings -- including enabling CPAN::Reporter -- you need to do it manually:</p><blockquote><div><p> <tt>cpan&gt; o conf test_report 1<br>cpan&gt; o conf commit</tt></p></div> </blockquote><p>The first time CPAN::Reporter runs, it will create a default configuration file where you will need to put your email address, so just test CPAN::Reporter again to create the config file:</p><blockquote><div><p> <tt>cpan&gt; test CPAN::Reporter</tt></p></div> </blockquote><p>Set your email address in <code>~/.cpanreporter/config.ini</code> (found in "My Documents" for Win32) and you're ready to contribute to <a href="http://testers.cpan.org/">CPAN Testers</a>.</p><p>There are certainly still bugs to work out, but the more people who give it a try, the sooner the bugs will be found. So please give it a go.</p><p>-- dagolden</p> dagolden 2006-08-27T00:29:12+00:00 journal Bringing Test::Reporter to CPAN.pm http://use.perl.org/~dagolden/journal/30571?from=rss <p>I've just uploaded <a href="http://search.cpan.org/perldoc?CPAN%3A%3AReporter">CPAN::Reporter</a> to CPAN and it should soon be available on mirrors worldwide.</p><p>As part of my goal of improving the availability of test information for <a href="http://vanillaperl.com/">Vanilla Perl</a>, I decided to try to get <a href="http://search.cpan.org/perldoc?Test%3A%3AReporter">Test::Reporter</a> working with <a href="http://search.cpan.org/perldoc?CPAN">CPAN</a> rather than <a href="http://search.cpan.org/perldoc?CPANPLUS">CPANPLUS</a>. The result is <a href="http://search.cpan.org/perldoc?CPAN%3A%3AReporter">CPAN::Reporter</a>, though, along the way, I also wound up writing <a href="http://search.cpan.org/perldoc?Tee">Tee</a> for portability.</p><p>Standard CPAN.pm doesn't yet support it, but I'm consulting with the CPAN maintainer and have created a subversion branch with working support. It's now at the point where it could use some real-world testing and feedback before it can be considered for inclusion in the main line of CPAN. If you're not afraid of running bleeding edge CPAN code, please consider giving it a try.</p><p>Here is an excerpt from the CPAN::Reporter Pod -- please see the full documentation for more details, including how to install the CPAN.pm branch that supports it.</p><blockquote><div><p><i> <b>SYNOPSIS</b></i> </p><ol> <li>Install a version of CPAN.pm that supports CPAN::Reporter</li><li>Install CPAN::Reporter</li><li>Edit<nobr> <wbr></nobr>.cpanreporter/config.ini</li><li>Test/install modules as normal with "cpan" or "CPAN::Shell"</li></ol><p> <b>DESCRIPTION</b> </p><p>CPAN::Reporter is an add-on for the CPAN.pm module that uses Test::Reporter to send the results of module tests to the CPAN Testers project.</p><p>The goal of the <a href="http://testers.cpan.org/">CPAN Testers</a> Projectis to test as many CPAN packages as possible on as many platforms as possible. This provides valuable feedback to module authors and potential users to identify bugs or platform compatibility issues and improves the overall quality and value of CPAN.</p><p>One way individuals can contribute is to send test results for each module that they test or install. Installing <tt>CPAN::Reporter</tt> gives the option of automatically generating and emailing test reports whenever tests are run via <tt>CPAN.pm</tt>.</p></div> </blockquote><p>Any other feedback or suggestions are greatly appreciated.</p> dagolden 2006-08-09T04:30:06+00:00 journal