nik's Journal http://use.perl.org/~nik/journal/ nik's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:21:26+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 nik's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~nik/journal/ Parameter validation with Params::Validate/Exception::Class http://use.perl.org/~nik/journal/32155?from=rss Yesterday I wrote up some notes on <a href="http://jc.ngo.org.uk/blog/2007/01/15/perl-parameter-validation-and-error-handling/">using Params::Validate and Exception::Class together</a> that I thought might be helpful. nik 2007-01-16T09:05:04+00:00 journal SVN::Web 0.50 released http://use.perl.org/~nik/journal/32129?from=rss After a slightly longer gestation period than I'd planned SVN::Web 0.50 is now available. More details in this <a href="http://jc.ngo.org.uk/blog/2007/01/12/svnweb-050-released/">blog entry</a>. nik 2007-01-12T17:28:54+00:00 journal use.perl journal full text RSS feed http://use.perl.org/~nik/journal/31814?from=rss If, like me, you'd like an RSS feed that includes the full text of all use.perl journals (not just 'friends'), then <a href="http://jc.ngo.org.uk/blog/2006/12/05/useperl-journals-and-full-text-feeds/">I may be able to help you</a>. nik 2006-12-05T21:54:37+00:00 useperl Hello Technorati http://use.perl.org/~nik/journal/31754?from=rss This entry exists so that I can claim this blog on Technorati. Move along, there's nothing to see here. <a href="http://www.technorati.com/claim/auuq5peiby">Technorati Profile</a> nik 2006-11-29T16:53:03+00:00 journal SVN::Web and Google code hosting http://use.perl.org/~nik/journal/31750?from=rss <p>[This is a repost from <a href="http://jc.ngo.org.uk/blog/2006/11/29/svnweb-and-google-code-hosting/">here</a>.]</p><p>You&#8217;re probably aware of <a href="http://code.google.com/hosting/">Google&#8217;s code hosting service</a>. They use Subversion as their revision control system, so if you want to contribute to a project hosted there you really need a Subversion client.</p><p>I was reading the <a href="http://code.google.com/hosting/faq.html">FAQ for the hosting service</a> the other day and a <a href="http://code.google.com/hosting/faq.html#viewvc">particular entry </a>struck me.</p><blockquote><div><p> <b>Do you have support for ViewVC or similar repository-browsing tools?</b> </p><p>This feature isn&#8217;t implemented yet, but you can still browse the latest version of your project&#8217;s code by entering the repository URL into a web browser.</p></div> </blockquote><p>Since <a href="http://search.cpan.org/~nikc/SVN-Web/">SVN::Web</a> now supports (well, it does on my <a href="http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web/branches/svn-client">development site</a>, which will be released as 0.50 real soon now) accessing remote repositories, it should be pretty easy to have it talk to the repositories hosted at Google.</p><p>A minor wrinkle is that SVN::Web expects to be configured with a list of available repositories. But that&#8217;s only a small code change to solve. So I&#8217;ve done that, and put together a proof-of-concept SVN::Web installation.</p><p>The practical upshot of which is that if you have a project at Google Code Hosting, you can take this URL: <code>http://jc.ngo.org.uk:81/svnweb/</code>, append the name of your project, and get a decent web interface on to your repository.</p><p>For example, the Reconstructor project (chosen at random) gets this URL: <a href="http://jc.ngo.org.uk:81/svnweb/reconstructor">http://jc.ngo.org.uk:81/svnweb/reconstructor</a>.</p><p>This is very definitely a <b>proof of concept</b>, and may well disappear if it gets completely hammered, and/or starts sucking up all my hosted bandwidth. </p> nik 2006-11-29T09:50:56+00:00 journal Test / scratch slashcode system http://use.perl.org/~nik/journal/31650?from=rss <p>I'm considering writing Plagger::Plugin::Publish::Blogger, using Net::Blogger to publish journal entries found by Plagger to blogs that Net::Blogger supports -- which includes Slashcode journals.</p><p>If possible, I'd like to avoid having to install a complete copy of Slash to test this.</p><p>Does anyone know of any existing test/scratch Slashcode installations on the 'net that support this?</p> nik 2006-11-18T09:10:52+00:00 journal "..", LHS larger than RHS gives empty list. Why? http://use.perl.org/~nik/journal/31626?from=rss <p>From perlop</p><blockquote><div><p>Binary ".." is the range operator, which is really two different operators depending on the context. In list context, it returns a list of values counting (up by ones) from the left value to the right value. <b>If the left value is greater than the right value then it returns the empty list.</b></p></div> </blockquote><p>(emphasis added). So:</p><p>@a = (1<nobr> <wbr></nobr>.. 3); # @a = (1, 2, 3)<br> @b = (3<nobr> <wbr></nobr>.. 1); # @b = (), *not* @b = (3, 2, 1)<br> @c = reverse (1<nobr> <wbr></nobr>..3); # @c = (3, 2, 1)</p><p>Why is this? I had occasion to do this <a href="http://jc.ngo.org.uk/svnweb/jc/revision/?rev=1274">the other day</a>, and my first cut at the code did the equivalent of @b above, instead of @c, which led to a few moments headscratching.</p><p>It struck me as odd that Perl, which so often DWIMs, gets it wrong here.</p> nik 2006-11-16T13:48:20+00:00 journal Beta testers for SVN::Web 0.50 wanted http://use.perl.org/~nik/journal/31559?from=rss <p>I'm &gt; this close to releasing SVN::Web 0.50. There are several major<br>changes in this version, the most dramatic of which is that SVN::Web now<br>supports accessing remote repositories.</p><p>In other words, you can have something like this in your config.yaml:</p><p> &nbsp; &nbsp; repos:<br> &nbsp; &nbsp; &nbsp; &nbsp; local_repo: file:///path/to/repo<br> &nbsp; &nbsp; &nbsp; &nbsp; some_repo: svn://host/path/to/repo<br> &nbsp; &nbsp; &nbsp; &nbsp; other_repo: http://host/path/to/repo</p><p>and it'll work. The work to do this has also meant that the Diff and<br>Revision operations are considerably faster and more memory efficient. Also, SVN::Web::Blame is now implemented, which I know some people have been<br>waiting for.</p><p>If you'd like to try out 0.50, fetch the code from:</p><p> &nbsp; &nbsp; &nbsp; svn://jc.ngo.org.uk/nik/CPAN/SVN-Web/branches/svn-client/</p><p>and run:</p><p>% perldoc CHANGES.pod<br>% perldoc UPGRADING.pod<br>% perl Build.PL<br>%<nobr> <wbr></nobr>./Build<br>%<nobr> <wbr></nobr>./Build test<br>%<nobr> <wbr></nobr>./Build install --install_base<nobr> <wbr></nobr>/some/path</p><p>(use the --install_base option if you don't want to overwrite a pre-existing<br>SVN::Web installation)</p><p>Feedback welcomed.</p> nik 2006-11-10T08:48:01+00:00 journal Sync Wordpress blog to use.perl? http://use.perl.org/~nik/journal/31450?from=rss Anyone know of an off-the-shelf way to propogate posts from a Wordpress blog to a use.perl blog? nik 2006-10-30T20:00:13+00:00 journal I won an Ultra 40 / Playing with Plagger http://use.perl.org/~nik/journal/31436?from=rss <p>Rar. Details are <a href="http://jc.ngo.org.uk/blog/2006/10/27/i-didnt-win/">here</a>. One of the things I plan on working on is Perl + DTrace.</p><p>As an aside, I'm experimenting with what I'm calling a "Personal Planet" (no idea if that's already been coined, it just came to me). Having heard numerous people rave about Plagger I've set up an installation to aggregate feeds from the various places that I might publish content. Here, the previously linked blog, RSS feeds of commits to my Subversion repo, etc. You can see the result <a href="http://jc.ngo.org.uk/~nik/planet-nik/">here</a>. Since this post is going to be included in that feed it's all going to feel a bit meta.</p> nik 2006-10-27T19:59:58+00:00 journal SVN::Web, remote repos redux, and syntax highlighting http://use.perl.org/~nik/journal/31321?from=rss <p>On 8th September I wrote about adding support to SVN::Web for browsing repositories hosted remotely.</p><p>As I said at the <a href="http://use.perl.org/~nik/journal/30930">time</a>,</p><blockquote><div><p>Also, it only seems to work for file:/// and svn:// URIs at the moment.</p></div></blockquote><p> <a href="http://use.perl.org/~jk2addict/">j2kaddict</a> suggested that I look at using SVN::Client rather than SVN::Ra, so I've been doing that off and in my spare time.</p><p>I've finally tracked down the last of the memory allocation errors (SVN::*'s integration with Subversion's memory allocation routines requires more thought than is typically required with Perl) and now have something that's nearer release, on the <a href="http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web/branches/svn-client/">svn-client branch</a>.</p><p>This has slowed down some operations, but sped up others. For example, carrying out diffs (and viewing everything that's changed in a particular revision) is now much faster, as the diffing is carried out by the Subversion libs, instead of using Text::Diff.</p><p>The tradeoff is that some features have currently fallen by the wayside. For example, the diff output is currently raw, and not nicely highlighted.</p><p>Which brings me on to my next investigation -- I need to find a decent mechanism for doing syntax highlighting with Perl.</p><p>I've looked at Syntax::Highlight::Engine::Kate, but it looks as though if I want to change the look of the highlighting I need to subclass a bunch of modules, which is effort I'd like to avoid if possible.</p><p>Syntax::Highlight::Universal looks interesting, but doesn't build on my local FreeBSD box. That doesn't bode well for it being easy to install for people who are going to be using my code that depends on it. It also looks like it introduces a dependency on Java, which is a bit heavier than I really want.</p> nik 2006-10-15T09:50:57+00:00 journal SVN::Web, easier third party action integration http://use.perl.org/~nik/journal/31278?from=rss <p>I knew there was something else I wanted to mention in my previous entry. I've been doing a little more hacking on SVN::Web recently.</p><p>One of the things that's bugged me about SVN::Web is the hoops a user needs to go through if they want to write their own SVN::Web::action class.</p><p>As well as writing the class (which is relatively easy), they need to provide templates for the interface, and localisation files for any language strings that they use.</p><p>This has been unduly difficult in the past. Although SVN::Web has supported looking for templates in multiple directories for some time now, localisation files have always had to live in one location.</p><p>Worse, you couldn't 'override' a localisation string that ships with SVN::Web with one of your own, without changing the files that ship with SVN::Web.</p><p>This makes upgrading more complicated than it needs to be, and means that the instructions for installing any third party actions would be more complicated than is really necessary.</p><p>So, over the weekend I finally did the work to make SVN::Web support localisation files in multiple directories. This means that you can use the localisation files that ship with SVN::Web, and selectively override one or more of the localisation strings by creating your own language.po file in your own directory -- future SVN::Web upgrades won't overwrite it. And installing third party actions is now simpler, since they can ship with their own localisation files, and now it's just a config.yaml tweak to have them used.</p><p>To do this I had to change the i18n infrastructure somewhat. Previously (I inherited this from clkao) SVN::Web used Locale::Maketext::Simple. As the name implies, this does make localisation quite simple, but at the cost of some flexibility. One of the things it couldn't do was support localisations in multiple directories.</p><p>So I had to roll my own SVN::Web::I18N, using Locale::Maketext and Locale::Maketext::Lexicon. The code itself is pretty simple, but I had a few headscratching moments trying to work out how to tie it together, as I found the Locale::Maketext* documentation a little confusing.</p><p>These changes were in r1178 (<a href="http://jc.ngo.org.uk/svnweb/jc/revision?rev=1178">http://jc.ngo.org.uk/svnweb/jc/revision?rev=1178</a>), and you can see SVN::Web::I18N too (<a href="http://jc.ngo.org.uk/svnweb/jc/view/nik/CPAN/SVN-Web/trunk/lib/SVN/Web/I18N.pm">http://jc.ngo.org.uk/svnweb/jc/view/nik/CPAN/SVN-Web/trunk/lib/SVN/Web/I18N.pm</a><nobr>)<wbr></nobr> .</p><p>Now if I can just get SVN::Client support working I can get on and release SVN::Web 0.50. And with that out of the door I can release a couple of other modules I've had hanging around -- SVN::Web::Timeline (<a href="http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web/branches/timeline/">http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web/branches/timeline/</a>), which generates a nifty AJAX timeline of commits to the source code, and SVN::Web::Search (<a href="http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web-Search/trunk/">http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web-Search/trunk/</a>) which indexes Subversion repositories and lets you search over them.</p> nik 2006-10-10T10:21:39+00:00 journal MiltonKeynes.pm tech meeting tonight, SOAP/XML verbosity http://use.perl.org/~nik/journal/31277?from=rss <p>There's a MiltonKeynes.pm (<a href="http://miltonkeynes.pm.org/">http://miltonkeynes.pm.org/</a>) technical meeting this evening, organised by Tom Hukins.</p><p>Now I just need to finish my slides...</p><p>Which are nothing to do with what I've been working on recently. $dayjob has me wrapping a third party's SOAP API. We need to do this because although they have a very pretty web site for carrying out various back-end operations, it doesn't meet our audit requirements. It would also mean retraining our ops people to stop using our systems and start using theirs.</p><p>This is my first foray in to SOAP, and it's going reasonably well. But what hadn't occured to me is how much of the verboseness of XML would have to spill over in to my code.</p><p>I'm trying to make sure that the wrapper I'm writing is going to be easy to use for someone who's experienced with Perl, as well as looking close to the underlying API for someone who's familiar with that -- I'm not entirely sure why, as I'm not sure this code is ever going to see the light of day outside of $dayjob, but I get a definite sense of satisfaction out of doing these things properly.</p><p>Anyway, I digress.</p><p>So the responses back from the API all come back as XML, naturally, and SOAP::Lite does a reasonably good job of turning these in to useful Perl datastructures. I then convert these (hashes, normally) in to objects, and return the objects.</p><p>This forces the caller to use get_ and set_ methods to retrieve the data, which is less error prone than having them specify the hash keys directly.</p><p>And this is where the XML verbosity spills over in to the client interface. Handed XML like this:</p><blockquote><div><p> <tt>&lt;OptimisticLockValue&gt;...&lt;/OptimisticLockValue&gt;<br>&lt;CustomerId&gt;...&lt;/CustomerId&gt;<br>&lt;<nobr>U<wbr></nobr> seDomainDefaultSettings&gt;...&lt;/UseDomainDefaultSettings&gt;<br>...</tt></p></div> </blockquote><p>the client API looks like this:</p><blockquote><div><p> <tt>$ss-&gt;get_optimistic_lock_value();<br>$ss-&gt;get_customer_id();<br>$ss-&gt;get_use_domain<nobr>_<wbr></nobr> default_settings();</tt></p></div> </blockquote><p>(not sure why u.p is inserting spaces in to some of those).</p><p>Still, I suppose it makes the code more readable in the long run.</p> nik 2006-10-10T10:06:56+00:00 journal Perl and different compilers / optimisation options http://use.perl.org/~nik/journal/30936?from=rss <p>I've done a little bit of work recently to see how differently Perl performs on different operating systems compiled with different compilers and optimisation settings.</p><p>Running FreeBSD and Solaris on a Sun Ultra 40 (2 x AMD Opteron 2.8GHz CPUs, 4GB RAM), with Perl compiled with gcc and Sun's compiler (with and without optimisation), using SpamAssassin 3.1.5 as the test workload showed that Perl on FreeBSD compiled with 'gcc -O2' is distinctly faster than Perl on Solaris compiled with either gcc or Sun's compiler.</p><p>More details at:</p><p><a href="http://jc.ngo.org.uk/blog/2006/09/05/day-58-of-60-developer-benchmarks-pt-3/">http://jc.ngo.org.uk/blog/2006/09/05/day-58-of-60-developer-benchmarks-pt-3/</a></p><p>and</p><p><a href="http://jc.ngo.org.uk/blog/2006/09/06/day-59-of-60-developer-benchmarks-pt-4/">http://jc.ngo.org.uk/blog/2006/09/06/day-59-of-60-developer-benchmarks-pt-4/</a></p> nik 2006-09-09T08:11:04+00:00 journal SVN::Web and remote repositories http://use.perl.org/~nik/journal/30930?from=rss <p>I've had a couple of journeys in to London over the last few weeks, and I've been able to use the time on the train to include support for remote Subversion repositories in to SVN::Web.</p><p>In use it's identical to 0.49, except that you can now specify the repositories to browse using Subversion URLs.</p><p>So if you have this config fragment in your config.yaml:</p><p> &nbsp; &nbsp; &nbsp; repos:<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; my-repo:<nobr> <wbr></nobr>/path/to/my/repo</p><p>that still works. But you can now do:</p><p> &nbsp; &nbsp; &nbsp; repos:<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; my-repo: file:///path/to/my/repo</p><p>That's no great change. But you can also do this:</p><p> &nbsp; &nbsp; &nbsp; repos:<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; niks-repo: svn://jc.ngo.org.uk/<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; kde-repo: svn://anonsvn.kde.org/home/kde/</p><p>to link to remote repositories and browse them. So if your favourite open source project is using Subversion, but is using SomeOther(tm) web repository browser frontend you can fix that, without insisting that they install SVN::Web<nobr> <wbr></nobr>:-)</p><p>Obviously, this is going to be somewhat slower, so I recommend using the caching options in SVN::Web to speed things up. Also, it only seems to work for file:/// and svn:// URIs at the moment.</p><p>See this message <a href="http://jc.ngo.org.uk/pipermail/svnweb/2006-September/000018.html">http://jc.ngo.org.uk/pipermail/svnweb/2006-September/000018.html</a> for more details. The work's currently on a branch, retrievable with</p><p>svn checkout svn://jc.ngo.org.uk/nik/CPAN/SVN-Web/branches/svn_ra/</p><p>Feedback welcomed.</p> nik 2006-09-08T16:05:25+00:00 journal ministat redux http://use.perl.org/~nik/journal/30460?from=rss <p>ministat is now much more capable. It's grown decent documentation, a number of useful options, and can now generate plots in color.</p><p>Here are a couple of examples:</p><ul> <li> <a href="http://people.freebsd.org/~nik/ministat.png">Colour, default symbols</a> </li><li> <a href="http://people.freebsd.org/~nik/ministat2.png">Colour, custom symbols</a> </li></ul><p>And here's what the plain text output looks like:</p><blockquote><div><p> <tt>x d1<br>+ d2<br>* d3<br>: = Mean<br>M = Median<br>+----------------------------------------------------------------------<nobr>-<wbr></nobr> --+<br>|&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*&nbsp; &nbsp; *&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br>|&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;*&nbsp; &nbsp; *&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br>|x&nbsp; &nbsp; x&nbsp; &nbsp;x&nbsp; &nbsp; x&nbsp; &nbsp;x&nbsp; &nbsp; *&nbsp; &nbsp;*&nbsp; &nbsp; *&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;+&nbsp; &nbsp; +&nbsp; &nbsp;+&nbsp; &nbsp; +&nbsp; &nbsp;+&nbsp; &nbsp; &nbsp;|<br>|x&nbsp; &nbsp; x&nbsp; &nbsp;x&nbsp; &nbsp; x&nbsp; &nbsp;x&nbsp; &nbsp; *&nbsp; &nbsp;*&nbsp; &nbsp; *&nbsp; &nbsp; *&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; +&nbsp; &nbsp; +&nbsp; &nbsp;+&nbsp; &nbsp; +&nbsp; &nbsp;+&nbsp; &nbsp; +|<br>|&nbsp; &nbsp;|_____M:_______|&nbsp; &nbsp; &nbsp; |_M_:___|&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|_____M_:______|&nbsp; &nbsp; &nbsp;|<br>+--------------------------------------------------------------------------<nobr>+<wbr></nobr> <br>&nbsp; &nbsp; N&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Min&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Max&nbsp; &nbsp; &nbsp; &nbsp; Median&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Mean&nbsp; &nbsp; &nbsp; &nbsp; Stddev<br>x&nbsp; 11&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;6&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;3&nbsp; &nbsp; &nbsp;3.2727273&nbsp; &nbsp; &nbsp;1.6787441<br>+&nbsp; 11&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 12&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 17&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 14&nbsp; &nbsp; &nbsp;14.272727&nbsp; &nbsp; &nbsp;1.6787441<br>Difference at 95.0% confidence<br>&nbsp; &nbsp; &nbsp; &nbsp; 11 +/- 1.4932<br>&nbsp; &nbsp; &nbsp; &nbsp; 336.111% +/- 45.6255%<br>&nbsp; &nbsp; &nbsp; &nbsp; (Student's t, pooled s = 1.67874)<br>*&nbsp; 11&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;6&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;9&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;7&nbsp; &nbsp; &nbsp;7.3636364&nbsp; &nbsp; 0.92441628<br>Difference at 95.0% confidence<br>&nbsp; &nbsp; &nbsp; &nbsp; 4.09091 +/- 1.20535<br>&nbsp; &nbsp; &nbsp; &nbsp; 125% +/- 36.8301%<br>&nbsp; &nbsp; &nbsp; &nbsp; (Student's t, pooled s = 1.35512)</tt></p></div> </blockquote><p>Get the <a href="http://jc.ngo.org.uk/svnweb/jc/view/nik/ministat/trunk/ministat">ministat source code</a> and play around.</p> nik 2006-07-28T16:07:15+00:00 journal ministat http://use.perl.org/~nik/journal/30437?from=rss <p>[ Originally posted <a href="http://jc.ngo.org.uk/blog/2006/07/26/day-17-of-60-ministat/">here</a>. ]</p><p>I&#8217;ve spent some of today porting some useful statistics reporting software from C to Perl.</p><p> <a href="http://www.freebsd.org/cgi/cvsweb.cgi/src/tools/tools/ministat/">ministat</a> reads in two or more files of data and uses the Student&#8217;s t test to determine if there is any statistical difference between the means of the datasets. This is especially useful when comparing benchmark results.</p><p>For instance, I figure that this will be useful to compare data from several Sendmail runs, where the number of queue directories differ between the runs. It should highlight any benefits between different numbers of queue directories.</p><p>Rather than explain more here, I&#8217;ll point you at the <a href="http://jc.ngo.org.uk/svnweb/jc/view/nik/ministat/trunk/ministat">code</a>. There&#8217;s documentation towards the end of the file. Note that this still needs some work &#8212; there&#8217;s no proper command line option handling at the moment, some of the documentation needs fleshing out, and I wouldn&#8217;t use this as an example of good Perl code, as it still looks far too much like a Perl program that&#8217;s been written in C.</p><p>When I&#8217;ve fixed that I&#8217;ll put it up on CPAN.</p> nik 2006-07-26T21:06:53+00:00 journal Problems with M4, backticks, and changequote. http://use.perl.org/~nik/journal/30423?from=rss Anyone know much about M4? I've having a problem with backticks and changequote, and can't figure out why it's not working. If you might have an idea, I'd appreciate it if you'd take the time to take a look at <a href="http://jc.ngo.org.uk/blog/2006/07/25/day-16-of-60-i-strongly-dislike-m4/">this post</a> and let me know what I might be doing wrong. nik 2006-07-25T21:29:09+00:00 journal AJAX timelines and SVN::Web http://use.perl.org/~nik/journal/30201?from=rss <p>A day or so ago I started seeing references to <a href="http://simile.mit.edu/timeline/">AJAX timelines, by Simile</a> in a few blogs.</p><p>Thinking that they looked rather interesting, I've taken a couple of hours to integrate them in to SVN::Web.</p><p>The work's been carried out on <a href="http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web/branches/timeline/">this branch</a>, if you want to follow along.</p><p>The code works well enough to draw a <a href="http://jc.ngo.org.uk/svnweb/jc/timeline/nik/">scrollable timeline of Subversion commits</a>.</p> nik 2006-07-07T08:57:35+00:00 journal SVN::Web / dtrace http://use.perl.org/~nik/journal/30168?from=rss <p>It's been a busy few months.</p><p>Most of my Perl work has been concentrated on SVN::Web. Since I last wrote in here there have been 6 other releases.</p><p>As well as the usual round of bug fixes, new features include:</p><p>* Better RSS support.</p><p>* Localisation selection from the web interface.</p><p>* Caching</p><p>* A few thousand additional tests, to check that mod_perl and apache/cgi support work.</p><p>* mod_perl2 support</p><p>* A standalone webserver that can be used for testing, or environments where full blown Apache isn't necessary.</p><p>The other thing I plan on doing is taking a deeper look at dtrace on Solaris (and possibly FreeBSD). This with an eye to building on some earlier that people have done instrumenting Perl with dtrace. More details on this over at <a href="http://jc.ngo.org.uk/blog/">http://jc.ngo.org.uk/blog/</a>.</p> nik 2006-07-04T16:02:35+00:00 journal SVN::Web 0.42 released http://use.perl.org/~nik/journal/28027?from=rss <p>I've just released SVN::Web 0.42. This release is a mix of bug fixes and architectural changes to make it easier for people to build on SVN::Web.</p><p>One of the big changes is that now, instead of calling die() at various points, the majority of the code is now exception based, using SVN::Web::X (which derives from Exception::Class).</p><p>I'm not convinced that I'm quite using exceptions in the 'proper' fashion. Instead of having different classes of exception (::PathIsMissing,<nobr> <wbr></nobr>::RevisionDoesNotExist, and so on), there's a single exception type (the aforementioned SVN::Web::X. When throwing an exception the code includes a short error message, and then the localisation system is responsible for turning that in to a pretty web page.</p><p>Since SVN::Web only has two notional layers (SVN::Web, which dispatches to SVN::Web:: actions, and the actions themselves) I don't think that I'm missing much here.</p><p>There's a handful of new functionality -- you can now easily view diffs between arbitrary revisions of a file, for instance, which is quite handy.</p><p>And a few things have changed under the hood to make it easier for third-party SVN::Web:: to hook in and behave like fully integrated parts of the original codebase. This will make it much easier for my next trick, SVN::Web::Search, which will put a search interface on top of all the log messages, revisions, dates, and authors in your repository...</p> nik 2005-12-16T16:11:38+00:00 journal SVN::Web, Template::Plugin::{Subst, MultiFilter} http://use.perl.org/~nik/journal/27521?from=rss <p>As described in an earlier entry, new versions of <a href="http://search.cpan.org/dist/SVN-Web/">SVN::Web</a> escaped over the weekend. First there was 0.40, which contained the bulk of the new code. Then 0.41 followed when I realised I'd made a few packaging cock ups.</p><p>One of the things you can do with the new version is pass SVN commit messages through multiple TT filters. Each filter can decorate the commit message in some way -- recognise URLs, e-mail addresses, RT ticket numbers, subversion revision numbers, etc, and convert them in to hyperlinks.</p><p>Say you've written "Fixes rt#1234" in your commit message. You'd like "rt#1234" to link straight to your bug database.</p><p>I thought that would be easy. Obviously, in straight Perl it's just the equivalent of:</p><blockquote><div><p> <tt>$msg =~ s/rt#(\d+)/&lt;a href="/rt.cgi?t=$1"&gt;$1&lt;/a&gt;/;</tt></p></div> </blockquote><p>I was quite surprised that TT doesn't offer this functionality. The built in <tt>replace</tt> method/filter doesn't cut it, as it doesn't handle backreferences. The TT:</p><blockquote><div><p> <tt>[% msg.replace('rt#(\d+)', '&lt;a href="/rt.cgi?t=$1"&gt;$1&lt;/a&gt;') %]</tt></p></div> </blockquote><p>results in the literal <tt>$1</tt> being inserted into the output instead of the expected back reference.</p><p>The two popular suggestions in the TT list archives are to either use s/// directly in a <tt>[% PERL %]</tt> block, or write a custom filter to do the specific search and replace.</p><p>But <tt>[% PERL %]</tt> blocks might not be enabled in every TT install. And writing multiple filters (one to recognise RT tickets, one for subversion revision numbers, one for Sourceforge tickets,<nobr> <wbr></nobr>...) would rapidly get boring.</p><p>Hence, <a href="http://search.cpan.org/dist/Template-Plugin-Subst/">Template::Plugin::Subst</a>. With it, you can write:</p><blockquote><div><p> <tt>[% msg.subst('rt#(\d+)', '&lt;a href="/rt.cgi?t=$1"&gt;$1&lt;/a&gt;') %]</tt></p></div> </blockquote><p>and it will do the right thing. And as well as working as vmethod, it also works as a filter, so you can use that syntax instead.</p><p>The other thing that TT can't easily and cleanly do is stack multiple filters together, ignoring the ones that aren't installed.</p><p>SVN::Web ships with a config file that specifies a number of TT plugins to filter log messages through (to convert them to HTML, and make various things in the log clickable). These include plugins that might not be installed. In TT you can't say "Run all these filters, ignoring the ones that aren't installed". Trying to use a missing plugin (sensibly) generates a run-time error.</p><p>So, SVN::Web's <a href="http://jc.ngo.org.uk/~nik/cgi-bin/svnweb/index.cgi/jc/view/nik/CPAN/SVN-Web/trunk/lib/SVN/Web.pm">log_msg_filter()</a> implements this functionality. And the default SVN::Web configuration file can safely specify several different plugins, without the user having to worry about whether or not they're installed. Without them, everything works, and if they are installed then the user gets a slightly nicer output.</p><p>At the moment this is hardcoded in to SVN::Web, but I plan on releasing Template::Plugin::MultiFilter in the near future so that others can easily use this functionality.</p> nik 2005-11-09T11:03:52+00:00 journal New version of SVN::Web in the offing http://use.perl.org/~nik/journal/27415?from=rss <p>I've been spending much of the last few weeks polishing up SVN::Web. It's a simple way to browse SVN repositories online, view diffs, compare revisions, check out log entries, and so forth.</p><p>Originally written by Chia-liang Kao, of SVK fame, he's handed the maintainer reins over to me.</p><p>Changes in the new version include:</p><p>An <a href="http://jc.ngo.org.uk/~nik/cgi-bin/svnweb/index.cgi/jc/browse/nik/CPAN/SVN-Web/trunk/lib/SVN/Web/Template/trac/">overhauled set of templates</a>, so the out-of-the-box experience is much nicer.</p><p>A 'view' action that shows the contents of a file, along with the most recent log message.</p><p>More customisation options, making it easier to add or remove actions from the list of things that SVN::Web is allowed to do.</p><p>Added huge swathes of information to the documentation. It should now be much easier for anyone else to contribute code and/or templates to SVN::Web.</p><p>A fix for rt#12431.</p><p>A mechanism to 'stack' Template Toolkit filters. You can run easily run log messages through an arbitrary number of filters, without causing problems if one or more of the filters is not installed.</p><p>Inline display of some content from the repository where appropriate (e.g., viewing an image works properly).</p><p> <a href="http://jc.ngo.org.uk/~nik/cgi-bin/svnweb/index.cgi/jc/revision/?rev=467">Fixed a bug</a> that meant the results from the automated tests weren't valid. Fixed the bugs that this uncovered.</p><p>And <a href="http://jc.ngo.org.uk/~nik/cgi-bin/svnweb/index.cgi/jc/log/nik/CPAN/SVN-Web/trunk/?limit=120&amp;submit=Show">much more</a>. It'll be hitting CPAN shortly. In the meantime, you can see the code (and the new, shiny, trac inspired interface) <a href="http://jc.ngo.org.uk/~nik/cgi-bin/svnweb/index.cgi/jc/browse/nik/CPAN/SVN-Web/trunk/">here</a>.</p> nik 2005-11-02T12:52:29+00:00 journal Finding the memory hogs http://use.perl.org/~nik/journal/26968?from=rss <p>I sent this to the london.pm mailing list, and was surprised to get few responses. That probably means it's a hard problem. Still...</p><p> <tt>I'm having a stab at reducing the memory footprint of a Perl application I'm working with. When it's running it's ~80MB resident. I know it plays fast and loose with memory, allocating hashes and the like at the drop of a hat. What I'd like to know are the long hanging fruit that I can try and recode to be a little more memory friendly.</tt> </p><p> <tt>Devel::Symdump + Devel::Size gets me some of the way there. I can instrument the application by waiting until it's gone through its initialisation phase (after which the memory usage is reasonably constant), and then walk the symbol table, dumping the size of everything that's found.</tt> </p><p> <tt>But, natch, that won't work for lexical variables, many of which have been created in subs and had references returned.</tt> </p><p> <tt>Poking through CPAN hasn't turned up anything useful (although some of the B:: namespace looks like it could be coerced in to this). Anyone got any pointers to approaches that are likely to work?</tt> </p><p>Any takers?</p> nik 2005-10-02T20:10:06+00:00 journal 4 new Test::* modules http://use.perl.org/~nik/journal/25599?from=rss <p>I've been busy over the past week producing four new Test::* modules.</p><p>These are a bit different from most other Test::* modules, which are normally written to help you test your code. These, however, are designed to help you test the environment that your code is going to be running in.</p><p> <a href="http://search.cpan.org/~nikc/Test-Symlink/">Test::Symlink</a> lets you easily verify that a symlink exists, and that it links to the correct location.</p><blockquote><div><p> <tt>&nbsp; &nbsp; symlink_ok($src =&gt; $dst);</tt></p></div> </blockquote><p> <a href="http://search.cpan.org/~nikc/Test-Unix-Group/">Test::Unix::Group</a> and <a href="http://search.cpan.org/~nikc/Test-Unix-User/">Test:::Unix::User</a> let you verify that a user, group, and home directory exist.</p><blockquote><div><p> <tt>&nbsp; &nbsp; user_ok({ name =&gt; 'nik', uid =&gt; 1000}, 'nik has uid 1000');<br> <br>&nbsp; &nbsp; homedir_ok({ name =&gt; 'nik', group =&gt; 'staff' },<br>&nbsp; &nbsp; &nbsp; &nbsp; "nik's homedir is owned by the staff group");<br> <br>&nbsp; &nbsp; group_ok({ name =&gt; 'wheel', members =&gt; [qw(root nik)]},<br>&nbsp; &nbsp; &nbsp; &nbsp; "'wheel' has the correct members");</tt></p></div> </blockquote><p>Finally, there's <a href="http://search.cpan.org/~nikc/Test-Net-Connect/">Test::Net::Connect</a>, which lets you make sure that you can make a network connection to a particular host.</p><blockquote><div><p> <tt>&nbsp; &nbsp; connect_ok({ host =&gt; 'www.example.com', port =&gt; 80 },<br>&nbsp; &nbsp; &nbsp; &nbsp; "Web server is reachable from here");</tt></p></div> </blockquote> nik 2005-07-09T10:59:40+00:00 journal Test Anything in C http://use.perl.org/~nik/journal/22444?from=rss <p>libtap made it to version 1.0 the other day, and there don't seem to be too many embarassing mistakes, so...</p><p>libtap is an implementation of the Test::Harness protocol for reporting test successes/failures (ok, not ok, skip, TODO, etc) in C, making it ideal for when you need to write test cases in C (Perl not available, Inline::C not available, that sort of thing). Full documentation is included.</p><p>More information, including links to the tarball, MD5 checksums, read-only access to the Subversion repository, and the bug/feature ticket system is at <a href="http://jc.ngo.org.uk/trac-bin/trac.cgi/wiki">my Trac site</a>.</p><p>This has only been tested on FreeBSD 5.x and 6.x at the moment; if you're on those systems then you can use ports/devel/libtap for ease of installation. Tests on other systems (and tickets reporting issues / patches to fix them) would be very welcome.</p> nik 2004-12-24T09:49:16+00:00 journal libtap getting more useful http://use.perl.org/~nik/journal/22253?from=rss <p> <a href="http://use.perl.org/~nik/journal/22137">Last week</a> I wrote about some initial progress I was making with libtap, an implementation of the Test::More / Test::Harness protocol for communicating test results for tests written in C.</p><p>Since then I've managed to make a bit more progress.</p><ul> <li> <p>There's a <a href="http://jc.ngo.org.uk/cgi-bin/trac.cgi">website</a> for it now. This is also my opportunity to experiment with <a href="http://www.edgewall.com/trac/">trac</a>, a combination wiki, ticket system, and subversion repo browser.</p></li><li> <p>This means I've got <a href="http://jc.ngo.org.uk/trac-bin/trac.cgi/report/9">tickets</a> that let me keep track of suggestions, and see how much more work there is to go.</p></li><li> <p>Functionality improved, as can be seen at the <a href="http://jc.ngo.org.uk/trac-bin/trac.cgi/timeline?from=12%2F10%2F04&amp;daysback=30&amp;ticket=on&amp;changeset=on">timeline</a> and the <a href="http://jc.ngo.org.uk/trac-bin/trac.cgi/roadmap">roadmap</a>.</p></li><li> <p>I turned on anonymous SVN access, <a href="http://jc.ngo.org.uk/trac-bin/trac.cgi/wiki/ObtainingTheSoftware">described here</a>. As a handy side effect, source code to development versions of my CPAN modules is available too.</p></li></ul><p>All I need to do now is work on closing the tickets...</p> nik 2004-12-10T18:19:31+00:00 journal Today, I am mostly hating... http://use.perl.org/~nik/journal/22251?from=rss <p>... SPF.</p><p>More specifically, I am hating load-balancers that function by acting as DNS servers, handing out different A records for the services they balance depending on host load.</p><p>Self-same load-balancers can't publish TXT records...</p> nik 2004-12-10T12:04:03+00:00 journal Bad thing, good thing http://use.perl.org/~nik/journal/22168?from=rss <p> <b>Bad thing:</b> My coffee machine's just blown up. There's ground coffee all over the kitchen.</p><p> <b>Good thing:</b> Krups' customer service. "Yes sir, just fax us a copy of the receipt and we'll ship a replacement to you free of charge."</p><p>I was expecting to have to box the old one up and send it back to them -- as Murphy dictates, I'd dumped the packaging it originally came in 2 weeks ago, something I do only after a bulky purchase has been in the house and working flawlessly for at least three months...</p> nik 2004-12-06T16:56:02+00:00 journal Test::Harness's TAP in C -- libtap http://use.perl.org/~nik/journal/22137?from=rss <p>A few weeks back I did the work to make most of the FreeBSD regression test suite produce output that's compatible with Test::Harness and prove(1).</p><p>Back then there wasn't a name for that format. Now, of course, <a href="http://use.perl.org/~petdance/journal/22057">there is</a>.</p><p>This was very much a rough cut. A lot of the tests were written in C, consisting of a few hundred (or in some cases a few thousand) ASSERT() calls, with a <tt>printf("PASS\n");</tt> at the bottom -- if your test didn't dump core you knew it worked.</p><p>Changing most of those to add <tt>printf("1..1\n");</tt> and <tt>printf("ok 1\n");</tt> was the work of an afternoon.</p><p>On the TODO list was to implement as much of TAP as possible in C -- having plans, ok(), and so on. By doing that, and removing the ASSERT()s, a failing test doesn't prevent all the other tests from running, and you get a truer picture about how many tests overall are being run. And we all know that seeing the test count go up is a strong incentive to keep writing them.</p><p>So, here's a very early draft of that; <a href="http://jc.ngo.org.uk/trac-bin/trac.cgi/browser/nik/libtap/trunk/src/">libtap</a>. Right now, plan(), ok(), and diag() functionality is there, which is almost all the way to what I need for FreeBSD regression suite.</p> nik 2004-12-04T16:50:19+00:00 journal