Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

nik (3476)

nik
  nikc@cpan.org
AOL IM: nikclayton (Add Buddy, Send Message)

Journal of nik (3476)

Sunday October 15, 2006
04:50 AM

SVN::Web, remote repos redux, and syntax highlighting

On 8th September I wrote about adding support to SVN::Web for browsing repositories hosted remotely.

As I said at the time,

Also, it only seems to work for file:/// and svn:// URIs at the moment.

j2kaddict suggested that I look at using SVN::Client rather than SVN::Ra, so I've been doing that off and in my spare time.

I've finally tracked down the last of the memory allocation errors (SVN::*'s integration with Subversion's memory allocation routines requires more thought than is typically required with Perl) and now have something that's nearer release, on the svn-client branch.

This has slowed down some operations, but sped up others. For example, carrying out diffs (and viewing everything that's changed in a particular revision) is now much faster, as the diffing is carried out by the Subversion libs, instead of using Text::Diff.

The tradeoff is that some features have currently fallen by the wayside. For example, the diff output is currently raw, and not nicely highlighted.

Which brings me on to my next investigation -- I need to find a decent mechanism for doing syntax highlighting with Perl.

I've looked at Syntax::Highlight::Engine::Kate, but it looks as though if I want to change the look of the highlighting I need to subclass a bunch of modules, which is effort I'd like to avoid if possible.

Syntax::Highlight::Universal looks interesting, but doesn't build on my local FreeBSD box. That doesn't bode well for it being easy to install for people who are going to be using my code that depends on it. It also looks like it introduces a dependency on Java, which is a bit heavier than I really want.

Tuesday October 10, 2006
05:21 AM

SVN::Web, easier third party action integration

I knew there was something else I wanted to mention in my previous entry. I've been doing a little more hacking on SVN::Web recently.

One of the things that's bugged me about SVN::Web is the hoops a user needs to go through if they want to write their own SVN::Web::action class.

As well as writing the class (which is relatively easy), they need to provide templates for the interface, and localisation files for any language strings that they use.

This has been unduly difficult in the past. Although SVN::Web has supported looking for templates in multiple directories for some time now, localisation files have always had to live in one location.

Worse, you couldn't 'override' a localisation string that ships with SVN::Web with one of your own, without changing the files that ship with SVN::Web.

This makes upgrading more complicated than it needs to be, and means that the instructions for installing any third party actions would be more complicated than is really necessary.

So, over the weekend I finally did the work to make SVN::Web support localisation files in multiple directories. This means that you can use the localisation files that ship with SVN::Web, and selectively override one or more of the localisation strings by creating your own language.po file in your own directory -- future SVN::Web upgrades won't overwrite it. And installing third party actions is now simpler, since they can ship with their own localisation files, and now it's just a config.yaml tweak to have them used.

To do this I had to change the i18n infrastructure somewhat. Previously (I inherited this from clkao) SVN::Web used Locale::Maketext::Simple. As the name implies, this does make localisation quite simple, but at the cost of some flexibility. One of the things it couldn't do was support localisations in multiple directories.

So I had to roll my own SVN::Web::I18N, using Locale::Maketext and Locale::Maketext::Lexicon. The code itself is pretty simple, but I had a few headscratching moments trying to work out how to tie it together, as I found the Locale::Maketext* documentation a little confusing.

These changes were in r1178 (http://jc.ngo.org.uk/svnweb/jc/revision?rev=1178), and you can see SVN::Web::I18N too (http://jc.ngo.org.uk/svnweb/jc/view/nik/CPAN/SVN-Web/trunk/lib/SVN/Web/I18N.pm) .

Now if I can just get SVN::Client support working I can get on and release SVN::Web 0.50. And with that out of the door I can release a couple of other modules I've had hanging around -- SVN::Web::Timeline (http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web/branches/timeline/), which generates a nifty AJAX timeline of commits to the source code, and SVN::Web::Search (http://jc.ngo.org.uk/svnweb/jc/browse/nik/CPAN/SVN-Web-Search/trunk/) which indexes Subversion repositories and lets you search over them.

05:06 AM

MiltonKeynes.pm tech meeting tonight, SOAP/XML verbosity

There's a MiltonKeynes.pm (http://miltonkeynes.pm.org/) technical meeting this evening, organised by Tom Hukins.

Now I just need to finish my slides...

Which are nothing to do with what I've been working on recently. $dayjob has me wrapping a third party's SOAP API. We need to do this because although they have a very pretty web site for carrying out various back-end operations, it doesn't meet our audit requirements. It would also mean retraining our ops people to stop using our systems and start using theirs.

This is my first foray in to SOAP, and it's going reasonably well. But what hadn't occured to me is how much of the verboseness of XML would have to spill over in to my code.

I'm trying to make sure that the wrapper I'm writing is going to be easy to use for someone who's experienced with Perl, as well as looking close to the underlying API for someone who's familiar with that -- I'm not entirely sure why, as I'm not sure this code is ever going to see the light of day outside of $dayjob, but I get a definite sense of satisfaction out of doing these things properly.

Anyway, I digress.

So the responses back from the API all come back as XML, naturally, and SOAP::Lite does a reasonably good job of turning these in to useful Perl datastructures. I then convert these (hashes, normally) in to objects, and return the objects.

This forces the caller to use get_ and set_ methods to retrieve the data, which is less error prone than having them specify the hash keys directly.

And this is where the XML verbosity spills over in to the client interface. Handed XML like this:

<OptimisticLockValue>...</OptimisticLockValue>
<CustomerId>...</CustomerId>
<U seDomainDefaultSettings>...</UseDomainDefaultSettings>
...

the client API looks like this:

$ss->get_optimistic_lock_value();
$ss->get_customer_id();
$ss->get_use_domain_ default_settings();

(not sure why u.p is inserting spaces in to some of those).

Still, I suppose it makes the code more readable in the long run.

Saturday September 09, 2006
03:11 AM

Perl and different compilers / optimisation options

I've done a little bit of work recently to see how differently Perl performs on different operating systems compiled with different compilers and optimisation settings.

Running FreeBSD and Solaris on a Sun Ultra 40 (2 x AMD Opteron 2.8GHz CPUs, 4GB RAM), with Perl compiled with gcc and Sun's compiler (with and without optimisation), using SpamAssassin 3.1.5 as the test workload showed that Perl on FreeBSD compiled with 'gcc -O2' is distinctly faster than Perl on Solaris compiled with either gcc or Sun's compiler.

More details at:

http://jc.ngo.org.uk/blog/2006/09/05/day-58-of-60-developer-benchmarks-pt-3/

and

http://jc.ngo.org.uk/blog/2006/09/06/day-59-of-60-developer-benchmarks-pt-4/

Friday September 08, 2006
11:05 AM

SVN::Web and remote repositories

I've had a couple of journeys in to London over the last few weeks, and I've been able to use the time on the train to include support for remote Subversion repositories in to SVN::Web.

In use it's identical to 0.49, except that you can now specify the repositories to browse using Subversion URLs.

So if you have this config fragment in your config.yaml:

      repos:
          my-repo: /path/to/my/repo

that still works. But you can now do:

      repos:
          my-repo: file:///path/to/my/repo

That's no great change. But you can also do this:

      repos:
          niks-repo: svn://jc.ngo.org.uk/
          kde-repo: svn://anonsvn.kde.org/home/kde/

to link to remote repositories and browse them. So if your favourite open source project is using Subversion, but is using SomeOther(tm) web repository browser frontend you can fix that, without insisting that they install SVN::Web :-)

Obviously, this is going to be somewhat slower, so I recommend using the caching options in SVN::Web to speed things up. Also, it only seems to work for file:/// and svn:// URIs at the moment.

See this message http://jc.ngo.org.uk/pipermail/svnweb/2006-September/000018.html for more details. The work's currently on a branch, retrievable with

svn checkout svn://jc.ngo.org.uk/nik/CPAN/SVN-Web/branches/svn_ra/

Feedback welcomed.

Friday July 28, 2006
11:07 AM

ministat redux

ministat is now much more capable. It's grown decent documentation, a number of useful options, and can now generate plots in color.

Here are a couple of examples:

And here's what the plain text output looks like:

x d1
+ d2
* d3
: = Mean
M = Median
+----------------------------------------------------------------------- --+
|                           *    *                                         |
|                           *    *                                         |
|x    x   x    x   x    *   *    *                 +    +   +    +   +     |
|x    x   x    x   x    *   *    *    *            +    +   +    +   +    +|
|   |_____M:_______|      |_M_:___|                   |_____M_:______|     |
+--------------------------------------------------------------------------+
    N           Min           Max        Median          Mean        Stddev
x  11             1             6             3     3.2727273     1.6787441
+  11            12            17            14     14.272727     1.6787441
Difference at 95.0% confidence
        11 +/- 1.4932
        336.111% +/- 45.6255%
        (Student's t, pooled s = 1.67874)
*  11             6             9             7     7.3636364    0.92441628
Difference at 95.0% confidence
        4.09091 +/- 1.20535
        125% +/- 36.8301%
        (Student's t, pooled s = 1.35512)

Get the ministat source code and play around.

Wednesday July 26, 2006
04:06 PM

ministat

[ Originally posted here. ]

I’ve spent some of today porting some useful statistics reporting software from C to Perl.

ministat reads in two or more files of data and uses the Student’s t test to determine if there is any statistical difference between the means of the datasets. This is especially useful when comparing benchmark results.

For instance, I figure that this will be useful to compare data from several Sendmail runs, where the number of queue directories differ between the runs. It should highlight any benefits between different numbers of queue directories.

Rather than explain more here, I’ll point you at the code. There’s documentation towards the end of the file. Note that this still needs some work — there’s no proper command line option handling at the moment, some of the documentation needs fleshing out, and I wouldn’t use this as an example of good Perl code, as it still looks far too much like a Perl program that’s been written in C.

When I’ve fixed that I’ll put it up on CPAN.

Tuesday July 25, 2006
04:29 PM

Problems with M4, backticks, and changequote.

Anyone know much about M4? I've having a problem with backticks and changequote, and can't figure out why it's not working. If you might have an idea, I'd appreciate it if you'd take the time to take a look at this post and let me know what I might be doing wrong.
Friday July 07, 2006
03:57 AM

AJAX timelines and SVN::Web

A day or so ago I started seeing references to AJAX timelines, by Simile in a few blogs.

Thinking that they looked rather interesting, I've taken a couple of hours to integrate them in to SVN::Web.

The work's been carried out on this branch, if you want to follow along.

The code works well enough to draw a scrollable timeline of Subversion commits.

Tuesday July 04, 2006
11:02 AM

SVN::Web / dtrace

It's been a busy few months.

Most of my Perl work has been concentrated on SVN::Web. Since I last wrote in here there have been 6 other releases.

As well as the usual round of bug fixes, new features include:

* Better RSS support.

* Localisation selection from the web interface.

* Caching

* A few thousand additional tests, to check that mod_perl and apache/cgi support work.

* mod_perl2 support

* A standalone webserver that can be used for testing, or environments where full blown Apache isn't necessary.

The other thing I plan on doing is taking a deeper look at dtrace on Solaris (and possibly FreeBSD). This with an eye to building on some earlier that people have done instrumenting Perl with dtrace. More details on this over at http://jc.ngo.org.uk/blog/.