Shlomi Fish's Journal

Meta: Script to Filter the Master use.perl.org Blogs' Feed

Shlomi Fish — 2009-05-13T14:30:04+00:00

As expected from the latest trend in the Perl blogosphere this post will be about Roles. And Moose! And Roles in Moose! And Moose in Roles! And Roles outside Moose…

Seriously now, this is a post about a completely non-Moosey and non-Roley script I wrote to filter the use.perl.org master journals' feed. What this script does is fetch the Atom feed of all the use.perl.org journals' posts, and filters out the entries of the use.perl.org authors that the invoker specified.

Here is out to use it. First of all: svn checkout it (or otherwise fetch it using HTTP). Then you can simply use:

perl filter-use-perl-journals.pl -o everything.atom

Then you can serve everything.atom with a web-server to read it using your web feeds' aggregator.

To simply create a non-filtered copy of the feed. Now let's say you want to get rid of posts from my journal (because it sucks). In that case, say:

perl filter-use-perl-journals.pl -o non-shlomif.atom \ --blacklist="Shlomi Fish"

Now, let's say you also want to get rid of Ovid's posts. I have no idea why you'd want to do that, because his journal is great, but it's just for the sake of the example. In that case, do:

perl filter-use-perl-journals.pl -o non-shlomif-and-ovid.atom \ --blacklist="Shlomi Fish" --blacklist="Ovid"

Finally, there's the --rand flag which is useful in case you're running the script with cron. What it does is wait for a period of time of random length, before fetching the feed, so the HTTP server would not be overloaded at regular periods. This requires a working /dev/urandom for the time being.

The script is made available under the MIT/X11 Licence so its use is completely unencumbered. I wrote this script today, because I have a personal use for it, but other people may find it useful too. It requires a recent version of Perl (5.8.x should be enough I think) and XML-Feed.

Gabor Szabo on High-Level Programming with Perl 6 on 22-Marc

Shlomi Fish — 2009-03-17T08:51:45+00:00

The Tel Aviv Open Source Club will host the first part of a series of talks by Gábor Szabó about "High-Level Programming Concepts Using Perl 6" - on 22-March-2009.

The meeting will take place at Tel Aviv University (in Tel Aviv, Israel), at the Schreiber MathsCS building, room 008 on 18:30. More information can be found on the meeting's page on the wiki.

With any other problems, feel free to contact me.

Abstract

High-level programming concepts using Perl 6

A series of presentations on learning and using Perl 6 from the ground up to the special features.

Many would think that Perl 6 is just a new version of Perl and that it might only be interesting for Perl programmers. However, Perl 6 is in fact a compiled language running on a virtual machine that embraces many new concepts not found in most programming languages.

The presentations will be equally interesting for Perl, Java and C# programmers.

During the series of talks we will start by learning the basics of the language and will get to various high level concepts. For now we plan 2 sessions but if we need more time we'll schedule more meetings.

Note

After the talk we will go to the café at the main entrance of the university where we can continue the discussion. If people bring portable computers, we can get the off the ground on the spot. VirtualBox images will be provided with everything that is needed for playing with Perl 6 set up inside. So you may opt to bring a computer with VirtualBox installed.

We are always looking for presentations on interesting topics. If you have an interesting idea for a talk, feel free to contact us and we'll co-ordinate a date.

"The Perl Future" on Heise

Shlomi Fish — 2009-01-14T09:58:59+00:00

Piers Cawley writes about the new happenings in the Perl world in "Healthcheck: Perl - The Perl Future" on Heise Open Source. Even though I was already aware of most of what he covers there, I found it a good read, and it is doubly recommended for people who are not following Perl news as thoroughly as I do.

Also see the LWN.net coverage (where I've learned about it).

XML::RSS-1.40 - Now with a Lot of Array Goodness

Shlomi Fish — 2008-12-01T17:18:58+00:00

After going over some of the recent and not-so-recent XML::RSS bugs, I noticed a common pattern of people wanting support for replicable XML elements. What they wanted was that when two elements with identical names were specified (in many RSS use cases where doing this would make sense), then the value pointed by the keyname would become a reference to an array instead of just concatenated together in an unhelpful manner.

In the past days, I decided to work on this meta-problem in one concentrated swoop. I naturally had to implement it in several places for both output and parsing, though tended to do them in separate commits.

XML-RSS version 1.40 contains the result of this effort. I should note that some use-cases for multiple tags probably won't do the right thing, and the internal code has garnered some ugliness. I ended up extracting some methods, but I can still be happier from the quality of code. I suppose I can always refactor it later.

All of this work reduced the number of active bugs in XML-RSS to 3, which I intend to deal with shortly, if all goes well.

In other news, I released a new version of WWW::Search::MSN, which has been adapted to the new HTML generated by the MSN search. The hardest part in the process, was figuring out how to instruct the libwww-perl that the WWW::Search front-end used to send a "Language:" header to cancel the location-based localisation.

That was my boring "recent hacktivity" report. You may now go on with whatever you did earlier.

Why Dist-Zilla is Probably Not For Me

Shlomi Fish — 2008-11-16T08:44:37+00:00

rjbs' introduction to Dist-Zilla piqued my interest, and I used CPANPLUS-Dist-Mdv to prepare .rpm's for it and its depenedencies and install them. However, I wondered about a potential problem with it, before I even tried it, and speaking with rjbs on IRC confirmed that it exists.

Dist-Zilla generates the resulting .pm, scripts, etc. from templates, and as a result the lines that are reported by errors and warnings are not the same as the ones you've edited. This makes tracing lines back to their source much more difficult. Since most of my times is spent debugging and handling errors (whether I encounter them or I find them on CPAN testers or in bug reports), and I want to edit the source directly, I think that Dist-Zilla is not for me. What instead I'd like is a way to change common, repeated text (like version numbers, licensing, URLs, etc.) commonly across all the files in a certain distribution, or in many distributions. That and also that module-starter (or whatever) provide scaffolding for creating a new .pm file.

I appreciate rjbs's efforts, but I'm going to stick with module-starter (and contribute to it).

File-Find-Object Refactoring

Shlomi Fish — 2008-10-27T18:34:38+00:00

I haven't updated this journal in a long time, and mostly been writing in my LiveJournal Technical blog. I've emailed to the Perlsphere maintainer about including that blog there (once from my home address and once from gmail.com), but received no reply, and so far it is not included there.

In any case, this post is about File-Find-Object, which is an object-oriented alternative to the core File::Find module. Originally by Nanardon, I've maintained it since version 0.0.3.

Recently, I've been meaning to write a project, and contemplated whether to use File-Find-Object or roll-out my own directory traversal logic, but then decided that improving F-F-O so it will do what I want was a better idea, in an "Eating one's own dog food" and not-re-inventing the wheel fashion. So I needed to extend F-F-O to do what I want.

While I took a closer look at the code to inspect it, I found it had some "bad smells", and decided to fix it by refactoring, as a necessary pre-requisite for extending it.

The first thing I did was notice that many methods accepted a $current parameter that was passed from one method to another, and then used. As it turned out, most of these simply originate from $self->_current(), which I now used to retrieve the value in each method, without passing a parameter.

Another fact I noticed was that there were many if ($self eq $current) checks in the code. Since $current is dynamic, I decided to create a predicate _is_top() which will encapsulate it and to create the following method maker:

sub _top_it { my ($pkg, $methods) = @_; no strict 'refs'; foreach my $method (@$methods) { *{$pkg."::".$method} = do { my $m = $method; my $top = "_top_$m"; my $non = "_non_top_$m"; sub { my $self = shift; return $self->_is_top() ? $self->$top(@_) : $self->$non(@_) ; }; }; } return; }

Thus, when _is_top evaluates to true I call _top_mymethod and otherwise _non_top_mymethod. This is a variation on the "replace conditional with polymorphism" refactoring.

Now ->_current() returned the ->_current_idx()'th item from an internal stack representing the directories which the object is traversing. I wanted to see where "_current_idx" was set and discovered it was incremented when an item was pushed to the stack, and decremented when an item was popped. As a result, I eliminated "_current_idx" completely and replaced _current() with $self->_dir_stack()->[-1]. That removed a lot of cruft from the code.

I also was able to do what I wanted, and make sure the paths are maintained as the base path for the traversal followed by a list of extra components of each inner directory.

I noticed that I flat-copied the return of a method returning an array reference several times (E.g: [ @{$self->_components()} ]) and so created another method maker - this time for "_copy" methods.

And naturally, I extracted many methods.

All this enabled me to create ->next_obj() and ->item_obj() API methods that return objects instead of plain strings. Naturally, this is not refactoring, but extending.

While I was in the neighbourhood, I discovered that the code had a format-string-issue, which I fixed, and released File-Find-Object-0.1.1 immediately.

After the release, I received three failure reports from CPAN Testers. Two of them were for a missing dependency, that wasn't installed due to a bug in the tests' smoking setup. One of them, however, was for Win32, where it was a real bug. I was able to reproduce it using Strawberry Perl on my WinXP computer, and released File-Find-Object-0.1.2 that corrected it. The problem was that on non-cygwin-Win32 all inodes are returned as 0 in the calls to stat().

Today, I started writing the project that required all this work on File-Find-Object. So far it doesn't do much, but it's a start.

OSDClub Tel Aviv Meeting: Ori Idan about the Semantic Web

Shlomi Fish — 2008-09-18T14:24:08+00:00

The Tel Aviv Open Source Developers Club (OSDClub) (formerly the Tel Aviv Linux Club and other clubs such as Perl-Israel) will hold a meeting on 21/September (next Sunday). Ori Idan will deliver a presentation about the Semantic Web.

Ori is the director of the Israeli branch of the World-Wide-Web Consortium (W3C), and is a very good presenter, so it is recommended to attend.

The meeting will take place at 18:30 in the Schreiber Maths and Computer Science building of Tel Aviv University, room 008. Attendance is free of charge and everyone are welcome.

Resolution for maintperl-5.8.x's IPC::SysV failure

Shlomi Fish — 2008-09-04T12:11:52+00:00

As a followup to this post about "make test in maintperl-5.8.x Fails on Linux", I should note that we have investigated it on perl5-porters in the past days. After a little investigation, I realised that there was a stray "SysV.pm" file in my perl-5.8.x tree, which caused all the problems.

As it turned out, it was not removed because I used "rsync -auvz" to synchronise the tree (as instructed in the perl5-porters FAQ) instead of "rsync -auvz --delete-after", which removes the no-longer present files. After running rsync with --delete-after, and building again - "make test" was successful.

"Another crisis was solved!".

Recent Hacktivity Summary

Shlomi Fish — 2008-08-21T20:12:53+00:00

Well, I added more tests to Module-Starter, and while I was writing the tests, I discovered some bugs in it. So now there's a patch with the tests and the bug fixes and another one with only the bug fixes. Both of them are unapplied.

Having been trying to learn Ruby recently, I decided to write a Conway's Game of Life implementation for it. It took me most of one evening to get it working and then part of the other day to make it more idiomatic with the help of the people on Freenode. It felt good in a way.

Then I decided to finally see why my Test-Run failed many tests recently. As it turns out, this was due to broken TAP-Parser compatibility: the skip message in skip_all field now accepts only "SKIP " and not something like "skipping: ". Fixing the Test-Run tests fixed it and I released new versions.

I also noticed that many of the Test-Run-CmdLine plugin had the following pattern:

sub _init { my $self = shift; $self->NEXT::_init(@_); $self->add_to_backend_plugins("AlternateInterpreters"); }

So I overloaded the construction function, only to add a plugin. I decided to fix this by doing an accumulation of such plugin specifiers in every plugin, and so got only:

sub private_backend_plugins { my $self = shift; return [qw(AlternateInterpreters)]; }

Which is much cleaner and less intrusive.

Next on my plate was Freecell Solver. I decided to work on make_pysol_freecell_board.py which is a Python script that generates the initial boards of PySol. The script was crudely written to begin with and it became even cruder as time went by. I decided to give it a serious face-lift. So what I did was:

Convert the columns (of the card games) from strings to arrays. Each card was stored there individually.
Convert the cards to a class, instead of integers and strings. Create similar classes for managing columsn and boards.
Extract many functions and methods.
Create a class to manipulate the various game types, and forward the different logic based on it.
Pythonised the script by employing some Python paradigms.

You can find what I have so far in the repository. It's much better than what I started with two days ago. Writing such Python code now seems more fun than I recall it, and I actually enjoyed it.

In regards to SMOP, they convinced me to install Ruby-1.9.x under a prefix, which I did, but then it yelled at me for not having a Haskell cabal thingy. Turns out that the Pugs Makefile.PL installs it somewhere under the home-directory, which I didn't want to happen, because I want to keep it tidy. Again, this reminded me of Drivl.com recipe on how to make square corners, and I decided to give up on SMOP again.

And I should note that I was able resolve a long-standing problem I had with XML-LibXML/XML-LibXSLT on my Mandriva system, and now I simplified the XML-Grammar-Fortune build-system.

I also spent some time writing a backup system for some of the Israeli MediaWikis that I manage. This involved a bunch of Perl scripts.

So - Perl, Ruby and Python - all in a few days work. Cheers everybody.

Random Hacktivity Summary

Shlomi Fish — 2008-08-12T20:28:29+00:00

This is another boring hacktivity summary, but this time mostly Perl-related - so I'm posting it here.

First order of business is Games-Solitaire-Verify. In its first versions I implemented textual return values for the verify_and_perform_move() method. However, this makes it harder and more error-prone to programmatically understand why exactly the verifier does not like your move . True to my ideals that errors that need to be caught and processed must be objects, I converted everything to return Exception::Class-subclasses. E::C really made my job easy and now my main gripe with everything is that my error class names have become unweildy. Some examples are:

Games::Solitaire::Verify::Exception::Move::Src::Col::NotEnoughCards
Games::Solitaire::Verify::Exception::Move::Src::Col::NonSequence
Games::Solitaire::Verify::Exception::Move::Dest::Col::OnlyKingsCanFillEmpty

Oh well. In any case, now that I've converted everthing to classes, I forgot to add stringified versions of the errors (which Exception::Class supports), so if something breaks and is reported to the shell, it may not be very human-friendly. I guess it means a bit more work there.

Afterwards, I decided to work a bit on Test-Run, and after I read its todo list, remembered that I still intended to create some specialised Module-Starter plugins for it. Then I remembered that I asked about M-S' test suite, and found out it didn't have a comperensive one. So, I decided to remedy this and wrote a patch to add a test script. For the test script I parsed the generated files step-by-step using a class and some test assertion methods.

Since I had no repository access, the first versions of the patch were written by repeatedly running svn diff into files, but I decided I want a more robust version control for the local host. So I looked into svk but it had acquired a bug on my system.. Then I decided to try bzr-svn, but it had a non-starter bug. Now since I dislike git quite a bit, I searched for something Mercurial-based and found hgsvn. This worked surprisingly well for my purposes and I was able to version the local copy. Mercurial seems pretty nifty so far.

The Module-Starter patch was not integrated into the repository yet. Module-Starter has been suffering from quite a lot of neglect, and rjbs (who performed its latest upload to the CPAN) told us on IRC he now uses a different solution that was written by someone else and has not been made public yet. I'd really like to see something better emerge, but I hope I won't have to fork Module::Starter under a different name.

I decided to stop patching it for the time being, but am getting anxious. It makes me frustrated to have an uncommitted patch and no repository access with which I can apply it.

As I was working on Module-Starter I talked with the SMOP guys, and decided to try give it a spin. After checking out the source and doing the make -f Makefile.cvs, ./configure, make dance, I ran into a few missing ".pm" files error. The first one was caused by the lack of the rest of the pugs tree, and the later ones were easily installable from CPAN. But then I ran into a strange error:

../../../misc/elfish/elfX/../../STD_red/STD_red_run:93: syntax error, unexpected '>' whiteout = ->(s){s.gsub(/[^ \n]/,' ')};

As it turned out, I needed Ruby-1.9.x (the development version that naturally is not avaialable in Mandriva or other Linux distributions), and decided that there's no way I'm installing it. This reminded me of Drivl.com recipe on how to make square corners, and I decided to give up on SMOP.

After all this, I also added a lot of text to the CMS/Web-devel-framework/Web-design Wikia, which I founded and had neglected previously, and to the pages in the Star Trek "Expanded Wikia" which I maintain, which were a nice change in scenery from all the code I wrote.

Today I didn't do too much productive work, after having my usual post-productivty disorientation, and instead spent the first half of the day chatting on IRC (to a very interesting American linguist-wannabe), but was also able to bike, for the first time since Saturday. Oh, and I saw Wall-E on Sunday, which I liked. Been a while since I've been to the cinema.

make test in maintperl-5.8.x Fails on Linux

Shlomi Fish — 2008-08-09T11:25:44+00:00

Due to the fact that my email to perl5-porters bounced for some reason, and because the email I filed using perlbug was not registered in the bugtracker for some reason, I'm posting it here:

It seems that on Mandriva Linux Cooker (the Mandriva bleeding edge distribution) on maint-perl IPC::SysV fails "make test": {{{{{{{{{{{{ Failed 4 tests out of 1095, 99.63% okay. ../ext/IPC/SysV/t/ipcsysv.t ../ext/IPC/SysV/t/msg.t ../ext/IPC/SysV/t/sem.t ../ext/IPC/SysV/t/shm.t ### Since not all tests were successful, you may want to run some of ### them individually and examine any diagnostic messages they produce. ### See the INSTALL document's section on "make test". ### You have a good chance to get more information by running ### ./perl harness ### in the 't' directory since most (>=80%) of the tests succeeded. ### You may have to set your dynamic library search path, ### LD_LIBRARY_PATH, to point to the build directory: ### setenv LD_LIBRARY_PATH `pwd`:$LD_LIBRARY_PATH; cd t; ./perl harness ### LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd t; ./perl harness ### export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH; cd t; ./perl harness ### for csh-style shells, like tcsh; or for traditional/modern ### Bourne-style shells, like bash, ksh, and zsh, respectively. u=5.17 s=1.62 cu=359.73 cs=35.41 scripts=1095 tests=132095 make[2]: *** [_test_tty] Error 1 make[2]: Leaving directory `/home/shlomi/Download/unpack/perl/perl5/maint-perl/perl-5.8.x-121819 0282' make[1]: *** [_test] Error 2 make[1]: Leaving directory `/home/shlomi/Download/unpack/perl/perl5/maint-perl/perl-5.8.x-121819 0282' make: *** [test] Error 2 }}}}}}}}}}}} I noticed it was the only upgrade since I updated perl-5.8.x-latest. I'll try to investigate further. Regards, Shlomi Fish [Please do not change anything below this line] ----------------------------------------------------------------- --- F lags: category=library severity=medium --- Site configuration information for perl v5.8.8: Configured by shlomi at Fri Aug 8 13:14:33 IDT 2008. Summary of my perl5 (revision 5 version 8 subversion 8 patch 34096) configuration: Platform: osname=linux, osvers=2.6.26-desktop-2mnb, archname=i686-linux uname='linux telaviv1.shlomifish.org 2.6.26-desktop-2mnb #1 smp wed jul 23 11:32:46 brt 2008 i686 intel(r) pentium(r) 4 cpu 2.40ghz gnulinux ' config_args='-de -Dprefix=/home/shlomi/apps/perl/perl-5.8.x-latest -Doptimize=-g' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='-g', cppflags='-DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='4.3.1 20080626 (prerelease)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/lib64 libs=-lnsl -lndbm -lgdbm -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.8.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.8' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib' Locally applied patches: MAINT33934 --- @INC for perl v5.8.8: /home/shlomi/apps/perl/modules/lib/perl5/site_perl/5.10.0 /home/shlomi/apps/perl/modules/lib/perl5/site_perl/5.8.8 /home/shlomi/apps/perl/modules/lib/site_perl/5.10.0 /home/shlomi/apps/perl/modules/lib/site_perl/5.8.8/i686-linux /home/shlomi/apps/perl/modules/lib/site_perl/5.8.8 /home/shlomi/apps/perl/modules/lib/perl5/5.10.0 /home/shlomi/apps/perl/modules/lib/perl5/5.8.8 /home/shlomi/apps/perl/perl-5.8.x-latest/lib/5.8.8/i686-linux /home/shlomi/apps/perl/perl-5.8.x-latest/lib/5.8.8 /home/shlomi/apps/perl/perl-5.8.x-latest/lib/site_perl/5.8.8/i686-linux /home/shlomi/apps/perl/perl-5.8.x-latest/lib/site_perl/5.8.8 . --- Environment for perl v5.8.8: HOME=/home/shlomi LANG=en_GB.UTF-8 LANGUAGE=en_US LC_ADDRESS=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 LC_IDENTIFICATION=en_GB.UTF-8 LC_MEASUREMENT=en_GB.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_NAME=en_GB.UTF-8 LC_NUMERIC=en_GB.UTF-8 LC_PAPER=en_US.UTF-8 LC_SOURCED=1 LC_TELEPHONE=en_US.UTF-8 LC_TIME=en_GB.UTF-8 LD_LIBRARY_PATH=/home/shlomi/Download/unpack/gui/X/nouveau/mesa/mesa/lib LOGDIR (unset) PATH=/usr/java/jdk1.5.0_09/bin:/home/shlomi/Download/unpack/graphics/fop/fop-0.9 3:/home/shlomi/apps/perl/modules/local/bin:/home/shlomi/apps/latemp/bin:/home/sh lomi/apps/file/gringotts/bin:/home/shlomi/apps/gimageview/bin:/home/shlomi/apps/ test/quadpres/bin:/home/shlomi/apps/docbook-builder/local/bin:/home/shlomi/bin:/ usr/local/bin:/bin:/usr/bin:/usr/games:/usr/lib/qt4/bin:/usr/bin:/opt/kde3/bin:/ usr/lib/ssh:/usr/lib/qt4/bin:/usr/bin:/opt/kde3/bin PERL5LIB=/home/shlomi/apps/perl/modules/lib/perl5/site_perl/5.10.0:/home/shlomi/ apps/perl/modules/lib/perl5/site_perl/5.8.8:/home/shlomi/apps/perl/modules/lib/s ite_perl/5.10.0:/home/shlomi/apps/perl/modules/lib/site_perl/5.8.8:/home/shlomi/ apps/perl/modules/lib/perl5/5.10.0:/home/shlomi/apps/perl/modules/lib/perl5/5.8. 8 PERL_BADLANG (unset) SHELL=/bin/bash

Can anyone reproduce it as well? Just do: rsync -auvz rsync://ftp.linux.activestate.com/perl-5.8.x/ perl-5.8.x-latest and then build perl and run "make test"?

I could not reproduce this problem by running "perl Makefile.PL" ; "make" ; "make test" in the latest IPC::SysV. For all the failed tests, the problem seems to be:

shlomi:~/Download/unpack/perl/perl5/maint-perl/perl-5.8.x-1218190282/t$ ./perl -MTestInit ../ext/IPC/SysV/t/msg.t IPC::SysV object version 1.99_07 does not match bootstrap parameter 1.05 at ../lib/DynaLoader.pm line 250. Compilation failed in require at ../ext/IPC/SysV/t/msg.t line 37. BEGIN failed--compilation aborted at ../ext/IPC/SysV/t/msg.t line 37. # Looks like your test died before it could output anything.

Web Feed Mangling Blues

Shlomi Fish — 2008-08-05T21:34:59+00:00

Back when I was looking for a way to aggregate several RSS feeds, a few years back, I found XML-RSS-Aggregate by Audrey Tang. Using it turned out to be problematic and I had to patch it, sub-class it, and write a lot of code above it to have it behave with my feeds.

Then after I asked Audrey about it on the IRC, she told me that XML-RSS-Aggregate was no longer recommended and that XML-Feed was then the way to go. So I happily switched to using XML-Feed and even packaged it for Mandriva, and all was all. I wrote a 5 star review for it on CPAN ratings, and wrote a 1 star review on XML-RSS-Aggregate directing people to XML-Feed instead.

My problems started again, when after setting up the feed for my XML-Grammar-Fortune-based fortunes, I discovered that the Atom feed did not validate and the RSS feed had many warnings. I ended up maintaining XML-Feed in my own repository, and tried to contact the author of XML-Feed about becoming a co-maintainer.

Now, yesterday, I joined #plagger on Freenode, trying to see how I can make progress there, and miyagawa told me about Data-Feed, which is a new module, which aims to be even better than XML-Feed. I thought to myself "Thanks God!" and decided to try to install it.

Not wanting to pollute my system, I decided to prepare Mandriva RPMs of it and its dependencies, which you can find in the link. I ran into a problem with its tests failing due to the fact what Data-Feed tried doing only worked with XML-RSS-LibXML and not with XML-RSS which was all I had installed. I reported a bug about it, and am hoping for the best.

Today, I decided to try to convert the XML-Grammar-Fortune-Synd code to use Data-Feed instead of XML-Feed and see if it would help me. But then I tried to look up some classes with which I was familiar from XML-RSS and realised that Data-Feed had practically non-existent documentation. So I could not look up that information even if I wanted to.

So I decided to stick with XML-Feed for the time being. Right now XML-Grammar-Fortune-Synd depends on XML-Feed with my downstream patches, and fails all tests, and the Atom feed still does not validate. So it's only mostly broken.

I got the web-feed manglin' Blues!

The Demise of O'Reilly-Net?

Shlomi Fish — 2008-07-26T07:45:18+00:00

I recall a time, not long ago, when the O'Reilly-Net sites: OnLAMP.com, Perl.com, LinuxDevCenter.com, etc. used to carry weekly or close-to-weekly in-depth articles about various IT-related subjects, that tended to be high-quality and provided a lot of good information. But lately, it seems there were only a few blog posts on OnLAMP.com about open-source in Windows and other such relatively low-interest or low-quality topics (and practically nothing on Perl.com or LinuxDevCenter) and no new articles. The latest article on OnLAMP.com is from 20-May this year (over two months ago) and there were huge gaps between the latest articles there.

Now, I recently sent a suggestion for an article, with an outline, to the Perl.com editor (chromatic) and to the OnLAMP.com editor (James Turner), and received no reply. I sent it again, and again received no reply. I sent the second message on 9-July. Now, even if they didn't like the outline, then the right thing to do would be to promptly reply saying that they're not interested in it and that I should seek other venues of publication. I'm pretty sure that at least chromatic is alive and has some spare time, because I've seen him active on use.perl.org and on @perl.org mailing lists.

I submitted some articles to O'ReillyNet in the past and encountered a few delays in response, which I suppose is expected, but such a long delay is no longer acceptable. My guess is that I'm not the only one who sent such suggestions to O'Reilly-Net and received no response.

If the O'Reilly-Net sites would like to keep their edge, they should make sure they don't turn into another one of the dime-a-dozen blog sites, and instead start offering high-quality articles and essays again. But this will require a better responsiveness on the part of the editors. Would chromatic and/or Mr. Turner care to comment about that?

Overcoming Misbehaviours in Code I Did Not Write

Shlomi Fish — 2008-07-23T18:36:14+00:00

Yesterday, when I tried to validate the RSS feed of my fortune cookies, I ran into a Unicode encoding problem. Seems like a Unicode character ("→") was converted into its individual bytes and then encoded using SGML entities. After a long time of debugging it, I found out the problem was with XML-Atom and was easily fixed using:

$XML::Atom::ForceUnicode = 1;

This behaviour is documented in the XML::Atom::Feed documentation, which I didn't bother to read because I believed XML::Feed would do the right thing. The reason I ran into this problem there was because I generated an Atom-based XML::Feed (so I can also have an Atom feed and then converted it to RSS.

Today I encountered, a similar problem, this time with CPANPLUS. I wanted to write a script to syndicate the list of CPAN modules for a the Perl-Israel Israeli Perl Projects page. One thing I noticed was that calling $author->distributions() took a long time. After some debugging, I noticed that the culprit was in CPANPLUS::Module's dslip() method which went over all modules list in O(N) time looking for modules whose prefix is the current module. This was fixed using:

{ no warnings; # This is an optimisation because dslip is incredibly slow, # and it badly affects $module->clone() . *CPANPLUS::Module::dslip = sub { return ' ' x 5;}; }

This made my script run much faster. Not instantaneously, but definitely faster.

So this way, I was able to fix the two problems that I had. Now I'm happy and can go on with the rest of my life.

Reflections on Syndicating my Fortune Cookies as Atom

Shlomi Fish — 2008-07-15T13:26:31+00:00

Another technical entry, on my fortune cookies' FortuneXML work again. When I last left you, I was able to transform the XML sources of the fortune cookies to XHTML and to plaintext, and ran into a nasty stack smash that prevented me from uploading the module to the CPAN.

Then, I decided that now that each fortune had its own unique URL, and that people didn't have the time to regularly see what has changed, it was high time to get a web-feed syndication for it. I decided to use XML-Feed directly, and to generate an Atom feed. (So far I only did RSS using XML-RSS).

It seemed logical that I'd simply go over all the individual id="" attributes in the XML files I maintained, and see which ones were added. Then I'll generate a feed containing the most recent 20 or so. However, the XML does not contain the dates where they were added. So I decided that for simplicity I'll record the dates all the IDs were added into a YAML file, which in turn will be updated there. (A database of some sort may be better eventually.)

Now, how to find the most recent 20 dates from the YAML and the XMLs? One option would be to collect all the dates and sort them, but that seemed wasteful to me. Recalling my Computer Science Data Structures studies I remembered that I could use a priority-queue for that, and remembered there was a module for that on the CPAN. A search for "Priority Queue" or "PQ" yieleded nothing, but I then searched for Heap and found Heap.pm, which implements several Heaps that can be used as a priority queue.

Heap is a very impressive module. I remember that back when I studied at the Technion, it used to be the only usable sourceware (and open-source) implementation I found of Fibonacci heaps. I decided to use the Binary heap this time, though.

I had to write some wrapper code to get it working. I decided against writing automated tests before I wrote the code (= "Test-Driven Development") and just write the code and get it to working, because I found that I didn't exactly know what it would generate. Anyway, I now have the script working.

I encountered some problems in trying to find how to render just one fortune element and not the entire file. After not finding anything about it in the XML::LibXSLT documentation, I opted to tweak the XSLT stylesheet and create an optional parameter for it to act upon.

Eventually, I got an Atom feed. Firefox displays it fine, but it doesn't validate. Part of the problem is that I needed to trim down the feed entries' content from the "<html>" and other such tags. But another issue is that XML-Feed does not support all the required Atomisms.

I've noticed both XML-Feed and XML-atom have suffered from neglect (including the lack of support for those Atomisms), and decided to adopt them. I set up a repository for XML-Feed on BerliOS and started to get into it. First thing I did was get rid of ExtUtils::AutoInstall, and I have also given repository access to another person who is interested in revamping them. Thanks to BingOS for helping me with some Module::Install-ism.

As it turns out, the XML-LibXSLT bug was actually a Cooker-specific bug, which seems like it is caused from problematic compilation flags.

So world-domination through Unix-like fortune cookies is still making progress. I'll probably release XML-Grammar-Fortune in its current form soon since the smash stack I got is highly system-dependent.

Perl and Solitaire Games

Shlomi Fish — 2008-07-10T20:11:07+00:00

This entry will be a technical entry, but on a relatively lighter side: Solitaire games. Solitaire games (also known as "patience" games) are any of several kinds of single-player games played using one or more decks of cards. Despite common belief, the Windows-game called "Solitaire" is actually Klondike, which is only one variant of Solitaire. Freecell is another famous variant of Solitaire.

So anyway, I spent some of the past two weeks working on Games-Solitaire-Verify, which is a CPAN module to verify automated solutions of solitaire games. So far only Freecell is supported, but there's a lot of modularity in place for other variants. The intention of this module is to be used as part of the automated test suite of Freecell Solver, which is a solver for several variants of Solitaire, which lacks a comprehensive test suite.

I should note that I implemented the very first version of Freecell Solver in Perl, but it was horribly slow. Of course, I did some very illogical things speed-wise, like (1) putting all the states into an unsorted array and using linear search to scan it - and (2) constantly deserialising states into objects and data strcutures of individual cards, columns, and boards, and doing the comparison the hard way. So it was dog slow. I ended up re-implementing it as (saner) C code which turned out to be considerably faster than the Perl version. Moreover, my next C version was faster by a factor of about 100 (not because I was smarter, but I was less dumb), and since then I incorporated many other speed optimisations. (There are also several C and Assembly-hosted solvers that are much faster than mine.)

In any case, someone contacted me about the solver on Facebook, and after talking to him on Yahoo Messenger, we decided that he, I and another guy will talk on Freenode on the newly started #solitaire. There, I was quickly quickly joined by tybalt89, who is a master golfer, and who decided to write his own solver for Klondike in Perl. After a while he finished it and so I took a look at the source code.

From my impression it seems like the kind of code a golfer would write: strings instead of objects (with regular expressions and other text processing to manipulate them), a call to shuffle() from List::Util with a random seed, no use warnings, lots of obscure two-letter variable names, a lot of trailing "or" clauses, and it also has $hearts, $clubs, etc. instead of a hash. I talked about it with another master golfer and he said that "golfing rots the brain", and he said a program he gave as a solution to someone's problem had many golfish paradigms as well.

tybalt89 reported that his program is relatively fast, but still fails on many deals.

Meanwhile, I finished writing Games-Solitaire-Verify and released it on the CPAN. Then I decided to test if it could detect than invalid solution was indeed invalid. And it turned out an invalid solution was still reported as valid (!) making the reported verdict useless. What I found out was that the complete solution analysis silently stopped after the first move or two.

I fixed the bug and released a better version (with more tests). Then I decided to extract a few more classes (Games::Solitaire::Verify::Freecells and Games::Solitaire::Verify::Foundations) out of the relatively monolithic ::State class (which represents positions of the board, including those at midplay), and uploaded version 0.02.

I still need to adapt the module to verify solutions of other variants except Freecell, and that's what I'm planning to work on next. But it was good enough to add some system tests to Freecell Solver (using Perl, Test::More and TAP). What I hope is that tests such as this will allow me to revamp , refactor and enhance Freecell Solver with more confidence that I didn't break anything. I'm especially looking forward to the conversion from GNU Autoconf to CMake, which depends on the test suite to make sure it doesn't break anything.

Finally, I also found out that my C-based Freecell Solver program, did not always output things the right way, without trailing whitespace, etc. Since my primary motivation with Games-Solitaire-Verify was to verify the solutions of Freecell Solver, and because I wanted to compare to the precise output, I needed to emulate these mis-behaviours in my Perl code as well. I don't like it very much, but I guess I'd rather emulate Freecell Solver to test it, than fix it and risk breaking something. Nevertheless, my intention is to provide flags for a saner output for both the C and Perl programs in the future.

Happy Solitaire!

Freenode Channel for Catalyst - ##catalyst

Shlomi Fish — 2008-07-02T21:31:36+00:00

Since I'd like to start working on a new Catalyst-based project, and since I cannot and would rather not chat on irc.perl.org, and since #catalyst (with a single sharp-sign) on Freenode exists only to redirect people to irc.perl.org, and they don't tolerate any other discussion there (which is probably against Freenode policy, but that's life), then I've started a new channel.

Introducing ##catalyst on Freenode - the unofficial and completely non-hostile Catalyst channel. We're already 5 people and one super-intelligent bot there. If you're interested in Catalyst, then please consider joining it too.

You and I Will change the world.

Dealing with Approval Addiction (and Implied Stress Periods)

Shlomi Fish — 2008-06-29T20:05:23+00:00

Well, despite the fact that I hardly publicised my last essay about the "Closed Books", it has been chromatic'd. Rumours are that all the bloggers whose blog posts/essays were deprecated on chromatic's blog are now rich, famous and the object of the affection of many attractive members of the appropriate sex. Memo to self: prepare a limited edition T-shirt: "My blog post was chromatic'd. I pwn you as a blogger."

Seriously now, while the publicity was probably good for me, I was indeed a bit overwhelmed with what chromatic said, and felt down. It's not because I believed I was wrong, but because I respected chromatic, because he's a good programmer, a good author, a good editor (with whom I collaborated on several articles on O'Reilly-Net), an interesting blogger, and a good guy. (And yes - these are all compliments). I'm not sure he's really a friend of mine, but I certainly respect him.

Now Paul Graham says in "What you'll wish you'd known" (footnote 4) that:

The second biggest regret was caring so much about unimportant things. And especially about what other people thought of them.
I think what they really mean, in the latter case, is caring what random people thought of them. Adults care just as much what other people think, but they get to be more selective about the other people.
I have about thirty friends whose opinions I care about, and the opinion of the rest of the world barely affects me. The problem in high school is that your peers are chosen for you by accidents of age and geography, rather than by you based on respect for their judgement.

Now in my case, I may be somewhat immature because I tend to sometimes care about what many people think of me, rather than just my close friends (offline or online). Now being down (or "depressed") because someone disapproves of you, is perfectly natural and normal. However, since I have Bipolar disorder, it threw me into the so-called "hypomania" (= "below-mania"), which is not healthy, and disrupts my functionality.

Hypomanias are a variation of "Clinical Depressions" or "Clinical anxieties". Dealing with the latter is described in the highly recommended book "Feeling Good", which I believe is a necessary read even for non-depressive people, in order to understand how people think, and as a preventitive measure.

I don't accuse chromatic of making me hypomanic. I've received my share of past criticism in the past, and will receive again. I also was often criticised for insulting people myself, due to the fact I tend to be tactless. How you deal with criticism is ultimately the responsibility of the receiving end, as even for me, most criticism will not affect me.

In any case, Feeling Good mentions four general "addictions" that can make one clinically depressed (or Hypomanic):

Approval Addiction.
Productivity Addiction - you care about your work, how productive you are, how much you achieve, etc. (Much more common among men.)
Love addiction - you want to be loved a lot. More common among women.
Perfectionism - you want to be perfect in everything you do.

Now based on reading the descriptions in the book, I believe I have been having Approval Addiction and Productivity Addiction, and neither of the other two.

I used to get into clinicial depressions and anxieties (due to approval, etc.) which are even worse to deal with than hypomanias are. And during Manias, which I also had, but must avoid at all costs from now on, I lose most control of myself.

I started writing an essay titled "Dealing with Hypomanias". It is not professional Psychological advice, as I am not a qualified therapist, but it is given as a way to dispense my advice from the Point-of-View of a Bipolar person, who's learned a little about it. It's still very rough on the edges, as it started from a plain-text email, but any constructive comments would be welcome.

While I may be a bit immature due to my approval addiction, I'm still mature enough to learn from my mistakes. At one time, a prominent member of the Linux-IL mailing list said there that one should not take the advice I've given in the "End of Info-Tech Slavery" (an essay I still mostly stand by, but am about to update with a sequel/correction), and rather take the advice in my links. I was offended from it. Then I understood he was over-generalising and not criticising any of the points I raised in my essay, or explaining why he disliked them, and realised that he was very intelligent, but nevertheless what people call an "idiot". So when he said he doesn't think my Random Tweakers idea for a startup have commercial value (again without properly explaining why or without asking me how I intend to make money), I didn't take what he said to heart.

So now I think chromatic will "suffer" a similar fate in his criticisms against me. No, chromatic is certainly not an idiot, at least not in most regards. But now I know better than to take what he says to heart. I'll listen to what he says and read it at my free time, and often find merit in what he says, but he's still someone I know better than to be affected by him.

I'm not disabling comments here, because I have a policy against it, due to the fact that I believe a blog post without comments is anti-social and defeats the point. But please be gentle, civil, rational and logical. I'm still a bit hypomanic now, though it's getting better, and don't need any more grief. (However, I don't guarantee I'll read what you said immediately, or reply to it.)

I feel this was probably the most off-topic use.perl.org journal post I ever posted, but I've seen much more off-topic blog posts on use.perl.org, and it is relevant to open-source software, as I'm certainly not the only FOSS geek who is either Bipolar or much less Unipolar/Depressive.

And in case you wanted to suggest that: yes, I am taking medication - it doesn't absolutely prevent the hypomanias, but it may help (not sure about that). And I am seeing a Therapist, who gave me a lot of good advice. I should do the so-called "Cognitive Exercises" more-often, but I'm usually carried away with chatting or working on code or text to do them, but they do help a lot.

Here's hoping we can deal with our frustrations more easily, because people are not perfect and the world is not perfect, but they are still pretty darned good.

New Essay about Open-sourcing Books

Shlomi Fish — 2008-06-19T20:51:21+00:00

A conversation on Ask Bjørn Hansen's blog (famous for being an administrator of perl.org and involved in other Perl-related projects) prompted me to write this essay, which talks about why it is unwise and harmful not to make one's books publically available online. Perl is mentioned there and so is Ruby and git and other technologies.

Comment here or on the announcement of the essay on my homepage's blog, where no registration is required for commenting.

XML-Grammar-Fortune

Shlomi Fish — 2008-06-12T09:29:26+00:00

One of my many computerised passions is to collect quotes in UNIX-like fortune format. Throughout the years, I have formed a moderately large collection of them in several files. As time went on, I noticed a few problems. First of all, they were all in large plaintext files, and pointing someone to a quote involved giving a link to the fortune, and saying "search for Foobar". Moreover, since they were just chunks of text, they couldn't hold any meta-data.

At one time, I heard of someone who created an XML grammar to describe Unix fortunes, but a Google search was no help in finding that. And I also have the grand "Fortunes Mania" vision for a community site that for collecting and sorting quotes. This vision was very intimidating, but recently, I decided to take a small baby step by defining a grammar for fortunes as XMLs. So I present to you the XML-Grammar-Fortune distribution.

I've taken quite a lot of time to think about what I wanted there. One thing I concluded was that there are several different types of fortune cookies: run-of-the-mill quotes, IRC conversations, excerpts from screenplays, structured plaintext, HTML, etc. Therefore, the XML grammar should be able to have several different types of sub-nodes, which each corresponds to a certain class of fortune cookies

Until now I've used DTDs for defining my XML schemas, but for XML-Grammar-Fortune, I decided to learn Relax NG, which I was told was easier than the W3C XML Schemas. I was very impressed from Relax NG - it's easy, it's fun, and it's powerful. One problem I've encountered was that, when validating a document using it, XML::LibXML (version perl-XML-LibXML-1.66-1mdv2008.1), does not give the line number where the validation error has occured. To overcome such problems, one needs to look at the diffs or bisect the document.

Anyway, I defined a Relax NG Schema for the documents, and made sure that some basic examples will validate (test-driven-development-style). Then I worked on an XSLT stylesheet to convert them to XHTML.

When I started, I only had one fortune type - <raw>, which is a gigantic <pre> block with some meta-data. I gradually implemented more fortune types: irc, quote and screenplay, whose RNG and XSLT were based on XML-Grammar-Screenplay, with a lot of ugly copying-and-pasting.

I gradually converted more and more fortunes to have a richer XML semantics. The XML grammar requires an id for each fortune, and also allows specifying a title-element, and some fields in the <info> tag, like "author" or "work". For example all the "Friends" fortunes were converted to XML by first normalising the screenplay and then using a script I wrote to convert them to XML.

So I had all the fortunes as XMLs, but now the plaintext versions went out of sync. So I coded a Perl module to convert them from XML to plaintext.

I should note that due to a problem with XML-LibXSLT and perl-5.10.0, I didn't upload it to CPAN yet, because I do not want to receive so many failure reports.

On a different note: my former co-worker has read "Perl for Perl Newbies" in order to learn Perl, liked it a lot, and told me I should add more to it. That also feels good.

Nominating szabgab for the White Camel Award

Shlomi Fish — 2008-06-03T21:20:46+00:00

I alread sent it to jose-at-pm.org and advocacy-at-perl.org, but here it is just in case:

I'd like to nominate Gábor Szabó (the Hugarian-Israeli Perl programmer, not the Jazz musician) for his contributions to the Israeli and Global Perl Communities. See:

Gabor Szabo is responsible for the fact that the Israeli Perl community, and that of other dynamic languages has took off. So far he:

Set up an alternate mailing list, which proved to be more active - perl@perl.org.il.
Set up a tradition of monthly Israeli Perl meetings, which have turned out to be popular.
Organised 3 YAPC::Israel conferences - YAPC::Israel::2003, YAPC::Israel::2004 and YAPC::Israel::2005 and an OSDC::Israel conference - OSDC::Israel::2006.
He helped organise the Hungarian Perl Workshops in Hungary, and a Hungarian YAPC.
He set up CPAN Forum - as a way to get help and ask questions with individual CPAN distributions. Also allows users to tag their modules using delicious-like tags.
His commmercial venture - http://www.pti.co.il/ - has been responsible for training material and sessions for Perl and other FOSS technologies, as well as general FOSS development and support. Some of its material has been made available online free-of-charge.
He has written many weblog entries, comments, emails and other material about communal and technical issues.
Aside from all that, he has done numerous contributions of code, and documentation.

Next OSDClub Tel Aviv Meetings

Shlomi Fish — 2008-06-02T23:25:09+00:00

On Tuesday, 3-June-2008 (tomorrow or today depending how you look on it), on 18:30, the OSDClub of Tel Aviv, which is a joint venture of the Tel Aviv Linux club, and Perl-Israel (and some other FOSS clubs who want to join the fun), will hold a social meeting. It will take place at the Café of Tel Aviv University near the junction of Einstein and Haim Levanon Streets. This social was scheduled because Zvi is in Israel. (If you don't know who he is, then come to meet him.)

Much later, on Sunday, 15-June-2008, OSDClub Tel Aviv will meet to share some Vim/gvim (= the text editor) Tips and Tricks. We'll meet in Schreiber (Math & CS) building in Tel Aviv University. We have more meaty presentations planned for July, but we hope these two meetings will be a a useful start after a long neglect.

No outgoing email to @perl.org and @pm.org

Shlomi Fish — 2008-05-29T19:54:05+00:00

Hi all! Due to a technical problem, I have been unable to send any outgoing email to the @perl.org and @pm.org domains from either my home email address or my gmail.com email address. This is while outgoing mail to other places is mostly working, and most people have no problem sending email to the perl.org and pm.org domains. I receive bounces on any mailing list addresses, or @perl.org forwards.

This is the reason why I've been unusually quiet there lately, so don't worry. I can still be reached in many ways, and have many ideas for things I'd like to post on my blogs and on my homepage.

Anyway, cheers, and sorry for the inconvenience.

Perl vs. Ruby on Two Idioms

Shlomi Fish — 2008-05-22T10:22:57+00:00

I've been meaning to blog about it for a few weeks, so here goes. In a Ruby-Israel thread someone asked how to convert an array to a hash that contains all of its keys as members (and "true" as a value). They came up with:

a = [1, 2, 3, 'other'] h = a.inject({}) {|h, v| h[v] = true; h }

In Perl, it's:

@a = (1,2,3, 'other'); my %h = (map { $_ => 1 } @a);

Much cleaner. I also though of a way to do a flat concatenation of an array of array references. Like [[1,2,3],[4,5,6],[7,["One", "Two",],9]] into [1,2,3,4,5,6,7,["One", "Two",],9]. In Ruby it is:

[[1,2],[3,4],[5,6,["Hello","There"],7]].inject([]) { |a,e| a+e }

While in Perl it is:

(map { @$_ } ([1,2],[3,4],[5,6,["Hello","There",],7]))

Both of these are caused by the fact that lists are not references in Perl, and that one can initialise arrays and hashes from them. This is while in Ruby or Python (and most Lisps) they are references. That or Ruby lacks more primitives.

PerlBuildSystem (PBS) is now FOSS

Shlomi Fish — 2008-05-17T21:31:10+00:00

Got no Google, got no email, got no love - gotta blog!

I met Nadim (NKH) on Freenode's #perl, and he seemed very pleasant. I told him how I heard everybody got a kick out of his App::Asciio graphics app. Then when visiting his CPAN page, I noticed that he's the author of PerlBuildSystem, which had been released under a problematic "not-for-military-use" licence, which sparked an active discussion on module-authors back at the time.

I asked Nadim if he can make it non-restricted, and he said that he could. So he uploaded a new version under the Artistic License 2.0, which is the Parrot licence, is FOSS, and is GPL-compatible. W00t! Nadim++.

He also said that he is working on a heavily improved version, for which he uses git, and should be released sometimes. I'm contemplating converting my homepage from a really-funky GNU make configuration to something more decent, and now that PBS is unrestricted I may take a look.

Why People Are Passionate About Perl

Shlomi Fish — 2008-05-09T20:04:14+00:00

Well, Per brian d foy's blog post, I'd like to answer the question "Why I'm Passionate About Perl". First of all, I should note my "The Joy of Perl" essay, which I wrote back in 2004. It gave a lot of good reasons why I like Perl so much and am still passionate about it. But now to answer brian's questions:

The person who introduced me to Perl showed me that... - I still remember him (Ran) telling me that one can write a TCP/IP client in 4 lines, and a TCP/IP server in 10 lines. (Or something like that). I ended up not understanding what regular expressions were all about and he explained that they matched patterns in texts. Back then, I had to learn perl 5 from perl*.pod (what are now the perldocs).

I first started using Perl to... - had to learn Perl (and Unix) because I wanted the job working for Ran's company. I liked both Perl and Unix so much that I understood why I had not been content with all the technologies I encountered previously. (DOS and Windows 3.11).

I kept using Perl because... that was what I knew and I was comfortable writing it, and found that most various techniques were readily available in some form or another. For example, after reading SICP I was able to easily implement the closure-based techniques shown there in Perl. Then after I learned Object-Oriented Programming in Perl, I found out how nice it was, and how much better it was than C++. (Which if you ask me now supports Object-Oriented Programming roughly as much as COBOL supports Functional Programming.)

I later on learned how to effectively use the CPAN, use accessors, and many other tricks and techniques. I got involved in the Local and International Perl community, which was also a lot of fun.

I'm not a Perl fanatic and see myself sometimes using, learning or experimenting with other technologies. But I still like Perl 5 the most.

I can't stop thinking about Perl... - actually I often can. The amount of time I spend coding is small, and Perl is even less than that.

I get other people to use Perl by quietly telling them how Perl is important and how it is enlightening and useful, and also telling them about the things I'm doing with Perl.

I also program in Bash, C/C++, PHP and a little Python (and small experimental stuff in many other languages) but I like Perl better since: 1. It's a real, and safe programming language unlike Bash. 2. It's much more easier to write than C, C++ or Bash. 3. It's more comfortable to me than Python due to the TMTOWTDY, use strict and other factors. (Though I can understand why Python has its appeal to some people.) 4. It's much less hacky than Bash and PHP. 5. The Perl community is great, and has a very healthy attitude.

Note that I'm still using bash for the command line and for some really minimal scripts, and am happy with it, and prefer it over Perl.

Comments are welcome. (I leave the comments open, as I almost always do).

Looking for a Good Personal Blog Engine

Shlomi Fish — 2008-05-08T16:26:11+00:00

Dear Lazyweb,

I'm looking for a recommendation for a good personal blog engine that I'd install on my site. It should be Free Software (preferably GPL-compatible); it should be Perl, Python or PHP (Perl is preferable), possibly also Ruby; it should be able to use PostgreSQL as a backend; and it should be good: easy to install, mostly works out of the box, easy to extend, with an active developer community, readable (not necessarily too modular) code, good security practices, etc.

Here's what I tried so far:

MovableType - has a weirdo HTML caching system, and ended up putting a lot of world-writable files on my hard disk. Also doesn't work out of the box with recent PostgreSQL's (which is easy to fix).
WordPress - from using it elsewhere I had at least three occassions where it ate my comments and refused to allow me to repost them. Also, it has many bad defaults like non-threaded comments, A single "Submit" button with no preview, no pure-HTML input, and these weird ?id=$INDEX URLs which I hate. All of them can be fixed using Plugins, but it's still an extra hassle.
It also had a very poor security record, and pkrumins said he isn't content with it for his blog, and working on something from scratch.

There are many other blog engines out there, but I'm looking for a personal recommendation from experience. Comment below or drop me a line. I'll probably blog here about what my final verdict is.

New Essay: "What Makes Software High-Quality"

Shlomi Fish — 2008-05-06T11:42:11+00:00

I wrote a new Essay about what makes software applications high-quality, as well as which parameters and methods, while desirable and quality-enhancing, are not exactly quality. It was inspired by a post on a public mailing list I set up, and Perl and perl 5 are mentioned there a few times, as are many other open-source projects.

It is available in several formats: HTML to read online (on Page), DocBook/XML source (which can in turn render other formats), PDF (please don't print it even though you are legally allowed to). The licence is CC-by-2.5. Comments are welcome.

Test-Run Tests Breakage on BSD Systems

Shlomi Fish — 2008-04-18T20:29:06+00:00

OK, as I expected my previous entry sparked an active discussion - nothing like a good licences war to liven things up. But it was more civil than I expected. Here's a much more technical entry.

As I discovered, Test-Run-0.0115 consistently failed to pass "./Build test" on all BSD systems, while doing mostly fine on Linux. Inspecting the logs of the failure yielded a "File name too long" error. What happened was that I created a filename that was artificially very long (../t/../t/../t/), but still well within the limits of my Linux system's 4096 bytes limit for file paths. However, as I discovered the POSIX standard defined a minimum of 256 bytes for maximal paths which is what BSD is supporting.

The reason I had this long path in the first place was to make sure long paths are handled properly by the harness output after some customisations. This in turn was inspired by a problem I found when using Test-Run at my workplace for some internal test suite, which inspired me to write the Trim-displayed-Filenames plugin.

So after I received all these failure reports, I added some logic to t/output.t that makes use of POSIX::PATH_MAX() to keep the path at bay. A bit convulted, but it now passes on BSD systems (as well as Linux and Solaris), with a two isolated failures on Linux, which I have not looked into yet. I'd like to thank apeiron from Freenode for testing the pre-release in Mac OS X and verifying it works there.

In any case, I'm a bit tired of doing unknowledgable UNIX programming, and therefore would like to read the 2nd edition of Stevens' book (which is considered the Bible of UNIX programming). The book is kinda costy, and big (960 pages), so I think I'll renew my Safari subscription and see if I can read it there effectively. If I can't I'll just use it for something else, and order a paper-copy of the book.

And finally, I wonder how a 256-octets path limit can ever be enough. In this day and age of long filenames and UTF-8 ones (which require several bytes), one can expect that a path with a few especially long components will quickly overflow such a short limit. Can any BSD users comment?

The Problem with the "Same Terms as Perl" Licensing

Shlomi Fish — 2008-04-03T20:22:43+00:00

This entry was adapted from some comments I wrote for a different, mostly orthogonal, journal post. I hope it's not too flamebait. If you have something to say, please try to be civil and good-natured. Here goes nothing:

Most modules on CPAN carry a licensing blurb saying something like "you can distribute this software under the same terms as Perl itself". This obviously begs the question what does it mean. Here is what it means for different Perl versions:

Early versions of perl, before it adopted the GPL were under a very restrictive loosely-defined licence.
After a while perl adopted the GPL exclusively.
Eventually, seeing that some people and companies had problems with the GPL, Larry Wall phrased the so-called "Artistic License", which was supposed to be more permissive than the GPL, but purposely phrased vaguley, and as such was found to be non-free and ergo GPL-incompatible.
In any case, the perl implementation was dual-licensed under the "Artistic License" (version unspecified) and the GPL version 2 or any later version. perl5 is still licensed under these terms.
For Parrot and for future work on Perl 6, the Artistic License 2.0 was created based on the original Artistic License. This license has been approved as a free software licence by the Free Software Foundation and found to be compatible with the GPL version 2 or Later. Parrot now carries this licence.
There's also the Clarified Artistic License which is an earlier effort to correct the original Artistic License and is the minimal set of changes to make it FSF-free and GPL-compatible. I suppose that now the Artistic License 2.0 would be preferable.

The problem is that perl up to and including perl-5.10.0 and bleadperl (which were licensed under the GPL version 2 or above but only the "Artistic License" not the "Artistic License version 1.0 or above". I've already noted that the Artistic License is problematic, but the GPL has its own share of potential problems (see also its FAQ), and is otherwise widely mis-understood, over-hyped (both positively and negatively) and disputed. For example, unlike the Artistic License, it does not allow (at least by linking or normal function calls) non-GPL-compatible code, including all non-open-source code, to make use of it.

Now, since both the "GPL version 2 or above" and the original "Artistic License" are problematic, and the term "under the same terms as Perl" may probably mean just that if it's a perl5 module (although who knows what it would means if someone would ever write a compatible Perl 5 implementation under a different software licence), then licensing a module as such is probably no longer such a good idea. There are several alternatives:

Explicitly allow the Artistic License 2.0 (or possibly above) in the licensing.
License as Artistic 2.0 or above exclusively. Without the GPL as it is generally no longer needed.
Use a more permissive, non-copyleft licence, that allows re-licensing into different licences. The MIT X11 licence explicitly allows "sub-licensing" by third-parties and as such is a good choice. By doing that you are making the licence more permissive than the GPL or the Artistic licences, so you have been warned.

So my suggestion is that you use one of these licensing phrasings for your future work, and to re-license your old CPAN or Perl work (copyright ownership permitting) under such phrasings. It remains to be seen what will happen with perl5 itself which has 905 authors as of perl-5.10.0 many bringing in their own ownerships of the code. The Linux kernel now faces a similar problem if it would wish to adopt a different licence than the GPL version 2 (and no later version).

I suppose you can still make most use of perl5-like licensed code, in your own open-source, proprietary or in-house code, without getting sued, so I wouldn't worry too much. But it would still be a good idea to convert newer code (and code that can be easily converted) to licensing terms that are less ambigious, more usable, and that would play better with future versions of Perl.

For the record, most of my Perl and non-Perl open-source code is either Public Domain or MIT X11-licensed (which are both extremely permissive and allow sub-licensing), or if it is derived from a different code, then under the same licensing terms, but while disclaiming my explicit or implicit ownership (to allow the originator to relicense future versions of the code). The latter policy applies to my (relatively limited) contributions to perl5 where I am listed in the AUTHORS file.