Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Shlomi Fish (918)

Shlomi Fish
  shlomif@iglu.org.il
http://www.shlomifish.org/
AOL IM: ShlomiFish (Add Buddy, Send Message)
Yahoo! ID: shlomif2 (Add User, Send Message)
Jabber: ShlomiFish@jabber.org

I'm a hacker of Perl, C, Shell, and occasionally other languages. Perl is my favourite language by far. I'm a member of the Israeli Perl Mongers, and contribute to and advocate open-source technologies. Technorati Profile [technorati.com]

Journal of Shlomi Fish (918)

Wednesday May 13, 2009
09:30 AM

Meta: Script to Filter the Master use.perl.org Blogs' Feed

As expected from the latest trend in the Perl blogosphere this post will be about Roles. And Moose! And Roles in Moose! And Moose in Roles! And Roles outside Moose…

Seriously now, this is a post about a completely non-Moosey and non-Roley script I wrote to filter the use.perl.org master journals' feed. What this script does is fetch the Atom feed of all the use.perl.org journals' posts, and filters out the entries of the use.perl.org authors that the invoker specified.

Here is out to use it. First of all: svn checkout it (or otherwise fetch it using HTTP). Then you can simply use:

perl filter-use-perl-journals.pl -o everything.atom

Then you can serve everything.atom with a web-server to read it using your web feeds' aggregator.

To simply create a non-filtered copy of the feed. Now let's say you want to get rid of posts from my journal (because it sucks). In that case, say:

perl filter-use-perl-journals.pl -o non-shlomif.atom \
    --blacklist="Shlomi Fish"

Now, let's say you also want to get rid of Ovid's posts. I have no idea why you'd want to do that, because his journal is great, but it's just for the sake of the example. In that case, do:

perl filter-use-perl-journals.pl -o non-shlomif-and-ovid.atom \
    --blacklist="Shlomi Fish" --blacklist="Ovid"

Finally, there's the --rand flag which is useful in case you're running the script with cron. What it does is wait for a period of time of random length, before fetching the feed, so the HTTP server would not be overloaded at regular periods. This requires a working /dev/urandom for the time being.

The script is made available under the MIT/X11 Licence so its use is completely unencumbered. I wrote this script today, because I have a personal use for it, but other people may find it useful too. It requires a recent version of Perl (5.8.x should be enough I think) and XML-Feed.

Tuesday March 17, 2009
03:51 AM

Gabor Szabo on High-Level Programming with Perl 6 on 22-Marc

The Tel Aviv Open Source Club will host the first part of a series of talks by Gábor Szabó about "High-Level Programming Concepts Using Perl 6" - on 22-March-2009.

The meeting will take place at Tel Aviv University (in Tel Aviv, Israel), at the Schreiber MathsCS building, room 008 on 18:30. More information can be found on the meeting's page on the wiki.

With any other problems, feel free to contact me.

Abstract

High-level programming concepts using Perl 6

A series of presentations on learning and using Perl 6 from the ground up to the special features.

Many would think that Perl 6 is just a new version of Perl and that it might only be interesting for Perl programmers. However, Perl 6 is in fact a compiled language running on a virtual machine that embraces many new concepts not found in most programming languages.

The presentations will be equally interesting for Perl, Java and C# programmers.

During the series of talks we will start by learning the basics of the language and will get to various high level concepts. For now we plan 2 sessions but if we need more time we'll schedule more meetings.

Note

After the talk we will go to the café at the main entrance of the university where we can continue the discussion. If people bring portable computers, we can get the off the ground on the spot. VirtualBox images will be provided with everything that is needed for playing with Perl 6 set up inside. So you may opt to bring a computer with VirtualBox installed.

We are always looking for presentations on interesting topics. If you have an interesting idea for a talk, feel free to contact us and we'll co-ordinate a date.

Wednesday January 14, 2009
04:58 AM

"The Perl Future" on Heise

Piers Cawley writes about the new happenings in the Perl world in "Healthcheck: Perl - The Perl Future" on Heise Open Source. Even though I was already aware of most of what he covers there, I found it a good read, and it is doubly recommended for people who are not following Perl news as thoroughly as I do.

Also see the LWN.net coverage (where I've learned about it).

Monday December 01, 2008
12:18 PM

XML::RSS-1.40 - Now with a Lot of Array Goodness

After going over some of the recent and not-so-recent XML::RSS bugs, I noticed a common pattern of people wanting support for replicable XML elements. What they wanted was that when two elements with identical names were specified (in many RSS use cases where doing this would make sense), then the value pointed by the keyname would become a reference to an array instead of just concatenated together in an unhelpful manner.

In the past days, I decided to work on this meta-problem in one concentrated swoop. I naturally had to implement it in several places for both output and parsing, though tended to do them in separate commits.

XML-RSS version 1.40 contains the result of this effort. I should note that some use-cases for multiple tags probably won't do the right thing, and the internal code has garnered some ugliness. I ended up extracting some methods, but I can still be happier from the quality of code. I suppose I can always refactor it later.

All of this work reduced the number of active bugs in XML-RSS to 3, which I intend to deal with shortly, if all goes well.

In other news, I released a new version of WWW::Search::MSN, which has been adapted to the new HTML generated by the MSN search. The hardest part in the process, was figuring out how to instruct the libwww-perl that the WWW::Search front-end used to send a "Language:" header to cancel the location-based localisation.

That was my boring "recent hacktivity" report. You may now go on with whatever you did earlier.

Sunday November 16, 2008
03:44 AM

Why Dist-Zilla is Probably Not For Me

rjbs' introduction to Dist-Zilla piqued my interest, and I used CPANPLUS-Dist-Mdv to prepare .rpm's for it and its depenedencies and install them. However, I wondered about a potential problem with it, before I even tried it, and speaking with rjbs on IRC confirmed that it exists.

Dist-Zilla generates the resulting .pm, scripts, etc. from templates, and as a result the lines that are reported by errors and warnings are not the same as the ones you've edited. This makes tracing lines back to their source much more difficult. Since most of my times is spent debugging and handling errors (whether I encounter them or I find them on CPAN testers or in bug reports), and I want to edit the source directly, I think that Dist-Zilla is not for me. What instead I'd like is a way to change common, repeated text (like version numbers, licensing, URLs, etc.) commonly across all the files in a certain distribution, or in many distributions. That and also that module-starter (or whatever) provide scaffolding for creating a new .pm file.

I appreciate rjbs's efforts, but I'm going to stick with module-starter (and contribute to it).

Monday October 27, 2008
01:34 PM

File-Find-Object Refactoring

I haven't updated this journal in a long time, and mostly been writing in my LiveJournal Technical blog. I've emailed to the Perlsphere maintainer about including that blog there (once from my home address and once from gmail.com), but received no reply, and so far it is not included there.

In any case, this post is about File-Find-Object, which is an object-oriented alternative to the core File::Find module. Originally by Nanardon, I've maintained it since version 0.0.3.

Recently, I've been meaning to write a project, and contemplated whether to use File-Find-Object or roll-out my own directory traversal logic, but then decided that improving F-F-O so it will do what I want was a better idea, in an "Eating one's own dog food" and not-re-inventing the wheel fashion. So I needed to extend F-F-O to do what I want.

While I took a closer look at the code to inspect it, I found it had some "bad smells", and decided to fix it by refactoring, as a necessary pre-requisite for extending it.

The first thing I did was notice that many methods accepted a $current parameter that was passed from one method to another, and then used. As it turned out, most of these simply originate from $self->_current(), which I now used to retrieve the value in each method, without passing a parameter.

Another fact I noticed was that there were many if ($self eq $current) checks in the code. Since $current is dynamic, I decided to create a predicate _is_top() which will encapsulate it and to create the following method maker:

sub _top_it
{
    my ($pkg, $methods) = @_;

    no strict 'refs';
    foreach my $method (@$methods)
    {
        *{$pkg."::".$method} =
            do {
                my $m = $method;
                my $top = "_top_$m";
                my $non = "_non_top_$m";
                sub {
                    my $self = shift;
                    return $self->_is_top()
                        ? $self->$top(@_)
                        : $self->$non(@_)
                        ;
                };
            };
    }

    return;
}

Thus, when _is_top evaluates to true I call _top_mymethod and otherwise _non_top_mymethod. This is a variation on the "replace conditional with polymorphism" refactoring.

Now ->_current() returned the ->_current_idx()'th item from an internal stack representing the directories which the object is traversing. I wanted to see where "_current_idx" was set and discovered it was incremented when an item was pushed to the stack, and decremented when an item was popped. As a result, I eliminated "_current_idx" completely and replaced _current() with $self->_dir_stack()->[-1]. That removed a lot of cruft from the code.

I also was able to do what I wanted, and make sure the paths are maintained as the base path for the traversal followed by a list of extra components of each inner directory.

I noticed that I flat-copied the return of a method returning an array reference several times (E.g: [ @{$self->_components()} ]) and so created another method maker - this time for "_copy" methods.

And naturally, I extracted many methods.

All this enabled me to create ->next_obj() and ->item_obj() API methods that return objects instead of plain strings. Naturally, this is not refactoring, but extending.

While I was in the neighbourhood, I discovered that the code had a format-string-issue, which I fixed, and released File-Find-Object-0.1.1 immediately.

After the release, I received three failure reports from CPAN Testers. Two of them were for a missing dependency, that wasn't installed due to a bug in the tests' smoking setup. One of them, however, was for Win32, where it was a real bug. I was able to reproduce it using Strawberry Perl on my WinXP computer, and released File-Find-Object-0.1.2 that corrected it. The problem was that on non-cygwin-Win32 all inodes are returned as 0 in the calls to stat().

Today, I started writing the project that required all this work on File-Find-Object. So far it doesn't do much, but it's a start.

Thursday September 18, 2008
09:24 AM

OSDClub Tel Aviv Meeting: Ori Idan about the Semantic Web

The Tel Aviv Open Source Developers Club (OSDClub) (formerly the Tel Aviv Linux Club and other clubs such as Perl-Israel) will hold a meeting on 21/September (next Sunday). Ori Idan will deliver a presentation about the Semantic Web.

Ori is the director of the Israeli branch of the World-Wide-Web Consortium (W3C), and is a very good presenter, so it is recommended to attend.

The meeting will take place at 18:30 in the Schreiber Maths and Computer Science building of Tel Aviv University, room 008. Attendance is free of charge and everyone are welcome.

Thursday September 04, 2008
07:11 AM

Resolution for maintperl-5.8.x's IPC::SysV failure

As a followup to this post about "make test in maintperl-5.8.x Fails on Linux", I should note that we have investigated it on perl5-porters in the past days. After a little investigation, I realised that there was a stray "SysV.pm" file in my perl-5.8.x tree, which caused all the problems.

As it turned out, it was not removed because I used "rsync -auvz" to synchronise the tree (as instructed in the perl5-porters FAQ) instead of "rsync -auvz --delete-after", which removes the no-longer present files. After running rsync with --delete-after, and building again - "make test" was successful.

"Another crisis was solved!".

Thursday August 21, 2008
03:12 PM

Recent Hacktivity Summary

Well, I added more tests to Module-Starter, and while I was writing the tests, I discovered some bugs in it. So now there's a patch with the tests and the bug fixes and another one with only the bug fixes. Both of them are unapplied.

Having been trying to learn Ruby recently, I decided to write a Conway's Game of Life implementation for it. It took me most of one evening to get it working and then part of the other day to make it more idiomatic with the help of the people on Freenode. It felt good in a way.

Then I decided to finally see why my Test-Run failed many tests recently. As it turns out, this was due to broken TAP-Parser compatibility: the skip message in skip_all field now accepts only "SKIP " and not something like "skipping: ". Fixing the Test-Run tests fixed it and I released new versions.

I also noticed that many of the Test-Run-CmdLine plugin had the following pattern:

sub _init
{
    my $self = shift;
    $self->NEXT::_init(@_);
    $self->add_to_backend_plugins("AlternateInterpreters");
}

So I overloaded the construction function, only to add a plugin. I decided to fix this by doing an accumulation of such plugin specifiers in every plugin, and so got only:

sub private_backend_plugins
{
    my $self = shift;

    return [qw(AlternateInterpreters)];
}

Which is much cleaner and less intrusive.

Next on my plate was Freecell Solver. I decided to work on make_pysol_freecell_board.py which is a Python script that generates the initial boards of PySol. The script was crudely written to begin with and it became even cruder as time went by. I decided to give it a serious face-lift. So what I did was:

  • Convert the columns (of the card games) from strings to arrays. Each card was stored there individually.

  • Convert the cards to a class, instead of integers and strings. Create similar classes for managing columsn and boards.

  • Extract many functions and methods.

  • Create a class to manipulate the various game types, and forward the different logic based on it.

  • Pythonised the script by employing some Python paradigms.

You can find what I have so far in the repository. It's much better than what I started with two days ago. Writing such Python code now seems more fun than I recall it, and I actually enjoyed it.

In regards to SMOP, they convinced me to install Ruby-1.9.x under a prefix, which I did, but then it yelled at me for not having a Haskell cabal thingy. Turns out that the Pugs Makefile.PL installs it somewhere under the home-directory, which I didn't want to happen, because I want to keep it tidy. Again, this reminded me of Drivl.com recipe on how to make square corners, and I decided to give up on SMOP again.

And I should note that I was able resolve a long-standing problem I had with XML-LibXML/XML-LibXSLT on my Mandriva system, and now I simplified the XML-Grammar-Fortune build-system.

I also spent some time writing a backup system for some of the Israeli MediaWikis that I manage. This involved a bunch of Perl scripts.

So - Perl, Ruby and Python - all in a few days work. Cheers everybody.

Tuesday August 12, 2008
03:28 PM

Random Hacktivity Summary

This is another boring hacktivity summary, but this time mostly Perl-related - so I'm posting it here.

First order of business is Games-Solitaire-Verify. In its first versions I implemented textual return values for the verify_and_perform_move() method. However, this makes it harder and more error-prone to programmatically understand why exactly the verifier does not like your move . True to my ideals that errors that need to be caught and processed must be objects, I converted everything to return Exception::Class-subclasses. E::C really made my job easy and now my main gripe with everything is that my error class names have become unweildy. Some examples are:

  • Games::Solitaire::Verify::Exception::Move::Src::Col::NotEnoughCards
  • Games::Solitaire::Verify::Exception::Move::Src::Col::NonSequence
  • Games::Solitaire::Verify::Exception::Move::Dest::Col::OnlyKingsCanFillEmpty

Oh well. In any case, now that I've converted everthing to classes, I forgot to add stringified versions of the errors (which Exception::Class supports), so if something breaks and is reported to the shell, it may not be very human-friendly. I guess it means a bit more work there.

Afterwards, I decided to work a bit on Test-Run, and after I read its todo list, remembered that I still intended to create some specialised Module-Starter plugins for it. Then I remembered that I asked about M-S' test suite, and found out it didn't have a comperensive one. So, I decided to remedy this and wrote a patch to add a test script. For the test script I parsed the generated files step-by-step using a class and some test assertion methods.

Since I had no repository access, the first versions of the patch were written by repeatedly running svn diff into files, but I decided I want a more robust version control for the local host. So I looked into svk but it had acquired a bug on my system.. Then I decided to try bzr-svn, but it had a non-starter bug. Now since I dislike git quite a bit, I searched for something Mercurial-based and found hgsvn. This worked surprisingly well for my purposes and I was able to version the local copy. Mercurial seems pretty nifty so far.

The Module-Starter patch was not integrated into the repository yet. Module-Starter has been suffering from quite a lot of neglect, and rjbs (who performed its latest upload to the CPAN) told us on IRC he now uses a different solution that was written by someone else and has not been made public yet. I'd really like to see something better emerge, but I hope I won't have to fork Module::Starter under a different name.

I decided to stop patching it for the time being, but am getting anxious. It makes me frustrated to have an uncommitted patch and no repository access with which I can apply it.

As I was working on Module-Starter I talked with the SMOP guys, and decided to try give it a spin. After checking out the source and doing the make -f Makefile.cvs, ./configure, make dance, I ran into a few missing ".pm" files error. The first one was caused by the lack of the rest of the pugs tree, and the later ones were easily installable from CPAN. But then I ran into a strange error:

../../../misc/elfish/elfX/../../STD_red/STD_red_run:93: syntax error, unexpected '>'
    whiteout = ->(s){s.gsub(/[^ \n]/,' ')};

As it turned out, I needed Ruby-1.9.x (the development version that naturally is not avaialable in Mandriva or other Linux distributions), and decided that there's no way I'm installing it. This reminded me of Drivl.com recipe on how to make square corners, and I decided to give up on SMOP.

After all this, I also added a lot of text to the CMS/Web-devel-framework/Web-design Wikia, which I founded and had neglected previously, and to the pages in the Star Trek "Expanded Wikia" which I maintain, which were a nice change in scenery from all the code I wrote.

Today I didn't do too much productive work, after having my usual post-productivty disorientation, and instead spent the first half of the day chatting on IRC (to a very interesting American linguist-wannabe), but was also able to bike, for the first time since Saturday. Oh, and I saw Wall-E on Sunday, which I liked. Been a while since I've been to the cinema.