Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Journal of johanvdb (1872)

Tuesday February 25, 2003
06:06 PM

A brand new life!

At last ... after 9 months of waiting me and my wife Evy got their first baby. A healthy daighter of 3.410kg, 49cm, she is named Milla ...
This is the best thing that ever happened to me ... I'm a happy guy!

Let us hope we can keep it that way ... I'm still a bit shaky about the troubles I had with a module on CPAN, which I was not-so-kindly asked to remove from the entire Internet ... (http://yro.slashdot.org/article.pl?sid=03/02/07/1849243&tid=156) Anyway ... I've written a text about it, but I'm reluctant publishing it, maybe it can trigger some unwanted action ...
Back to the Perl side of things, I hope that the 0.161 release of GSM::SMS is solid enough ... It passes all the tests and works for me on Win32 and Linux. Ooh, I've soo much hunger to start writing an OSS Multi Media Messaging project ... I've found an openmms thingy (http://www.openmms.de/), it is written in Perl, as the authors have talked about it at the latest YAPC (http://www.yapc.org/Europe/2002/abstracts.html, search for mms) ... but I guess they don't intend opening the code to the public ...

Is there anyone who has all the specs of MMS?

Johan

Tuesday December 24, 2002
07:19 AM

New stuff on the horizon ...

[ #9614 ]

At last, I got the new GSM::SMS release ready. Now it has support for Win32 and some other goodies like Flash SMS - messages that pop up directly on the mobile's screen.

I also have a holiday and I hope I can find the time to finish-up some of my ideas - RF link for one of my robots, making WWW::***** (note: post-edited this, some will know why ...) a bit better and some other stuff ...

But first, I'm going to cook for my family ...

Aha, a nice x-mas gift from plainblack.com, WebGUI 5.0 is released! That seems fun ... need to upgrade my server.

Johan

Thursday December 05, 2002
06:53 PM

Tempus fugit

I've been successfull in building the hardware for my wireless module that will power the data transmission between one of my autonomous robots and the main server. The first databits have been sucessfully send. My previous hardware design performed very bad, but the new design is really performant. So ... what has this to do with Perl? Well, the server application is being written in Perl. I hope to have a page up soon on which I can talk about my new efforts in autonomous robots ... a preliminary version can be found at ai.tektonica.com.

Signing off,
Johan

Monday November 18, 2002
08:20 PM

Perl threads getting slower?

I recently got absorbed by threads in Perl. Why? Well, because I'm working on a product - iSMS - that I hope to be of use to some with some money to spend, otherwise it'll go OSS as is its predesessor. Anyway ... I want to use threads instead of the fork() or IO multiplexing via select().

So I set off reading about and experimenting with the new Perl threading implementation. A search on Google for some 'prior art' on Perl threads gave me the following site that compares threading in different languages.

I just compiled 5.6.1 with -Duse5005threads and checked the Thread implementation with the threads implementation in 5.8.0, sorta ...

The following 'apps' were used:

---------------- Thread

#!/opt/perl561t/bin/perl

use strict;
use Thread qw(cond_wait cond_signal);

my $count = 0;
my $data = 0;
my $produced = 0;
my $consumed = 0;

sub consumer {
        my $n = shift;
        while (1) {
        lock($count);
        cond_wait($count) while ($count == 0);
        my $i = $data;
        $count = 0;
        $consumed++;
        last if ($i == $n);
        cond_signal($count);
        }
}

sub producer {
        my $n = shift;
        for (my $i=1; $inew(\&producer, $n);
        my $c = Thread->new(\&consumer, $n);
        $p->join;
        $c->join;
        print "$produced $consumed\n";
}

main;

---------------- threads

#!/opt/perl58/bin/perl

use strict;
use threads;
use threads::shared;

my $count : shared = 0;
my $data : shared = 0;
my $produced :shared = 0;
my $consumed : shared = 0;

sub consumer {
        my $n = shift;
        while (1) {
        lock($count);
        cond_wait($count) while ($count == 0);
        my $i = $data;
        $count = 0;
        $consumed++;
        last if ($i == $n);
        cond_signal($count);
        }
}

sub producer {
        my $n = shift;
        for (my $i=1; $inew(\&producer, $n);
        my $c = threads->new(\&consumer, $n);
        $p->join;
        $c->join;
        print "$produced $consumed\n";
}

main;

---------------------------------------

I got the following test-bench results and was a bit astonished

> time ./prodcon561.pl 100000
100000 100000

real 0m10.287s
user 0m5.360s
sys 0m4.930s

> time ./prodcon580.pl 100000
100000 100000

real 0m15.660s
user 0m11.780s
sys 0m3.880s

Subsequent runs give the same result ... ithreads are 50% slower than the 5005 threading model.
I've done a multithreading daemon in Perl 5.5(something) around 2000 to 2001 and found it rather stable enough for production capabilities ... although all advisory texts show me otherwise...

Anyway ... I guess this result has to do with the fact that ithreads don't share data by default. So an extra copy (clone) of variables can be the cause. Again ... speed has been traided for ease-of-use (and thread safety).

Hummm ... bad news ...

Anyway ... no hard feelings ... if I can get the expected performance ... no harm is done anyway.

For the curious - and now I assume other people are actually reading this weblog - I'm using threading in my *product* in the following way ...

First a boss/worker kind of thingy, in which incoming requests - handled by the boss - are passed to a worker which will handle the request. This way we get a better response time...

Secondly a thread is launched at application start that implements a HTTP daemon. This daemon is used for configuration of the application. This thread implements an embedded web server used for handling the configuration ...

Aha ... and as a last treat something I've been working on lately also - patterns in Perl. I was just skimming the new google news site for Perl news and found a rather intrigueing article a bout a guy who is trying to get his 'Patterns in Perl' book published ... It's a rather sad post, but the good thing is his work on patterns in perl. He deserves a publisher if you ask me ... even if it only was for hist effort (can't comment on the book quality though ... had no time to read it). The post can be found on:
http://www.advogato.org/article/579.html

We'll we are at it, you can also look at another PIP initiative: http://patternsinperl.com/

Regards,
Johan

Thursday October 31, 2002
04:16 AM

Perl, PVR and SAX

Recently I read an article on perl.com that really showed the strength of using SAX and family.

This got me going and experimenting with custom SAX event handlers and generators.

Now, the place I work is full of übergeeks - in the positive sense of the word - a few of them are building there own Personal Video Recorder and introduced me into a special XML format, called XmlTV. This format is used to describe TV program listings, like channel, hour and show that is on the telly.

Their favourite system, freevo, seems to be build with Python - and that is a pity off course. It should have been build in Perl. Looking at CPAN, we find a lot of modules that would aide in the development of such a thingy ... SDL for the graphics (no no, no browser here please ), handling of XMMS and MP3 files and off course LWP HTML::TreeBuilder and the XML modules for scraping usefull information for TV addicts from the web.

Well, all this led to me developing a thingy that can generate this XMLTV format - theoretically from any source - in Perl (*** note: I put it offline, sometimes it is too dangerous to write perl modules ***). And I've build it as a SAX event generator, so you can plug it in into any application you want (?) if you prefer to work with XML stuff, that is.

I'm also toying around with Perl 5.8.0 recently and especially with the threading capabilities of Perl 5.8.0. I've written the classic Producer/Consumer application. And that got me wondering about the differences between the threading model in Perl and Java. I don't now how, but the low level threading support in Perl seems a bit too low-level for some stuff.
having read a book on Java threading (shame on me), I really would like the following constructs in Perl:

    sub do_stuff : Synchronized {
          Wait;
          $do_stuff_var++;
          NotifyAll;
    }

and ...
    # critical code section
    synchronized {
        do_critical_stuff;
        more_critical_stuff;
    } ...

Let's see what we can do in this area ... more soon.

Thursday October 17, 2002
04:45 PM

Infected by the WWW::* virus ...

I read a post on use.perl.org about WWW::Gazetteer and thought it was a pretty slick idea.

Then I remembered I wrote a Palm app that allows me to grab program listings from a site that hosts tvguides (note: I don't want to speak about this anymore) and store them on my Palm. Off course, the grabbing is done with Perl and the harvester runs in a nice Perl/Tk app.

Anyway ... with all those nice packages in the WWW::* namespaces I felt I also had to contribute. I CPANified the web mining part of channel0 and put it on the CPAN under *** completely gone ****

Aha ... and now my daily job involves java development also, I can say java is a laborous language to develop in. Why do I need to write all the getters and setters? Please give me AUTOLOAD functionality or a Class::MethodMaker equivalent! And what about HERE docs?

Anyway ... a good read why a certain person prefers Perl before java can save the day!

Sunday October 13, 2002
02:27 PM

REST when tired of SOAP and XML-RPC

Webservices ... who needs them anyway? I just received my copy of The New Architect and it had an article about webservices and if they can work economically. Microsoft has put a big bet on web services, elaborating on Dave Winers XML-RPC protocol. A protocol that, nota-bene, has been developed for weblog interoperability. Now, for one thing XML-RPC is easy to understand, not bloated and usefull. SOAP on the other hand is bloated has a Signal To Npoise ratio of almost 0 (S/N = [payload bytes]/[markup bytes] = ratio of payload to markup), but has one big advantage. It has caught on by the big public, and I guess this has something to do with the ease to be able to make use of SOAP.
As a user of a SOAP API, you just have to fire up your IDE and point it to the WSDL file, describing the SOAP API for the web service you are going to use. After that, you can automagically call the API, like:
$soap->cost_of_book($ISBN);
And all this has been tauted in popular IT magazines ...

Anyway, SOAP has a lot of problems and one of them being that it is just not a standard protocol. It is an application protocol layered on top of another application protocol (HTTP).

In the past I've written up web scrapers for a multitude of services, and I was doing a primitive form of web services. Tim O'Reilly looks at this as using the URL as a sort of command line. He coined up the term 'URLline interface' at Emerging technologies keynote speech.

I always have had a guts feeling that SOAP is too bloated. I love XML-RPC, because it is lean and mean and the most important feature for me was the fact that on the server a method was called with the correct parameters filled in and on the client I just invoked a method call that would automagically be resolved on the server side.

Last week I came across a new concept - REST - that stands for REpresentational State Transfer. It describes the standard HTTP protocol as good enough - and even better - to implement web services. It compares HTTP and the URL as something that mimics the openess of the UNIX platform. On UNIX you can pipe output from one app to the input of the other. Building complex stuff with simple application that each perform an atomic unit of work.

Anyway, I'm going to spend some time this week to read up on the ideas of REST. Maybe it will give me a clearer idea if investing time in SOAP is worthwile ...

Johan

Friday October 11, 2002
07:18 AM

Perl is better than java! One reason why.

When developing in Perl, yesterday evening, I noticed something that favours Perl to java. When running my application, I did the following:
perl -I ./lib -MMyApp -emain

This is similar to java:
java -cp ./lib com.stupidcompany.myapp.MyApp
(Calls the magical main method on MyApp class)

So far, so good! When I finish up my application in Perl, I can install it in the 'site_perl' directory of installed modules (using ExtUtils::MakeMaker - cross platform and standard), and then I can do:
perl -MMyApp -emain

Java, on the opposite, does not has a notion of a central class/package repository on a local machine. Instead, java app vendors deliver a long list of .jar files and such that you must include in your CLASSPATH environment variable ... in the end, you have to type up a zillion classpath entries to get a stupid app running. (Everyone who has toyed -what else can you do in java? - with XML in java knows about the classpath hell).
java -cp "~/javalibs/a.jar;b.jar;c.jar;... ad infinitum" com.stupidcompany.magicmain.MagicMain

Perl is good, it provides me with a central repository for my modules!

Regards,
Johan

Wednesday October 02, 2002
04:24 AM

About configurating modules ...

[ #8126 ]
Some thought about the configuration of GSM::SMS ...

I got a very good tip, yesterday, from barries, author of XML::SAX::Machines on how to be able to do module configuration.

On a linux box, most config files are /etc, and this is just fine. When using Perl, you can argue to put a config in /etc or to use a module (.pm file) to store configuration stuff.

Now I settled on using a module for storing the configuration stuff. So now I'm able to do:
perl -MGSM::SMS::Config -esetup
and the command line based configuration manager will pop up. The same code gets invoked when running:
perl Makefile.PL

In a nutshell it works like this:

use Config;
use File::Spec;

...
my $file = File::Spec->catfile( $Config{'installsitelib'}, "GSM", "SMS", "Config", "Default.pm"
);
open F, ">$file";
print F $config;
close F;

The clever thing is using Config to find the install dir and File::Spec to have a platform independent way of creating a path to the pm file.

Ah yes, I don not use Term::ReadLine anymore, but reverted to ExtUtils::MakeMaker::prompt.

Johan

Friday September 27, 2002
11:55 AM

Some lessons in life ...

  • Never speak at the same time Damian Conway has a key note speech at a conference ... (YAPC::Europe::2002)
  • Never use Term::ReadLine in your Makefile.PL script, but use ExtUtils::MakeMaker::prompt
  • It is never too late to start posting on use.perl.org.

I learned these the hardway ... actually it was rather soft and hence nice lessons in life.
Did I mention this is my first use.perl post? Now I did.
I'm going to a makeshift party this evening, organised by my former employer (www.evisor.com), which has been bought by PwC which has been bought by IBM ... and which is now totally defunct ;-)