rcaputo's Journal

Reflex Progress

rcaputo — 2010-04-16T06:04:36+00:00

I've been making good progress on Reflex lately. I have to: I've applied to talk about it at YAPC::NA, so I need something to talk about! The code has been moving too fast for CPAN right now, but you can follow along at Reflex's repository on github. Here's a summary of the latest changes:

I've added support for as many callback mechanisms as I could think of, including one I'm not sure existed before. Callback consumers (the things being called back into) decide the kinds of callbacks they prefer. Objects doing the calling use a single, consistent method: emit(). Reflex handles the rest. The callback types supported so far:

Plain coderefs. Great for the closure addicts among us.
Class and Object methods. The Moose troop may approve.
Role-based methods, which I'll have to explain later.
Condvar- or promise-like wait() polling. This one isn't really a callback. It's here for people who would rather not use them.

There's even an example that runs with every supported callback type at once. Just because it can!

I've also implemented a few classes that support TCP (and maybe UNIX) clients and servers.

Reflex::Stream is a base class for stream I/O. It extends Reflex::Handle, adding sysread(), syswrite() and buffering. Example usage.

Reflex::Listener is a base class for connection-based servers. Give it an IO::Socket, and it will listen() for and accept() client connections. It also extends Reflex::Handle. See TcpEchoServer in this example.

Reflex::Connector is the other side of the connection-based client/server dialog. It will perform a non-blocking connect() on the IO::Socket you provide. Eventually it'll return that socket in its connected state, or notify you of failure. Reflex::Connector also extends Reflex::Handle. Reflex::Client (below) uses it.

Reflex::Client extends Reflex::Connector and integrates Reflex::Stream for a basic but usable non-blocking client class. Here's a small subclass of Reflex::Client.

And Reflex::Collection is a small object manager. The TcpEchoServer uses it to automatically clean up client objects when they disconnect. This is another common and tedious task that Reflex automates.

Still to do: Document and release all this new goodness, then (presuming it's accepted) write my talk! I hope to see you at YAPC::NA either way.

POE 1.280 is Faster

rcaputo — 2009-10-14T05:19:58+00:00

POE 1.280 has just been released, but it may take up to a day before your favorite CPAN mirror catches wind of it. You'll also need the latest POE::Test::Loops, version 1.030.

Those following along may notice that the version numbers jumped a little. That's because POE has undergone a couple optimizations that alter its behavior a little.

I/O Dispatch is Faster

I/O events jump the queue in POE 1.280. Skipping an enqueue and dequeue saves significant time. The drawback is that I/O is dispatched in a different order relative to other kinds of events. Most people won't care, but it has affected one of POE::Component::Client::HTTP's tests. See below for contact information if you think it's affected you.

Session Cleanup is Faster

In POE 1.280, session cleanup is done just before the event loop would wait for new events. Programs that aren't event bound should show decreased latency because the cleanup will be scheduled during idle times. This optimization changes the timing of session cleanup and their resulting _stop events. While I don't think it affects many people, it has already bitten Adam Kennedy. Please let us know if you suspect it's gnawing on you, too.

Staying in Touch

People who need realtime notice of POE changes may watch them at SourceForge, Ohlo and CIA.

POE's community may be found everywhere. Several members tend to accumulate in POE's mailing list and IRC channel at irc.perl.org #poe. You're welcome to join us, even if you don't have problems to report.

Thank you.

Upcoming POE Changes

rcaputo — 2009-10-09T17:40:13+00:00

Those of you who use POE for serious things should be aware of a couple changes introduced after 1.269.

First, here's a public service annoucement from POE: Developers who want to keep up with POE's latest changes are encouraged to follow its repository. You'll get instant notification of changes as often as you'd like, and you'll have maximum lead time to report new bugs or port production code to new features. I've also released Version 1.269_002 to the CPAN for developers who are more comfortable with that. Remember: CPAN shells don't install developer releases by default. And now back to the changes!

Synchronous I/O Dispatch

POE::Kernel dispatches I/O events synchronously after 1.269. Previously these events would be enqueued along with everything else, only to be dequeued and dispatched as soon as possible. Now they're dispatched as they happen, saving the overhead of a round trip through the event queue. Developers should notice an improvement in I/O bound programs.

This naturally changes the timing of I/O events. Where in the past they may have been enqueued behind other events, they now come first. This could cause problems for users who depend on the relative timing of I/O events compared to other kinds of events. I expect most developers won't notice the difference.

This optimization is most effective for code that uses POE::Kernel's select_read() and select_write() watchers directly. Additional layers of abstraction, such as POE::Component::Server::TCP or AnyEvent add overhead that has not been reduced by this optimization.

Mark and Sweep Session Garbage Collection

Session termination and memory reclamation have been combined and delayed. Combining them reduces the overhead of POE::Kernel's dispatcher by eliminating a few method calls per event. Delaying garbage collection allows POE to do this work when it's likely to be idle anyway. While the latter optimization doesn't reduce CPU consumption, it should reduce callback latency in the common case.

People who require _stop to be called immediately when a session releases its last resource may be disappointed. _stop events are dispatched as a side effect of garbage collection, which as I've mentioned has been delayed. Adam Kennedy's Test::POE::Stopping has been affected. I hope his is a special case.

More to Come

There will be more improvements in the future, but that's all for now. Stay tuned, and please report any problems you encounter early and often. Thanks!

POE Supports Other Event Loops, Really

rcaputo — 2009-09-11T05:57:26+00:00

Lately I've been hearing, repeatedly and loudly, that POE doesn't work with other event loops. I wrote the first bridge between POE and another event loop in May of 2000, so this is rather shocking news to me. I've had to rerun "make test" just to be sure I wasn't dreaming! :)

Here is a slightly edited repost of my reply to Tatsuhiko Miyagawa, who writes in regards to his YAPC::Asia presentation:

Rocco, the author of POE pointed out that slides 19-21 are misleading since POE also has the POE::Loop concept so that you can run your event loop under different backends like EV or Gtk. While that is true (and that is great), POE still can't run with the other main loop, like IO::Async::Loop->loop_forever. You still need to call POE::Kernel->run and to do so you should rewrite your main program to use POE everywhere, since existent IO::Async based program can't run any of POE::Component, which is totally possible with AnyEvent and AnyEvent::* modules.

I respectfully disagree, and I have code to back me up.

POE runs with other main loops. In fact, POE::Kernel->run() is often implemented in terms of native event dispatchers. POE::Loop classes implement event watchers using native watchers, and native callbacks are used to generate POE events. Your call to $kernel->select_read() becomes Event->io(), Gtk::Gdk->input_add(), etc.

Depending on the POE::Loop implementation, POE::Kernel->run() need not be called upon to dispatch events. This is CPAN, so your mileage may vary. If you're interested in improving CPAN, submit bug reports to the appropriate queues.

I mentioned I had code. Here's a working example of POE code and native Glib code running together in the same program, using Glib's MainLoop:

use Glib; # Install POE::Loop::Glib. use POE; POE::Session->create( inline_states => { _start => sub { $_[KERNEL]->delay(tick => 1) }, tick => sub { print "poe tick at ", time(), "...\n"; $_[KERNEL]->delay(tick => 1); }, }, ); my $glib_timer = Glib::Timeout->add( 500, sub { print "glib tick at ", time(), "...\n"; return 1; } ); # Dispatches Glib and POE events. Glib::MainLoop->new->run();

Of course you can call POE::Kernel->run() instead of Glib's main loop. This is a good thing. A single, portable run() method covers every supported event loop.

Addendum:

Things get really interesting when you realize that POE can be loaded into event loops that are already running. For example, Martijn van Beers and Chris Williams have been running POE inside irssi for years. Chris has even embedded an entire POE-based IRC server in irssi, which got a little mind-bendy when he connected to irssi from itself. Both cases work without rewriting irssi's main program.

Help me rename Lexical::Persistence

rcaputo — 2009-09-06T06:51:42+00:00

"Persistence" implies that data will be saved and loaded from disk, to a database, or some other persistent storage. That's not what the module does, so I think it needs a new name.

Lexical::Persistence objects are closures that make the values of lexical variables stick (or persist) in everything they call. In the following example, the Lexical::Persistence object makes sure that getter()'s version of $x contains the value stored by setter().

use Lexical::Persistence; my $lp = Lexical::Persistence->new(); $lp->call(\&setter); $lp->call(\&getter); sub setter { my $x = "some value" } sub getter { print my $x, "\n" }

Lexical::Persistence's closures are objects. Programs may examine and manipulate their contents between calls. The following code displays something like this:

My mind is going. I can feel it. My is going. I can feel it. My is going. I feel it. My going. I feel it. My going. I feel My I feel My I My

Here's the example:

use Lexical::Persistence; my $lp = Lexical::Persistence->new(); $lp->set_context( _ => { '@mind' => [qw(My mind is going. I can feel it.)] } ); while (1) { $lp->call(\&display); my $mind = $lp->get_context("_")->{'@mind'}; splice @$mind, rand(@$mind), 1; last unless @$mind; } sub display { my @mind; print "@mind\n"; }

So, what would be a better name for this module?

Trying Out this Git Thing

rcaputo — 2009-08-08T21:38:29+00:00

Distributing Development

Distributed version control is great. It'll bring on the patches, establish world peace, and get you laid. It adds extra steps to your workflow, but they make everything easier. Don't ask me how. Even if I understood that trick, the Magician's Code wouldn't let me explain it to you.

There's more than one way to distribute development, and many of them are free. Bazaar stood out as having the best mix of flexibility, extensibility and usability. It also has Launchpad, which provides frameworks for distributed project management and feature design.

While Bazaar looks nicer, doing DVCS right means going where the people are. For Perl, that's Git. Sorry, Bazaar, it's not you, it's me.

Getting my Feet Wet

POE is a perfect project to migrate to a distributed development model. It's large, it's popular, and it can always benefit from more developers. I mean, what can't, right?

POE's size and visibility also make it a bad choice for a first foray into distributed development. New software solutions, like new cleaning solutions, are best tried somewhere that won't show if they stain.

POE's scratch monkey is POE::Stage, a much smaller, much less visible project that, needless to say, can benefit from more developers. I recently migrated it from Subversion using Yuval Kogman's tools and advice. The popular svn2git alternative didn't work well for me.

Decisions, Decisions

There's more than one way to host a Git project. GitHub and Gitorious have been recommended for different reasons, but neither is a clear winner. Fortunately, a distributed repository can be hosted in more than one place. POE::Stage is available at both GitHub and Gitorious until I decide.

What Next?!

So I'm all set up. I've successfully migrated a project to Git, hosted it publicly (twice) and even awkwardly pushed a few changes.

The journey's only beginning, though. I have plenty of small projects to migrate while I commit the DVCS dance to muscle memory. I also have some specific Achievements to acquire before tackling the Boss Level, migrating POE itself.

It should be fun, or at least a good challenge. I'll keep you posted.

POE Changes Coming Soon

rcaputo — 2009-07-24T04:30:21+00:00

POE::Test::Loops 1.020 will be released tonight. This is a suite of reusable tests for developers who want to add external event loop support to POE. Version 1.020 acquires more signal handler tests from POE. Existing POE::Loop authors should try the new tests at their earliest convenience. Those who need the code right away can track the repository:

svn co \ https://poe.svn.sourceforge.net/svnroot/poe/trunk/poe-test-loops ptl

A developer version of POE will be released shortly after the new tests work their way around CPAN. The upcoming POE release incorporates Philip Gwyn's fixes for race conditions in POE's signal handlers. Users who have noticed strange behavior during prolonged periods of signal activity should test against this release as soon as possible. Those who want to get a jump on the changes can track POE's latest development through its repository:

svn co \ https://poe.svn.sourceforge.net/svnroot/poe/trunk/poe poe

POE changes are also tracked at SourceForge, Ohlo and CIA.

POE 1.007 will officially go out after a successful round of testing. You're welcome to join us in irc.perl.org #poe to watch the CPAN tester fallout.

POE 1.008 will be released shortly afterward. It will spin off some optional dependencies, mainly POE's "core" support for optional, external event loops. POE::Loop::Event, POE::Loop::Gtk and POE::Loop::Tk will join their siblings on the CPAN. Their test coverage should improve, and POE's distribution size and installation footprint will shrink.

Thank you.

Test post, please ignore.

rcaputo — 2009-07-13T06:17:14+00:00

BinGOs pointed out WWW::UsePerl::Journal and kindly pasted the code he uses to drive it. I've hacked it a little to embed the article title into a header block on the article. Since I'm writing these offline, I figure the title should be associated with the article for posterity.

Here's my hacked version of BinGOs' posting script:

#!/usr/bin/env perl use warnings; use strict; use WWW::UsePerl::Journal; my $user = 'rcaputo'; my $pass = '********'; # Headers. my $title; my $comments = 1; while (<>) { chomp; last if /^\s*$/; unless (/^\s*([^:]+?)\s*:\s*(\S.*?)\s*$/) { die "Unknown header line ($_)\n"; } my ($header, $value) = (lc($1), $2); # Avoid the lulz. $value =~ s/\Q$pass/********/g; if ($header eq "title") { $title = $value; next; } if ($header eq "comments") { if (lc($value) eq "yes") { $comments = 1; } elsif (lc($value) eq "no") { $comments = 0; } else { die "Comments header may only be yes or no.\n"; } } } # Body HTML. my $text = ''; while (<>) { # Only one per line. if (/^\s*<ecode\s*file="([^"]+)"\s*\/*\s*>\s*$/) { my $code = `/bin/cat $1`; $code =~ s/\n+$//; # Break up the ecode tags so this script can # present itself. $_ = "<" . "ecode>\n$code\n<" . "/ecode>"; } $text .= $_; } # Avoid the lulz. $text =~ s/\Q$pass/********/g; die "Article needs a Title header.\n" unless $title; die "Article needs body HTML.\n" unless $text; # Post. my $journal = WWW::UsePerl::Journal->new($user); my $post = $journal->login($pass); unless ( $post->postentry( title => $title, text => $text, topic => 34, # user journal comments => $comments, # allow comments? type => 2, # ??? promote => 'publish', # ??? ) ) { die "Failure.\n"; } print "Success.\n"; exit;

Here's the source of this article:

Title: Test post, please ignore. BinGOs pointed out WWW::UsePerl::Journal and kindly pasted the code he uses to drive it. I've hacked it a little to embed the article title into a header block on the article. Since I'm writing these offline, I figure the title should be associated with the article for posterity. Here's my hacked version of BinGOs' posting script: <ecode file="~/bin/useperlpost.pl"/> Here's the source of this article: <ecode file="~/Documents/useperl/testing-www-useperl-journal.html"/> Thanks for reading anyway.

Thanks for reading anyway.