Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

oliver (7451)

oliver
  (email not shown publicly)
http://users.ox.ac.uk/~oliver/

Data network admin for the University of Oxford. PAUSE ID "OLIVER".

Journal of oliver (7451)

Monday August 17, 2009
09:14 AM

Whither SourceForge?

Every day, it seems, another Perl module is posted up to GitHub. Collaboration, of course - great stuff!

But it got me thinking about why everyone is settling on GitHub. Surely not just the fact it's all web two point oh, combined with groupthink. What happened to poor old SourceForge, the previously canonical location for open source software on the web?

I wanted a bit more infrastructure for one of my own CPAN hosted modules, so went to look at GitHub. The first thing which struck me was the lack of mail list support, so in fact GitHub turned out to be a non-starter.

Having not visited SourceForge in a while, I found the site has had quite a nice front-end facelift, along with a bunch of new features.

Yes, there is actually git support (and bazaar, svn, mercurial, cvs). There's also a wiki, forums, image gallery, issue tracker, blog, and much more, if you want.

So I was left feeling a little underwhelmed by GitHub, but SourceForge at first glance seems to offer all the infrastructure I need instead.

Any comments on what I have missed? Would anyone who consciously chose GitHub over SourceForge like to comment?

Sunday November 30, 2008
06:57 AM

On presenting at the LPW08

Before too much of this slips from my mind, I want to jot down some notes about my experience this year as a speaker at the London Perl Workshop 2008. The talk I delivered was about instant web front-ends to databases (i.e. scaffolding and CRUD) and in particular my own module CatalystX::ListFramework::Builder.

Please be aware that none of the following should be seen as criticism (of the organisers, audience, etc) in any way! How I felt was, and is, my responsibility alone. I'm very grateful to have had the opportunity to speak at LPW. Also, I'm a fairly experienced public speaker; one other reason for writing this is that I feel I didn't perform particularly well on this occasion, and want notes I can refer back to (and that might help others, by being public).

I had a lot of trouble preparing this talk, I think because this is my second most popular module, and yet it's a subject matter about which I am really no expert. Lots of other people know much more about JavaScript, databases, DBIx::Class, Catalyst and Template::Toolkit than I. I didn't know how to pitch the talk - beginner, intermediate or advanced? There was easy scope for all three.

The timing of 20 minutes, in hindsight, was a mistake and I should have opted for a 40 slot instead. I had just maybe four slides too many and that meant I had to talk too quickly and perhaps not explain certain key concepts. There was not any time for Q&A and again looking back, this module is one of those where people are likely to want to ask odd little questions (it's more of a system, than a module).

When I arrived in the morning, I found some spare time and went up to look at my allocated room (lecture room 3). Unlike the other two rooms which had standard tiered seating, this was a classroom, with maybe 25 seats, and tables. This seemed okay because it would be more cosy and good for discussion. Unfortunately about 40 people seemed to turn up, which meant it was standing room only, really hot, and very daunting for me (I'll come back to this in a minute). The number of people certainly exacerbated my trouble over the pitch of the talk - with less people I could ask questions at the start and then tailor my speaking to point out features at the general level of the audience. In this case I had no such opportunity.

Now, everyone knows that London has its fair share of leading lights of Perl, and the LPW also attracts visitors from around the world as one of the best one day conferences. Maybe not many people knew who I was, but I certainly knew them, and this made me uneasy. This isn't to do with personalities, or being star-struck; I'm talking about experience. The very reason for me writing this module was to get to try JavaScript for the first time, and yet people here probably knew more than I did. Is that a reason not to give a talk at LPW? No! Normally as a speaker you can deal with one or two experts in the room, but I had a dozen, so my poor brain was working overtime on watching what I was saying, which didn't help me to focus on the delivery.

One thing I missed about LPW was a system to get feedback from the audience about how they felt the talk went. I love to get this stuff, the good but especially the bad. Sometimes things can't be helped and the attendee perhaps misunderstood the theme of the talk, but other times I just hadn't realised people like that existed, and next time I will be able to take their thoughts into account. I cannot stress enough that as a public speaker, getting honest, good and blunt feedback from attendees is really, really useful. So I'm particularly saddened that by the end of my talk I was hot, flustered, a little overwhelmed by the audience, and totally forgot to ask for feedback.

Also frustrating was that this year LPW just happened to clash with an important evening meal out in London for me, meaning I couldn't come to the after-show drinks. Perhaps that would have been a good opportunity to solicit feedback, or at least get drunk and forget all the stuff I've vented in this article.

My final point to note is that I do think I tried to cover far too much in the talk. For 20 minutes, an end-to-end life-cycle of CRUD is just way too much to cover. I should definitely have skipped some areas and focussed on others (e.g. the various ways to use the module - just pick one and let the audience RTFM to find the rest). More diagrams showing workflow, and how data moves around the system would have helped; the slides were too text heavy. These are side-effects, I think, of the pitch issue mentioned above. I wasn't sure what to speak about, and what not to speak about (which is equally as important!).

I'm really, really glad to have had the opportunity to speak at LPW. And I would do it again without second thought. I could also probably manage a talk at a London.pm tech meet, now that I have the cojones to speak in front of them. But yesterday wasn't my finest speaking hour; I'm sad that was the case. Apart from all the irrational thoughts such as letting people down, bad first impressions, etc, I let myself down. The pitch, and the venue, were both awkward, and I think that's the note to take away from this - better prep required to deal with these two gotchas.

Thanks for listening, if you made it this far! Overall, please know that I had a very positive experience at LPW08 - these are just notes to help make that even better, next time around. If you were at the talk, please do drop any feedback to <oliver.gorwits@oucs.ox.ac.uk>, or comment below. Good or bad, I don't mind, it all makes me happy.

Wednesday August 06, 2008
12:01 PM

DBIx::Class, Catalyst, CRUD, AJAX (and other buzzwords)

I teach a Perl course about once a year for folks in my organization, and one question the students always ask is, "I need a quick and simple web interface to this database we have, what should I use?" And there was no answer (until now, I suppose).

Okay, there are a few things which do a similar job in Perl; a couple of abandonware Catalyst applications, or frameworks which require you to write templates, but what I mean is the whole shebang - Database to ORM to Templates to Web Page, all from one installation. Closest thing I know of is Rose::DBx::Garden::Catalyst, but even that seemed to require too much effort for a lazybones like me.

So about a year ago a chap called Andy Payne who was working at Dragonstaff produced the ListBuilder, a framework using Catalyst, DBIx::Class, Template::Toolkit, and the ExtJS Javascript library to churn out nice looking functionable CRUD interfaces to database tables. I spent a fun evening hacking with it 'til around 3 in the morning.

My eyes were opened to the real possibility, that the whole interface should be generated from information which DBIx::Class already knows; there's no need for us to be explaining things twice. I booked a few days off work and we now have CatalystX::ListFramework::Builder (LFB).

LFB needs only a few lines of configuration (database connection parameters), and then also needs you to have DBIx::Class schema classes available which describe your database, but those can be automagically generated by the excellent DBIx::Class::Schema::Loader module.

Out the other end of the sausage machine comes a Javascript-rich list-based, paged view of a database table, supporting search, delete, create and edit. LFB handles one-to-one and one-to-many relations, showing related data and allowing creation and linking of related rows when editing.

If you like the look of the module and want to know something, please do drop me a line; equally if there's a missing feature you desperately need we can probably arrange to have you buy some of my time. In the rest of this article I'll explain a little about how LFB works.

Your complete Catalyst application looks like this:

package ListFrameworkUser;
use Catalyst qw(ConfigLoader +CatalystX::ListFramework::Builder);

__PACKAGE__->setup;
1;

You can take my word for it that the config really is trivial (an example is on the CPAN documentation page), with just database connection parameters and a URL path for the Javascript library. The base class used creates a Catalyst application on the fly in your application's namespace (ListFrameworkBuilder, here) which has all the Models, Views and Controllers needed to run LFB. It does this using a mixture of package-scoped eval(), %INC injection, and overriding a little of Catalyst's core (setup_components in particular, because of eval() vs. Module::Pluggable timing issues).

Once that is done, a begin() handler in Catalyst instructs a Model to introspect DBIx::Class for all the table, column, data type and relation information which it pops into the Catalyst application stash. This is used by Template::Toolkit templates to customize a truck-load of ExtJS Javascript which is sent to the browser as the main page, and which in turn draws the list-based table view.

From that, everything notches up a gear into Web 2.0 land, as the user interface conducts its business over JSON-based AJAX. As mentioned above, you can add, delete, edit and search for records in the paged list-based view, and there are a few more bells and whistles besides that.

What are the current limitations? Well, I have coded along the 80/20 principle so there are some missing features, and some parts are inefficient. For example only single column primary keys are supported, although perhaps this will be improved in the future. Also, I didn't want you to have to install much more besides LFB itself and the ExtJS 2.1 library, so a small number of static PNG icons are served from LFB rather than your web server. These are reasonable compromises given the aim of LFB which is to be a near-zero-conf quick-install web GUI for (probably) your legacy databases.

We're up to release 0.15 as I write this, and most of the recent work has been refactoring, and there is still a long way to go. Certainly, some of the logic used for creating related rows and so on in the database is sub-optimal (but it does support forward and reverse relations - belongs_to, might_have, has_one and has_many, in DBIx::Class parlance). Don't be too critical, but feedback and ideas (and hey, if you like it, some praise!) would be appreciated. I'm already working with a handful of early adopters to refine the UI.

Oh, and last but not least... you want to see a running demo of the system? Well as a reward for reading this whole article try this simple Albums application, at least until the use.perl.org effect brings down my friend Peter's little VM host :-)

Saturday March 01, 2008
01:53 PM

Google Sites

Google Sites has been released. It allows you very easily to create web pages, and of course they come out looking pretty stylish.

For me, a web-luddite, this is great. I had some horrific gawdy old thing before, which was an embarrassment preserved for eternity via the Internet Web Archive. I just cut and pasted the content into my new Google Site - a few minutes' work and I have a new homepage :-)

The system is very much designed for collaborative work; I suppose it's a version of Google Page Creator with support for hierarchical structure and so on. You can edit in WYSIWYG or HTML mode, and easily import HTML from other pages. Perhaps this is a challenger to having a wiki for a Perl project?

In related news, I'm back to using Firefox for browsing under OS X (from OmniWeb, and Shiira). FF is plenty fast enough, and I just couldn't live without the awesome extension system - particularly del.icio.us, Firefox Showcase, It's All Text!, SwitchProxy, and Tab Mix Plus.

Sunday November 04, 2007
08:51 AM

Moose quick-ref card

I recently had a few spare tuits and ran out of books to read, so the Moose object system popped to the top of my stack.

Like many new and cool projects, features are accellerating away from documentation, and often what a new starter wants is a low-down, which isn't there (yet). Catalyst and DBIx::Class were each like this at one point in time.

To help myself and others I have created a PDF Moose Quick-Ref Card with most of the functions and syntactic sugar consructs of the system.

Thanks go to folks in the #moose IRC channel, and the Programming With Moose WikiBook written by EvanCarroll. The ref card will work best if you can print it 2-up on one A4 sheet, cut that in half, then laminate the two A5 parts back-to-back.

Comments and suggestions for the ref-card will be gratefully received :-)

Sunday October 28, 2007
07:19 AM

Shiira the Mac web browser

On the Mac (of which I have two, so you could say I'm a regular and keen user), I don't bother with Safari but instead use OmniWeb, a non-free third party browser. It's packed with features to help juggle dozens of open pages at a time, and other things besides, so is a good power alternative to Safari.

OmniWeb is generally very stable (or at least has got a lot better in recent months), although painfully slow to start. That's its most brain-dead feature (a side effect of the workspaces feature). What I really do like is the easy management of popup and cookie preferences per page/window/site, and the tab thumbnails.

It looks like there may be a new contender for OmniWeb's title as the power browser for mac - Shiira. This is based on the Apple Web Kit so is basically like Safari 3 inside, but with added bells and whisles. This one's Free and free, and with the recent release of version 2, looks like something to keep an eye on. I'm submitting this post from Shiira.

Wednesday May 16, 2007
02:46 PM

a truth about RoR

I was pottering around Wikipedia as you do, and stumbled across a link to an interview with a Twitter (social network website) dev. I'm pragmatic about programming language choices, but I do hate hype. Here's the URL and a juicy quote which seems to cut through some of that:

Interview with Twitter Developer Alex Payne

"By various metrics Twitter is the biggest Rails site on the net right now. Running on Rails has forced us to deal with scaling issues - issues that any growing site eventually contends with - far sooner than I think we would on another framework.

"The common wisdom in the Rails community at this time is that scaling Rails is a matter of cost: just throw more CPUs at it. The problem is that more instances of Rails (running as part of a Mongrel cluster, in our case) means more requests to your database. At this point in time there’s no facility in Rails to talk to more than one database at a time. The solutions to this are caching the hell out of everything and setting up multiple read-only slave databases, neither of which are quick fixes to implement. So it’s not just cost, it’s time, and time is that much more precious when people can[’t] reach your site.

"None of these scaling approaches are as fun and easy as developing for Rails. All the convenience methods and syntactical sugar that makes Rails such a pleasure for coders ends up being absolutely punishing, performance-wise. Once you hit a certain threshold of traffic, either you need to strip out all the costly neat stuff that Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move the slow parts of your application out of Rails, or both.

"It’s also worth mentioning that there shouldn’t be doubt in anybody’s mind at this point that Ruby itself is slow. It’s great that people are hard at work on faster implementations of the language, but right now, it’s tough. If you’re looking to deploy a big web application and you’re language-agnostic, realize that the same operation in Ruby will take less time in Python. All of us working on Twitter are big Ruby fans, but I think it’s worth being frank that this isn’t one of those relativistic language issues. Ruby is slow."

Sunday December 31, 2006
08:03 AM

hacking # on mac keyboards

A very nice new toy recently landed on my desk at work, an Apple MacBook. It's my first Mac and I'm very impressed so far. One of the things I want to use it for is hacking Perl whilst on the move.

To get a # sign on the MacBook keyboard you use the Option (or Alt) key + 3. This seems terribly klunky to me, and # is of course used quite a bit in Perl. The other problem is that you might have the Alt key mapped to something else in a Terminal application.

This hack remaps another key on the keyboard to produce the # character. I chose the funny squiggle that's to the left of the number 1 key (§). This is the Section sign, used in document formatting. Just create a file at ~/Library/KeyBindings/DefaultKeyBinding.dict that contains the following:

{
    /* this will make all &#167; turn into # */
    "\UA7" = ("insertText:", "#");
}

Any app that uses Apple's Cocoa interface widgets for text input will pick this up after being restarted. There are some that don't (perhaps TextMate? Not checked that one so if you know, please comment).

A lot more information about this is available at this excellent page on the Cocoa Text System, including some other neat hacks. Enjoy!

Wednesday December 13, 2006
06:43 PM

LPW and some DBIx::Class

I'm quite surprised that I made it to the London Perl Workshop after quite a heavy night out with some friends in Oxford. Well, I did feel like death warmed up for most of the day, but still managed to enjoy the sessions through the hangover. I think that's praise!

Mostly I was in the Advanced track, and it was a delightful mix of the real-world and abstract, relevant and irreverent, scary and familiar :-) Jonathan Worthington made a fine start to the day with an excellent overview of some OO goodies in Perl 6. My day-job keeps me pretty busy with Perl 5 so I've not had a whiff of v6 pass my nose yet, and this was a superb talk at just the right level for me, with a nice dash of humour to boot. Next up was Dave Hodgkinson with some mod_perl scaling tips and tricks, so by this time I was in little puppy heaven! He brought some real honesty to the presentation which was very welcome and the chit-chat in the room after his short talk only added to the benefit.

The next two in the track, benchmarking gotchas by Abigail and Perl 5.10 regex by demerphq were equally good. I liked that Abigail made each slide a wee puzzle that made the audience race to find the logic error hidden within. Sorry to hear that Tom from miltonkeynes.pm couldn't make it to speak; hope he's feeling better now. demerphq and his clan clearly have some industrial scale cojones to tackle the regex code, but the results look impressive, especially exciting for us that run large Spamassassin installations. He also did all the work on his own time, full-time, courtesy of his previous redundancy package, so we owe our thanks for that generosity.

Jesse Vincent spoke after lunch about some scary code in the products of Best Practical. No real surprise there, as I've seen what RT does to a database, and it ain't pretty (DBIx::SearchBuilder, ugh!). Didn't really inspire me to install any other BP software, to be honest, after seeing his perversions (impressive and fun though they were) of Perl bugs. Finally I went to Alistair McGlinchy's "lightning" talk on network bandwidth. Interesting if not educating for me, listening to him describe his day-to-day job was quite entertaining. I then made a sharp exit for the next train home, reckoning I couldn't last a night out with London.pm on so little sleep, and I'd seen Jos Boumans's (excellent) talk before at YAPC::EU::2006, anyway.

In other Perlish news this week, I finally found the time to check out DBIx::Class. About a year and a half ago, whilst I was busily developing work's wireless guest service (think coffee-shop style thing), I wanted a good ORM to go with Catalyst. Matt Trout pm'd me a snippet of code from this new project he was working on, which looked pretty funky, but was way too raw to put into producton. I settled on Dave Rolsky's Alzabo, which isn't that scary, and has done sterling work since then. Aside: Dave Cross was a fool to miss Alzabo out of his round-up of ORM software when he layed into the topic in 2005.

Well, I came back to Matt's finished baby this week, and it's pretty damned cool. In fact, it's just about the same as Alzabo. By which I mean that if you sat down and thought about what you want an ORM to provide, which Dave R. did and Matt did, you'd not really be able to come up with much else. What differs between them, really, is that Matt's software is very much "of the moment" in terms of style and coding practice whereas Alzabo, even though it's fully OO of course, smells a bit "2004" to be honest. Alzabo isn't crap by any stretch, it's just that I'll be happier coding with DBIx::Class in the future because it better fits the way I happen to write code, and the way the authors of modules I use do as well.

Friday December 08, 2006
11:19 AM

MythTV tinkering: always a bad idea.

The REST stuff has had to go on the backburner for a while, because I wrote down what needed to be done to start that, and it's a long list!

At work I've been setting up a new PostgreSQL server, and migrating a few of our systems to use DB schema, which has been interesting. For the uninitiated, a schema is a layer between the table and the database that allows logical grouping of tables. You can have one schema per app, say, and allow referential integrity between schema, whereas before you wouldn't have been able to directly refer to relations in another app's database.

I tend to avoid posting in journals when computers are giving me a hard time, and boy, this past weekend, it was just like that! I decided to re-image my MythTV system, for two ends: One was to change from RAID0 to RAID1, and the other was to upgrade Linux distro from FC4 to Ubuntu Edgy.

To be honest, this stuff is always a PITA. Myth is the kind of app that prefers to be run as a black box - once the thing's up and running just don't ever touch it again, and you'll be fine. Try to fiddle here and there, or update the odd package, and you'll be done for. So I should have expected the worst with a massive change like this, you're right. My big problem is that I have this great TV card called a Hauppauge PVR-350. I love it. It does MPEG2 encoding and decoding in hardware, and has the best quality TV-out of any graphics card I've ever seen. Sadly the Linux drivers are nowadays pretty much abandonware, and even the MythTV project has dropped official support. I can agree with their reasons, whilst still being upset, mind. I spent a frustrating day and a half pissing around with kernel modules and parameters, rebooting, and generally turning the air blue, before calling time on the card after three happy years.

Luckily for me, work had just upgraded my workstation graphics card and I had an Nvidia Geforce4 MX4000 sitting in the drawer. It's passively cooled, making it excellent for this application, and of course the binary drivers are nice and stable. It also has the TV-out I need to replace the Hauppauge card. I can report an immediate and noticeable loss in TV picture quality, but digital terrestrial is good enough to reveal that I suppose. The picture is just a little grainy, and washed out, but it's still leagues better than analogue. Having said that, SWMBO tends not to notice when Myth chooses to record from analogue rather than digital anyway, so we're hardly quality snobs!

Apart from that headache, the system's running like a dream under Edgy, and in fact I've implemented a few nice hacks that weren't in the old FC4 setup. I also use that box as a general GNU/Linux hacking system, so the wonderful, and beer-free, NX Server got installed. I can highly recommend it if you need remote desktop on a Linux box, but have (or want) SSH as your only means of remote access. Being a Kerberos fanboy, the system also gets setup to authenticate to the OX.AC.UK realm, so remote access into work becomes straightforward.

One last thing before I sign off. I'm looking forward to the London Perl Workshop tomorrow. I didn't manage to arrange to go in time for the last one, and it looks to be a nice lineup this year. I'll also bump into some new friends from MiltonKeynes.pm, and some old friends from London, I hope, making it a nice day out. See you there, maybe!