Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

xsawyerx (8978)

xsawyerx
  (email not shown publicly)

Journal of xsawyerx (8978)

Wednesday February 03, 2010
06:39 AM

On Nagios, Thunk, Shinken and wrapper included marketing

original post can be found on my blog.

Nagios is probably the most famous and used monitoring program on the market. It's free, GPL and has nice features such as object representation of data, inheritance, plugin systems, passive testing, built-in Perl interpreter, result caching, pipe interface, alert delegations and so on and so on.

The web interface of Nagios is, however, incredible ugly. It's written in CGI the way the early CGI scripts were written. When you make a change to a server via the web interface, you get a few screens (avoiding Javascript is a benefit for some cell phones, I guess) and the old and quickly-annoying "You have done the action you wanted, please click this back link we created to go backwards" screen. You won't simply get automatically directed back to where you were before with a new message at the top in bright green saying "Action X done" or something like that. That would be too easy and Web 2.0. It uses frames (yuck!) to show the sidebar, you don't see the content of comments in hover, only when you click on the comment to get to the comment screen to view the comment. It's literally a pitfall and at least one company where I worked at rewrote the entire interface in ASP and .NET (I know, I know...) by parsing the Nagios log.

For that reason, it has always been a bit difficult (though possible) selling Nagios to the enterprise when your boss isn't tech savvy, and other programs, not much better or worse (OpenNMS for instance) find their way since they have a much better user interface.

A new fork of Nagios has begun with a PHP interface, calling Icinga. The point is to accept a lot of patches that were difficult to get into Nagios (that has only one actual developer - Ethan Galstad), and provide a beautiful web interface with Javascript. At least one Nagios community members sees the entire fork's point is the web interface, and assumes there's a good chance it will be merged back into Nagios, keeping the core as it is.

Apparently, there is a Perl Catalyst-based project to revamp the Nagios web interface, called Thunk. It is available on Github and also has a demo page. It's incredibly fast and seems promising. However, there is no website for it. Currently the only face it has is the Github page which seems generic and perhaps non-welcoming for some people who consider using it. You can also view the demo, but it still has the basic dull look of Nagios' oldschool interface.

Another new project relating to Nagios is Shinken, which is a Python rewrite of Nagios. Apparently someone thought that Nagios is great, except it's written in C, which makes it a barrier to include other peoples' work. I personally disagree for a variety of reasons which I won't go into. What does seem interesting is that Shinken is very welcoming, even though (at least to me) it seems like an exercise in futility. I'm assuming it will rapidly develop a stable core userbase, and the website is to thank, IMHO.

One more issue to note: Ethan Galstad is now developing Nagios XI, an enterprise solution. The "solution" boasts a new web interface, which I suspect will be the leading selling point of it.

So:

  • Nagios has a terrible interface.
  • Nagios XI has a pretty interface (and a free iPod Touch with every purchase, according to the site).
  • Icinga say they will put more patches in that were rejected or ignored for Nagios (but will keep core compatibility). However the strong point of Icinga (and where most efforts are now going to) is the web interface.
  • Thunk attacks the interface problem directly, but doesn't seem it will be adopted much because it doesn't even have a face.
  • Shinken tries to replace Nagios by rewriting the core to be in Python, but the best reason to adopt it (as with OpenNMS) is the web interface.

Now, of course there's a difference in Python, Perl, C and all that. However, in marketing, the selling value goes to the more presentable. In a System Monitoring conference I would have a pretty difficult time selling anything with just a Github page. It all boils down (in marketing terms) to the interface.

Tuesday February 02, 2010
08:49 AM

Marketing the Entire Box (including the wrapper)

this was originally posted on my new blogs.perl.org journal, which can be found here.

For a long while now I've been wondering about some observations I've made of Perl, Ruby, Python and PHP in marketing terms. I'm going to discuss them here in detail, and I hope it will gain us some insight into better marketing understanding or at least not bore anyone.

I've understood through the years that projects with beautiful websites have a better chance of getting picked up by users (even when the project itself is purely command-line) and definitely gives much more credit to the encompassing layer (like the programming language itself).

I've also noticed that a rather large portion of Ruby-related projects have very beautiful websites. You might have noticed this yourself as well. I tried to understand how this situation occurred and had a few discussions at $work with people whose opinion I appreciate, one being a Python and PHP programmer (or, as I like to see him, a programmer who knows PHP) and the other being a Ruby, RoR and overall Javascript whiz.

At first, my thoughts were that most Ruby programmers major in Rails, so they're into web, while the major web frameworks in Perl (Jifty, Catalyst, Mojo, Mojolicious, Mojolicious::Lite, Dancer, CGI::Application, etc.) only flowered in recent years, which could lend the thought that most Perl programmers aren't web-oriented or web-aware. I mean this obviously as "aware" in higher extent than most users. As mst eloquently expressed, we're still into IRC (me included, of course).

Quickly that theory was squashed when I entered the Django website. For a web framework, it is horrible. Especially for a web framework that speaks of cleanliness. The website design is horrendous. Same goes for the HAML website. PHP as a very successful web-oriented language shows interesting results. Most PHP programs/scripts come in various index sites (much like oldschool PERL scripts), few PHP frameworks have websites and fewer have beautiful websites. Most of them are awful. Frontpage awful.

So apparently, dealing with web doesn't mean you create beautiful websites. Yeah, I guess anyone could say that, but to put it in better words: knowing web doesn't mean you do web. So what is the reason it is almost given that a Ruby project (as small as it might be) will have a website, and it will be beautiful?

A thought that formed rapidly through the conversations was that Ruby programmers see the marketing as relating to not just the product, but its wrapper. That is, that many Ruby programmers understand at a very core level (more than most programmers - at least me) that the website which shows the project is the actual wrapper of the project and is just as important, if not more so. When I thought of a "nice wrapper", I just thought of a detailed POD and --help output. Obviously, this isn't the case for many Ruby programmers. Those Ruby programmers think "I cannot ship this application without wrapping it nicely, it is just incomplete."

I began to see the major difference between many Perl programmers and many Ruby programmers. I haven't ever written a website for a project I worked on. Evidently, perhaps many of them don't merit a website of their own, but perhaps some of them do! One of the prime examples for this, IMHO, is LessCSS which has a beautiful website for a rather simple filter, which could definitely be accomplished just as well in other languages such as Perl. I thought how simple it would be to write a version in Perl, stepped up to CPAN and here is CSS::LESSp and Apache2::Filter::LESS::CSS. I have to say that CPAN is indeed comfortable for me, but as a design, it isn't very attractive or compelling. CPAN is an index site, not a front page wrapper for your project.

When I see the LessCSS website, I'm seeing beauty which attracts me not only to LessCSS (which I don't really have any use for right now), but also (and this is key) to Ruby itself.

Returning to my quest, I had already sat down with our resident Ruby/Rails expert and a cup of tea and flux-seeds crackers. What I still wondered about was how can a Rails programmer know Ruby, Rails, Javascript (usually) and web design all at once and on such a high level for each to produce such beautiful websites. It's definitely not common, or is it?

We've laid the foundation that many programmers use prepared (sorry, there is no "pre-prepared" in English) templates and layouts, or hire an actual professional web designer. Hiring a web designer is not a cheap cost. Why are these people spending good sums of money on professional web designers to create (usually static) websites for such small programs? I couldn't understand it.

Then he mentioned some conventions he went to. When someone shows her peers and future employers projects she has done, the first impression is on the looks of things, not the sound or ability of them. If it's beautiful, she got to first base. If it works much better, but it doesn't even have a face, it stands much less chance of even getting a shot. This is what I was missing.

Marketing my project is not just marketing a code that does a task, it's marketing me!

Dancer doesn't just market a small web framework, it markets Perl and Plack (the encompassing technologies), but also the developer(s) of Dancer. Same goes for Catalyst, same goes for Moose and KiokuDB and Module::Starter and Test::More and on and on and on.

A response I'm expecting is "well, obviously!" and my answer would be the same, I do know it, however Understanding it is much more important than knowing it. To understand it means to write a website for whatever project I have done which I think can elevate me, and which I think can elevate Perl, or some other encompassing technology (like Moose or Dancer).

This morning I imagined having a space for project websites. Actual beautiful static website for each Perl project. I don't have the time to set up an infrastructure for hosting and all that. However, I did purchase PerlProjects.net and I will be giving NS hosting for free to anyone who wants to set up a website at <YourProject>.PerlProjects.net, no matter how small. Also, I want to set up a main site at PerlProjects.net which will be an index of... Perl Projects.

I do believe that having individual beautiful websites for projects (even the small or relatively exclusive ones) will help market Perl better, and any technology (of Perl or not) you're using in your project. This will help boost interest and consequently the number of programmers who know Perl will go up and of course job offerings will follow. If we can present an image of "Perl projects come in beautiful wrappers", we can present an image of "Perl is beautiful", and that is what we should strive for - at least in marketing terms.

If you feel like helping with PerlProjects.net (design, design, design) or want your own subdomain and free NS hosting, let me know!

Wednesday January 20, 2010
08:15 AM

On low attendance in TelAviv.pm meetings

this was originally posted on my new blogs.perl.org journal, which can be found here.

Shlomi Fish published a post in Israel.pm that I think is worth reading by anyone from Israel (or Israhell), especially if you're from the Tel Aviv region.

In a year that dealt mostly with marketing, there seemed to be a decline in people showing up to meetings. Even though Tel Aviv University provided us with a room (and now a better room), with a projector, boards, air conditioner and a lot of space, few people show up.

Whatever the reason is, if these trends continue, the meetings will cease. If you care about Perl, if you like to learn, if you like to teach and if you want to spread it, these meetings are for you (or for you to drag someone else to :).

I find it hard to believe there are no Perl programmers in Tel Aviv (I personally know a few), and that they all know everything already.

Seriously, wake up and give a shout out to Shlomi (or even me) about why you aren't showing up. Whatever reason it is, there's bound to be some solution to it.

P.S.:

No, I didn't really understand what you wanted, Shlomi. :) My answer was "whatever parser you want to write, it has two steps. Step one is Parse::RecDescent." I thought it was funny.

Tuesday January 19, 2010
05:22 PM

Elegance Fail

this was originally posted on my new blogs.perl.org journal, which can be found here.

Elegance might seem like a lost trait in programming these days, but it is live and vibrant in Perl. A rather large part of the Test namespace is devoted to providing an elegant way to write "run this code, get the result, compare it with this one".

Today I found myself at a loss of an elegant solution to a problem.

I want to run a set of tests. Theoretically I can write each subset of tests as a Role in a test object (there are at least three testing frameworks that allow this nicely) and then run the tests in the object. I can even use MooseX::POE (or regular POE, AnyEvent, Test::Aggregate and the list goes on) to run them asynchronously in order to save the time.

Two things bug me:

  • The majority of the tests count on file content, grep, cat, readlink and such. So basically I'm trying to run simple shell functions inside Perl. I could probably use Test::File for this and expand it to add what's missing.
  • I'm testing a remote server, not my own box.

I can:

  • Write the tests as if they're being run locally. Copy them over, run them and then delete them: I don't like this method. It's unclean.
  • Wrap everything in Net::OpenSSH (or POE::Component::OpenSSH): I don't like this method since it involves a lot of calls just to get the content. Even if I configure my OpenSSH to use a shared socket, it's just wasted ops, ya know?
  • Use an RPC server: that means running an RPC server (open iptables for this, of course), rewrite the commands in a long HEREDOC or q{} which is ugly and not very readable.

What else is there to do?

Right now the first option (writing the tests, copying them over and running them) seems like the best method because that way I get to write the tests the way I want to and could use Test::File and stuff like that to check what I want. However, I wish there was a more elegant way to solve this. GRID::Machine seems interesting. I could write test subroutines and send their references to the GRID machine.Of course I lose a matter of scoping and would definitely be tricky to manage the inheritance and roles connection I want to use. Frustrating.

Monday January 18, 2010
04:13 AM

Thoughts on my Moose lecture

this was originally posted on my new blogs.perl.org journal, which can be found here.

Yesterday I gave a talk on Moose (the post-modern metaclass-based object system for Perl 5) in TelAviv.pm. The slides are available on slideshare. They aren't a lot of slides because I mainly wanted to give an introduction to Moose for beginners and the gist of the lecture is me speaking, so the slides can't really express that.

Now, regarding the talk. There were few people present, which was a bit unfortunate but I'm assuming it's somewhat due to the change in location in the university. It was difficult to find, I have to admit. However, it's a new place, bigger and more comfortable.

If I could do it over again, I would definitely want more people and more participation so we could give some live examples instead of trying to squeeze it out of people. But overall, I'm happy with it.

I scheduled with Gabor to do the lecture on March in Rehovot.pm. If you're near Rehovot, stop by, say hello and learn how to use Moose!

I want to thank Shlomi Fish for organizing TelAviv.pm, the lectures (including this one), the promotion, receiving a free guest room from the university and all the other annoying/difficult things most people don't feel like doing.

Tuesday January 12, 2010
12:35 PM

Making Dancer Dance on CGI A.K.A. Blog Fix #543

this was originally posted on my new blogs.perl.org journal, which can be found here.

So, last blog entry I mentioned Dancer and that I want to use it as CGI (since it's Plack-aware) but I don't know how.

Nick Spacek commented on my entry saying that if it supports Plack/PSGI, there should be an easy way to run it as CGI.

Later that evening I sat down for hummus with my girlfriend (hummus is awesome, ok?). After talking to her about it, I decided to take a minute to check online on my phone if anyone has any explanations on it. In the first 5 results, I found a post Alexis Sukriah (who wrote Dancer!) wrote that same day, regarding my post, showing how to use Plack to run a Dancer app under CGI!

Case in point:

use Plack::Server::CGI;
use Plack::Util;

my $psgi = '/path/to/your/app/app.psgi';
my $app = Plack::Util::load_psgi($psgi);
Plack::Server::CGI->new->run($app);

... and the web server configuration to use it.
(example shown on Alexis' post)

account with lotsa forks and cares very much about community and communal programming. I have to say, that's a major high point in any module/program/framework I want to use.

So, I stand corrected. Plack is awesomer than originally thought.

miyagawa++, sukriah++ :)

03:44 AM

Hello Clarity

this was originally posted on my new blogs.perl.org journal, which can be found here.

For two weeks we've been working hard on defining a rather complex construct of DNS zone files, using multiple servers for multiple domains with cross referencing them and a lot of other complex-sounding terms.

We wrote DNS tests for the zones to make sure all the servers are configured correctly and the general DNS fetching provides correct information. This turned out to be quite difficult.

The original script is 130 lines. This is without taking into account even more testing we wanted. There was a lot of analyzing done which was rather repetitive and the overall code was ugly and not fun to read (to put it in mild terms). I decided to write a testing module for DNS zones - Test::DNS.

Using Test::DNS, we rewrote the script with a lot more options, which we wanted. The resulting script (with the addons) is 20 lines. It's clean and readable.

Here is how Test::DNS looks:

use Test::More tests => 5 * $num_of_domains;
use Test::DNS;
my $dns = Test::DNS->new();
foreach my $domain (@domains) {
    # assuming $domain is an object
    $dns->is_ptr( $domain->ns1 => $domain->ptr1 );
    $dns->is_ptr( $domain->ns2 => $domain->ptr2 );

    $dns->is_ns( $domain => [ map { "ns$_.$domain" } 1 .. 2 ] );

    # assuming there's overloading here
    $dns->is_a( "ns1.$domain" => $domain->ns1 );
    $dns->is_a( "ns2.$domain" => $domain->ns2 );

}

Test::DNS will be available soon on CPAN.

Enjoy!

03:42 AM

I Gotz Me a Dancer

this was originally posted on my new blogs.perl.org journal, which can be found here.

Sometimes a good time means relaxing with CPAN, reading a POD of something and trying to learn it. At least for me.

Last weekend I treated myself to playing with KiokuDB and Dancer. I'll write on KiokuDB later, this post is on Dancer.

Dancer is a Plack-aware web application framework written by Alexis Sukrieh. It has a built-in simple templating system, but supports Template::Toolkit (a must for me), routes handlers (named matching, regex matching, wildcard matching), simple rapid prototyping and support for multiple configurations and allows separate configurations for separate environments and stages of the software.

After playing with it for a rather short time, I already implemented CRUD. The code comes out clean (which I love), understandable and simple. Pure joy. You should totally check it out!

The only problem is that I want to write an interface which will be hosted on someone else's computer, which only supports CGI. I have no idea how to run it as CGI. Any ideas?

Update: Alexis Sukrieh wrote on how to do this here. Next post will reflect this!

Tuesday December 22, 2009
09:28 AM

Catalyst, MySQL, SQLite, H::FH, UTF-8 and more

this was originally posted on my new blogs.perl.org journal, which can be found here which is also the RSS feed for it.

I tried to update an online website with some changes. Generally, I run a production and a testing environment. Recently, however, I moved the code from using SQLite to MySQL and did not create a testing DB, so changes that require changing the text on the site are done in production. Not good? I know!

So the website is built in Catalyst. Originally used SQLite and then migrated to MySQL (which had to be done manually). It uses HTML::FormHandler to display the forms, with a generic CRUD layer I added.

When trying to load the form, I get weird characters for some of the page. From what I gathered, the data in the MySQL isn't kept in UTF8 but in latin1 but we declare the page as UTF8 encoding. The form isn't displayed in utf8 (which was changed using "use utf8;" in the form .pm file, or using Encode::Guess which yielded a better result). David Wheeler has a really interesting article on UTF8 in Perl here.

When that didn't work, I decided to convert the entire database to UTF-8. I read it using pure DBI, and using DBIC and explicit utf8 column declaration I inserted each table to a newly create database where the tables are charset UTF-8 with utf-8 collation. That didn't change anything. Apparently the latin1 was fine.

More investigation revealed that the HTML::FormHandler::Render::Simple was decoding some stuff which was screwing up a lot. Once that was fixed (AKA patching it up), more stuff remained unclear. It was as if H::FH wasn't able to read the correct record from the database. Trying to do it using $c->model('DB::Table')->search( { id => $id } ) worked just fine. Apparently the API in HTML::FormHandler changed or it has a critical bug.

I should read the manual. There is the manual, tutorial and an intro for it. They are all very very long and complex. They give me a headache. Honestly, I'm more comfortable reading the Perl XS tutorial rather than the current synopsis or ANY documentation of HTML::FormHandler.

Alas, I'll have to get rid of HTML::FormHandler. It's become so cumbersome I can't be bothered patching it any more even. I don't want to reinvent the form wheel again (because I probably won't put a lot into it). Maybe I'll add a layer on top of FormFu.

I remember asking mst about forms in #catalyst a long while ago. He said that "forms suck.. some people find FormFu helps make them suck less." I think I understand it better now.

Sunday December 13, 2009
11:49 AM

Presenting Module::Starter 1.54

this was originally posted on my new blogs.perl.org journal, which can be found here which is also the RSS feed for it.

Module::Starter 1.54 was recently released. I felt like blogging a bit about it.

Module::Starter is one of my favorite projects in the Perl community and on CPAN. I use it whenever I want to start a new module or program. It's very simple (read: beginner-friendly), modest and provides what I don't feel like doing myself. It has boilerplate files, text and tests and keeps updating those with habits that are considered "best practice", such as replacing the default licensing term we use.

Recently I've had the pleasure and honor of contributing to this project. I've made several changes mainly taking care of tickets and patches people submitted, making this release a fixer upper release. Following releases will contain some bigger and more exciting changes.

I want to take this chance to thank Andy Lester, Ricardo Signes, Shlomi Fish and anyone else involved with Module::Starter. Also, I want to thank some lesser-known contributors to Module::Starter, those that send bug reports, feature requests and patches (on my god, patches *drool*).

It seems like Ricardo's Dist::Zilla is the contender for Distribution Manager position, and with just cause. However, I rest assure that there are plenty of people (such as me) who prefer using small specific tools for each job they have. I prefer Module::Starter as a module starter, Module::Build as a Distribution Manager, etc.

That being said, you should still totally check out Dist::Zilla and see if it's the right tool for you. But don't worry, Module::Starter is not going away. :)