Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

hide (4398)

hide
  (email not shown publicly)
http://blogs.sackheads.org/hide

hide has been programming in Perl since 1998. Much of this time has been focused on system automation and data warehousing.

On CPAN as: SSORICHE [cpan.org]

Journal of hide (4398)

Friday January 28, 2005
10:06 PM

TO.pm

Last night I attended my first Toronto Perl Mongers meeting, and it was quite good. I'd wanted to attend for quite some time, but something always seemed to get in the way (mostly finishing our basement). One of the other road blocks is the meetings are held right downtown. I'm not a big fan of going downtown and luckily I have a couple of friends from work who were willing to go.

Meeting new people isn't one of my strong suits. I tend to become quiet, reserved and not talking to anyone. However going to the meeting gave me a chance to meet lamech, who I've talked to on IRC for the past couple years. We met for dinner before the meeting for Thai food at Spring Rolls. The Tom Yom Goon soup was amazing (and on such a cold day required0 and the Pad Thai was good.

At the meeting James FitzGibbon gave a talk on Testing the Testers. Having contributed to Test::WWW::Mechanize and using Test::Builder::Tester I had an interest in what he had to say. James did a good job of explaining testing and highlighting the benefits. Unfortunately it didn't appear as though there were many in the audience who had experience writing test code.

One of things that I got out of his presentation was use of Test::Builder::Tester::Color. On more than one occasion I've had a problem determining what the difference was between my expected output and what the actual output was.

Another interesting piece of his presentation, that was brought up as a tangent, was the use of Safe. I'd never heard of this module before, and James did a good job of explaining the benefits. I know of a couple of projects that I'm working on where it could be useful. At this point the conversation drifted to thoughts on a meeting devoted to defensive programming. Hopefully it'll happen soon.

I'm happy I finally got to attend a meeting and am looking forward to the next meeting.

Tuesday January 25, 2005
11:43 AM

What a day (so far).

Today I'm preparing to role out a new version of an application I've been working on for the past couple of weeks. This is a bit of legacy code that we use everyday, but required additional functionality. I took this opportunity to begin refactoring, and adding tests. It's a major rewrite, that starts to use modules from CPAN, and modules that I've written rather than one long script.

All was fine on my development box, the trouble began when I started to test the application on our production environment. I'd forgotten that this application lives on a box running with the OS installed Perl 5.6.0.

"No problem compile 5.8.x" you might say. We aren't allowed compilers in production, and I have no test box matching this configuration. "Install a binary", I managed to find one that was approved for use. The application requires XML::Parser (requiring a compiler). Tracked someone down with a compiler, right version of Perl, wrong compiler.

Back to trying to use 5.6.0. We have a local CPAN::Mini::Inject mirror that we use for our other applications, so this was a natural choice of where to install modules from (plus the application server does not have Internet access). Configured CPAN, try to point it at our local mirror. No luck. We use HTTP access (ftp not being allowed) and LWP had never been installed on this server (no wget, nor ncftp either).

Copy LWP over by hand, discover it's prerequisites, and do the same for each of those. I'm now stopped by not having a compiler and not having Perl 5.8.x to install an LWP prerequisite.

I've found someone to compile Perl and the modules that require compiling for me. Hopefully I'll be able to use CPANPLUS to install the rest of the modules and move on.

Saturday January 08, 2005
10:20 PM

Testing the exception.

When writing tests for my modules, my usual method is to create a separate .t file for each method within a module. This method does create quite a few .t files with generally few tests in each. I don't know whether this is a good or bad method but it's works great for me and the way I develop. .t files are created as I need each new method (starting with new), and tested with prove -lv (thanks for the -l Andy).

I tend to put Carp to good use. croak can be found throughout the modules I write wherever a method requires something (write access to a file, parameters passed, etc). To test that each croak works the way I expect I use Test::Exception's dies_ok.

In each .t file I'd have the following after the tests for proper use:

SKIP: {
  eval { use Test::Exception };
  skip 'Test::Exception not installed', 1 if $@;

  dies_ok { test code here }, 'message';
}

Test::Exception is installed on all of my development boxes, and all of our production boxes to ensure full testing before a module gets installed. This caused me not to realize that the above code does not work if you don't have Test::Exception installed. Something I did not notice until I released CPAN::Mini::Inject onto CPAN.

One of the cpantesters and rjbs brought this to my attention on the Windows platform. I'd thought I'd fixed it with the last release, until today.

I spent today installing Perl and configuring remote access on one of the kids Windows XP systems (my plan is to use it to test my code and to as a cpantester). First thing, install CPANPLUS, second try to install CPAN::Mini::Inject. Failures all over the place. Can not find Test::Exception.

I ended up moving all of the exception tests out of each method's .t file and into a zz.exceptions.t file. The following code is at the top skipping every test if Test::Exception isn't installed:

use Test::More;

BEGIN {
  eval "use Test::Exception";

  plan skip_all => "Test Exceptions required to test croaks" if $@;
  plan tests => 9; # # of tests.
}

Now the tests all pass on boxes with or without Test::Exception installed. Needless to say expect CPAN::Mini::Inject 0.16 shortly.

Tuesday January 04, 2005
08:28 PM

CPAN::Mini::Inject 0.12

After releasing 0.10 a bug was discovered by cpantesters (thank you!). I had coded skips in each test for file permission if testing on Windows. I managed to get the wrong name for the OS ('Win' instead of 'MSWin'). This caused numerous tests to fail.

After updating the failing tests I asked rjbs (thank you) if he would run the tests for me on Windows before I released this new version. More test failures.

I had previously changed the tests to use:

eval "use Test::Exception";
skip 'Test::Exception not installed', 2 if $@;
use Test::Exception;

to run on an AIX test box I had. This code while working on OS X, AIX, and Linux (FC2 and RHEL 3.0) broke on Windows. I changed the code back to:

eval { use Test::Exception };
skip 'Test::Exception not installed', 2 if $@;

The tests now work on Windows, OS X, Linux, and another AIX box I managed to test on, but still not on the original AIX box. As it appears the problem may be with the first AIX box rather than the eval {} code, this release uses eval {} in it's tests.

Thursday December 30, 2004
08:09 PM

CPAN::Mini::Inject 0.10

I just finished updating CPAN::Mini::Inject to fix a bug with dirmode settings. The dirmode would be set correctly during CPAN::Mini's mirroring process, but any files that CPAN::Mini::Inject added or created did not have the correct mode set. This made doing updates in a multi-developer environment difficult.

A little background into CPAN::Mini::Inject. I work on a number of projects where the modules I write can not be released onto CPAN. When working in a single server environment this isn't much of an issue, but my current project involves 4 different servers. Managing 15 private modules on each of these servers was becoming difficult. We use CPANPLUS on each server to manage publicly available modules, with CPAN::Mini::Inject we can now use CPANPLUS for our private modules as well.

As it's name suggests, CPAN::Mini::Inject allows you to add (inject) your own private modules into a CPAN::Mini mirror. Allowing the use of CPAN/CPANPLUS to install public and private modules with one procedure.

We also use a separate CPAN::Mini::Inject mirror to test the installation of our development modules before releasing. This ensures that during the process any prerequisites are installed, and that all tests pass.

Tuesday December 28, 2004
09:27 PM

del.icio.us and Rubric.

I love del.icio.us. I have to admit at first I did not see the benefits of it (like others that I've talked to), and each time I saw it I just closed it. It wasn't until I actually started using del.icio.us that I started to appreciate it. In my opinion the most useful features are the RSS feeds of the most popular URLs, and the inboxes of other users.

Our corporate intranet is huge. There is just so much information on it, you can't easily find everything you need. When people in our department do find information they need, it rarely gets shared, and a lot of time is spent by other team members looking for the same information. This led me to thinking "we need a del.icio.us type service on the intranet".

As a lot of our information is private, we can't use del.icio.us. I began thinking of coding a clone for use internally. A couple days after deciding to code my own rjbs released Rubric 0.00_01. I installed it and began working with rjbs (where working means complaining that things aren't working right) to fix multi-user use. The addition of open registration has allowed me to deploy Rubric to our department without burdening me with user creation.

While Rubric started out as a clone of del.icio.us, it's grown into something more. One of the new features is the ability to create a note, either attached to a URL or on it's own. Add to this the ability to require a user id and password before any information is displayed, and it makes for an excellent note management system.

I used to have a web application on my home intranet to maintain my wife and my notes (recipes, technical tips, code snippets, etc) written in PHP (that broke after an upgrade to PHP). Rubric has now taken it's place.

Lately, I've noticed that del.icio.us and nutr.itio.us (a popular interface for posting to del.icio.us) have been experiencing problems. I haven't been able to use nutr.itio.us since before Christmas, and I regularly experience problems with del.icio.us (connection refused, 500 errors, etc.). I assume these are to do with the rapid growth and popularity of both services. Luckily I now have my own Rubric web bookmarking application.

Monday December 20, 2004
07:33 PM

Class::DBI Part 2.

Thanks to comments from TeeJay and perrin, I've rewritten my database modules to use set_sql. This has improved the performance of my application plus it's given me the greater flexibility that I require (using SQL::Abstract [cpan.org]). As TeeJay suggested I followed Randal's Example [class-dbi.com].

I now see how I can integrate Class::DBI and SQL::Abstract into my day jobs application but I'm vacation from that for 2 weeks so that train of thought will have to wait.

Thursday December 16, 2004
09:01 PM

Class::DBI OH MY!

I've had this idea for a home project for a number of months and just recently started working on it. I quickly created a database design (that was scrapped and rewritten based on input) and a SAX parser to load the data. Having heard good things about CDBI I felt this would be a good project for me to learn it on. I've been using DBI and SQL at work for a while now, and I'm quite comfortable with writing SQL queries to retrieve the data I need, but as this was an exercise in learning CDBI I really wanted to make it work.

The problem being that I want to retrieve data from one table that matched criteria in two other tables:

            -----------
            - table_1 -
            -----------
     -------- pkey    ------
    |       - title   -     |
    |       - descr   -     |
   /|\      -----------    /|\
-----------             -----------
- table_2 -             - table_3 -
-----------             -----------
- pkey    -             - pkey    -
- fkey    -             - fkey    -
- date    -             - field_3 -
-----------             -----------

select title, descr
  from table_1, table_2, table_3
  where table_1.pkey=table_2.fkey and date='2004-12-16' and
     table_1.pkey=table_3.fkey and field_3 = 'test';

(please note these are only examples and haven't been tested)

While this was easy to come up with in SQL, I was having a tough time figuring out how to achieve the same results in CDBI. At this point I turned to rjbs for guidance. He knew what I was looking for and explained to me what I needed to do. I just wasn't getting it, what seems so simple in SQL was becoming stressful in CDBI. rjbs stuck with me and answered my questions.

I think I've got a solution now that involves retrieving the values of fkey from both table_2 and table_3, finding their intersection and returning the intersection through _ids_to_objects. Resulting in a list from table_1.

I want to thank rjbs for his help - Thank you!

Sunday October 31, 2004
08:45 PM

My First Module (boiling the ocean)

Today I released my first module to CPAN, CPAN::Mini::Inject [search.cpan.org]. While I maintain a number of modules at work, and have submitted patches to others on CPAN. This experience was quite different.

I've been working on this module off and on for a little while now. Just when I thought I was about to release it, I'd find some other feature that needed to be added, or another test that needed to be done. I got caught up in trying to make CPAN::Mini::Inject do everything that people may want it to do thus boiling the ocean. I was busy trying to make it a completed module rather than releasing functional code and allowing it to grow. It took a number of people telling me "just release it already", then a couple more modifications before I felt comfortable releasing it.

So now it's out there, for anyone to see and use.

Friday September 17, 2004
09:10 AM

Subversion

I've been using Subversion for a while now for my personal script version control at home. Recently I received a new notebook for work and decided to install a local Subversion repository on it to store my test scripts and documentation.

I've found in the past that I'd written out a short test script to prove a theory or see how a subroutine would work, then I'd change the script and lose what I'd done before. A couple of days later I'd be working on something and think "I've done this before in a test script" go to look for it, and realize I'd deleted that section, or changed it too drastically.

To solve this I've created two repositories one for documentation and the other for test scripts. I treat them just as I do any other programming project. Issue resoloved. I don't know why I hadn't thought of doing this before.