perrin's Friends' Journals perrin's Friends' use Perl Journals en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:07:53+00:00 pudge Technology hourly 1 1970-01-01T00:00+00:00 perrin's Friends' Journals Perl hashes in comparison to C, C++, Python and Ruby I just came across <a href="">this blog post</a> which benchmarks several hash implementations in C and C++ including the hashes or Python and Ruby. <br> <br> I was wondering if someone wanted to <a href="">fork the benchmarks on GitHub</a> and add a Perl entry. I haven't done C in a while and I've never done XS, so I'm probably not the best person for that job. <br> <br> Any volunteers? mpeters 2010-03-25T14:47:41+00:00 journal <p>I'm trying to like the new site but every time I try to use it something goes wrong. Every time I try to comment on a post I get this strange error message (It's happened to me every day for at least a week. I decided to ignore the first few times since all sites have problems and it is still beta, but I guess I just got too annoyed today):</p><blockquote><div><p>Comment Submission Error</p><p>Your comment submission failed for the following reasons: Text entered was wrong. Try again.</p></div></blockquote><p>Really? My text was wrong? Do tell, how could it have been made "right".</p><p>Not only is this error message completely unhelpful, but it takes me to a separate page where I have to click my browser's back button to try again. Really? That's so 90's.</p><p>I've tried clearing out any cookies that might be bad for some reason; I've tried changing my content, my name, my url, etc. Nothing seems to work. Anyone know what's going on or have a clue as to how I can fix this?</p> mpeters 2010-02-11T15:35:07+00:00 journal Obligatory giving of Thanks! I'm currently waiting for the turkey and stuffing to cook and am feeling really grateful for lots of things, so I thought I'd share some of my programming gratitude here. In no particular order, I'm thankful for: <ul> <li> <strong>Mark Stosberg</strong> - Not only is he the driving force behind the CGI::App community but he's also great at helping out with important modules that have fallen a little stale (, CGI::Session, Data::FormValidator). He's also been a patient follower and contributor to my Smolder project. </li><li> <strong>Brad Oaks</strong> - Not a CPAN contributor, but the main organizing effort behind and a fellow co-worker. Our technical meetings are good geekery and our social meetings are pretty fun too. Not all of us programmers are good at organizing social things, so I'm glad that he is. Not only is he a good sounding board for my ideas at work but he makes me think of things that I didn't see on my own. </li><li> <strong>Ricardo Signes</strong> - Great speaker, prolific module author and very funny guy. I've had the privilege of hanging out with him at lots of YAPCs and a couple Perl QA Workshops and it's always a blast. His willingness to delve into the abyss that is Email and make it sane (or saner) for the rest of us is enough to earn him permanent Perl sainthood. </li><li> <strong>Perrin Harkins</strong> - mod_perl guru and former co-worker, I really learned a lot from him about architecture, code structure, maintenance and a million other things. He's usually playing with lots of different new toys and sharing all his benchmarks and comparisons so the rest of us can make better decisions. </li><li> <strong>Jim Brandt</strong> - Current President of the Perl Foundation which means he has the hardest thankless job which everybody takes for granted in the Perl community. It's amazing how much gets done in a completely volunteer effort like the TPF and we should all be grateful for his work (and those of past presidents like Allison Randall and Richard Dice). </li></ul><p> I know I've left lots of people off this list, but the turkey needs basting, so I'll leave the rest of the thanks to other people.</p> mpeters 2009-11-26T19:17:19+00:00 journal Help needed on a new 5.10 warning I'm trying to run an existing application on 5.10 for the first time (5.10.0 for right now) and am getting a warning that is not making it easy for me to track down. Any help from the lazy web would be appreciated.<br> <br> It goes like this:<blockquote><div><p> <tt>&nbsp; Variable "$x" is not available at (re_eval 1070) line 1.</tt></p></div> </blockquote><p>I'm assuming this is pretty deep inside some dependency I'm using (of which my application probably has an old version which doesn't play nicely with 5.10). Google is no help here as it returns a lot of CPAN testers results with the same warning for various modules (<a href=""></a>, <a href=";v=10">;v=10</a>).<br> <br> My normal trick of using</p><blockquote><div><p> <tt>&nbsp; $SIG{__WARN__} = \*Carp::cluck;</tt></p></div> </blockquote><p>to get a nice stack trace of where the warning is coming from is not actually giving me a stack trace. Anyone have any extra ideas?</p> mpeters 2009-09-15T21:48:20+00:00 journal What Perl can learn from OpenBSD Will all the talk about trying to improve the Perl core development and release process it's interesting to see what other (non-commercially backed) open source project are doing - <a href=""></a> <br> <br> From the slashdot overview:<blockquote><div><p>Twelve years ago OpenBSD developers started engineering a release process that has resulted in quality software being delivered on a consistent 6 month schedule &#8212; 25 times in a row, exactly on the date promised, and with no critical bugs. This on-time delivery process is very different from how corporations manage their product releases and much more in tune with how volunteer driven communities are supposed to function.</p></div> </blockquote> mpeters 2009-07-17T01:05:59+00:00 journal My Slides from YAPC Just wanted to post links to my slides from YAPC for the 2 talks I did there. Not sure how interesting they are without my accompanying tenor, but here they are: <br> <br> <a href="">TAP in Depth</a> <br> <br> <a href="">Continuous Integration Testing in Perl</a> mpeters 2009-06-25T16:44:59+00:00 journal Open Source Advertising Just saw this <a href="">announcement on reddit</a>. <br> <br> Please take a few seconds to <a href="">find Perl and Parrot and upvote them</a>. And if you feel like downvoting some other projects, no one would hold that against you<nobr> <wbr></nobr>:) It's a nice way to show them that the un-dead Perl programmers can still click on links. mpeters 2009-06-10T18:27:28+00:00 journal Another perltidy lazyweb request Having been inspired by <a href="">Domm</a> to fix one of my perltidy annoyances (it says something about the greatness of a tool that I keep using it even though there are some annoyances...). <br> I have these<nobr> <wbr></nobr>.perltidyrc settings:<blockquote><div><p> <tt>--indent-columns=4<br>--cuddled-else<br>--nooutdent-long-quotes<br>--paren-tightness=<nobr>2<wbr></nobr> <br>--brace-tightness=2<br>--square-bracket-tightness=2</tt></p></div> </blockquote><p>And I have this code:</p><blockquote><div><p> <tt>Some::Class-&gt;create(<br>&nbsp; &nbsp; {<br>&nbsp; &nbsp; &nbsp; &nbsp; foo =&gt; 3,<br>&nbsp; &nbsp; &nbsp; &nbsp; bar =&gt; 2,<br>&nbsp; &nbsp; &nbsp; &nbsp; baz =&gt; 1,<br>&nbsp; &nbsp; }<br>);</tt></p></div> </blockquote><p>Now I want those opening and closing tokens to be closer together and not waste so much vertical space. So I added this:</p><blockquote><div><p> <tt>--stack-opening-tokens<br>--stack-closing-tokens</tt></p></div> </blockquote><p>Now my code looks like this:</p><blockquote><div><p> <tt>Some::Class-&gt;create({<br>&nbsp; &nbsp; &nbsp; &nbsp; foo =&gt; 3,<br>&nbsp; &nbsp; &nbsp; &nbsp; bar =&gt; 2,<br>&nbsp; &nbsp; &nbsp; &nbsp; baz =&gt; 1,<br>});</tt></p></div> </blockquote><p>Which is fabulous except that the inner args are indented 2 levels (8 spaces) instead of just 1 (4 spaces). So how do I get this?</p><blockquote><div><p> <tt>Some::Class-&gt;create({<br>&nbsp; &nbsp; foo =&gt; 3,<br>&nbsp; &nbsp; bar =&gt; 2,<br>&nbsp; &nbsp; baz =&gt; 1,<br>});</tt></p></div> </blockquote> mpeters 2009-06-09T20:37:41+00:00 journal Article about Person duplicate data Just thought I'd mention it since it's in another place: <a href=""></a> mpeters 2009-04-21T18:06:02+00:00 journal Module::Build::Smolder <p>Building off my lastest CPAN release module (Module::Build::TAPArchive) I've just released Module::Build::Smolder. Now if you want to use Smolder in your Module::Build based projects all you need to do is use this new subclass and then you can say:</p><p> &nbsp; &nbsp; $]<nobr> <wbr></nobr>./Build smolder<nobr> <wbr></nobr>...</p><p>The "..." lets you specify which Smolder server the results are sent to and for which project, etc. Both this and Module::Build::TAPArchive are preludes to another big announcement that I was hoping to have done this morning, but looks like it will have to wait until at least later tonight.</p> mpeters 2009-03-30T13:12:45+00:00 journal Module::Build::TAPArchive <p>TAP Archives are nice little packages of your test run that contain all the different TAP streams as well as test suite metadata. It's an easy way to store the results of a test run and then analyze them later. This is what Smolder uses but it's not specific to Smolder.</p><p>While it's easy to use TAP::Harness::Archive yourself to create your own harness it's kind of overblown for small projects. prove -a works, but sometimes you really want it as part of your standard perl Build.PL &amp;&amp;<nobr> <wbr></nobr>./Build test style of work.</p><p>Enter Module::Build::TAPArchive. Just use it instead of Module::Build in your Build.PL and then you can do</p><p> &nbsp; &nbsp; perl Build.PL &amp;&amp;<nobr> <wbr></nobr>./Build test_archive</p><p>It's not doing anything really difficult behind the scenes, it should just make life a little easier for anyone who wants to play with TAP archives.</p> mpeters 2009-03-29T13:45:57+00:00 journal eat your first cookie before asking for seconds I like Gabor a lot and I think his work with Padre is really going to pay off in efforts to attract new blood to the Perl community, but I couldn't read the latest post on his blog without writing this response (he doesn't have comments on his blog, instead he encourages people to respond to him on their own blogs). <br> <br> You should read <a href="">his original post</a> before continuing. <br> <br> Asking TPF to fund 2 more grant ideas when you haven't even finished the last one they gave you is just asking to be rejected. It reminds me of my 5 year old asking for more cereal when he still has half his bowl left. I'm extremely unlikely to get up and get him some more. <br> <br> I've been on the receiving end of the grant process and I know it's really hard to get motivated enough to finish those grants (the money doesn't help all that much). And in his case he admits to biting off a bit more than he can chew with the XS components. So I understand his desire to drop his current grant project. <br> <br> But he should have done that prior to applying for more grants. And even then I wouldn't blame TPF for not accepting him for the next round if he didn't finish his last. <br> <br> Like I said, Gabor is a great guy and really does good work, but I don't think he should see the rejection of his grants as a rejection of Padre by TPF. Instead look at it like they are good stewards watching over the Perl community's coffers and limited resources and being cautious with investments. mpeters 2009-03-05T20:54:09+00:00 journal Making Maps with Math::Geometry::Voronoi I've been working with generating overlays for google maps lately at my day job. Without going into too much detail, the data set I need to display is a set of points, each assigned to a given set. In the real world these sets form contiguous regions which I need to translate into shapes to draw over the map. The rub is that these regions come in really complex shapes - drawing convex hulls isn't an option. <p> My first attempt at the problem once I realized I couldn't draw a hull was to divide the world into a grid and color each box according to what I found inside, subdividing until each box contained only a single set's point(s). The results were blocky (duh) and not all that appealing. But as long as the data was dense enough (i.e. urban areas) it did a decent job of expressing the shapes. Unfortunately in rural areas where points are more spread out it looked terrible. </p><p> For my second attempt I decided to actually learn some computational geometry - I read a book and quite a few sites around the web. This lead me to <a href="">Voronoi diagrams</a>. There's a wide variety of Voronoi diagram producing code out there on the web - everything from highly-templated C++ monsters to Java to some excellent 20-year old C code. I chose the latter of course (Perl and old-school C go together like chocolate and peanut butter), and after some serious debugging I give you <a href="">Math::Geometry::Voronoi</a>. </p><p> <b>You can see a demo of it in action here: <a href=""></a>.</b> This is essentially what I'm doing with Google maps, except that I'm coloring multiple regions the same color and drawing a line around the border rather than each cell. </p><p> The speed of the diagramming code is really good. Don't let the demo fool you - that code is running in CGI mode on a shared server. Put it on a fast mod_perl server and it's definitely not going to be the bottleneck in a mapping application. That prize goes to Google Maps. Sweet app, but it sure is slow! </p><p> -sam</p> samtregar 2008-06-22T19:22:07+00:00 journal LD_LIBRARY_PATH trick PART 2 In <a href="">my last post</a> I talked about a trick with setting LD_LIBRARY_PATH in the same script that needs it. Then I ran into some problems with code being run by a <code>perl -e</code> scenario. My new solution (with some help from <a href="">rjbs</a>) is actually even simpler than what I'm doing now. The downside is it's not really portable. Should work for most *nixes and I only really need it to work on Linux, so I'm good. <br> <br> The answer lies in asking<nobr> <wbr></nobr>/proc what command line options I was invoked with:<blockquote><div><blockquote><div><p> <tt>$ENV{LID_LIBRARY_PATH} = '/some/computed/path';<br>exec(split("\0", `cat<nobr> <wbr></nobr>/proc/$$/cmdline`));</tt></p></div> </blockquote></div> </blockquote> mpeters 2008-06-13T18:49:44+00:00 journal LD_LIBRARY_PATH trick I have this problem a lot. I need to link some Perl module against some C library that is not in the standard system libs. Usually this is because it's a locally installed C thingy (like Swish-e or some payment processor API) that I don't want to pollute the system install. Plus it's nice and easy if the whole application is all bundled together. <br> <br> The easy way to do this is to set the <code>$ENV{LD_LIBRARY_PATH}</code> from your shell to point to the right place. But this is a bother, and not to mention really hard to do if you need some dynamic bits to figure out what the right path is. <br> <br> And for security reasons, programs can't set their own local version of <code>LD_LIBRARY_PATH</code> and have it be respected by the system's loader. It has to be done by the calling process (in most cases the shell). <br> <br> This little trick let's a script set it's own <code>LD_LIBRARY_PATH</code> by relaunching the same program with the same arguments as a new sub-process. Simply put a <code>BEGIN</code> block like this in your script:<blockquote><div><p> <tt>BEGIN {<br>&nbsp; &nbsp; if(! $ENV{LD_LIBRARY_PATH} ) {<br>&nbsp; &nbsp; &nbsp; &nbsp; $ENV{LD_LIBRARY_PATH} = join(':', @some_paths_I_need);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $self_cmd = "$^X $0 " . join(' ', @ARGV);<br>&nbsp; &nbsp; &nbsp; &nbsp; system($self_cmd);<br>&nbsp; &nbsp; &nbsp; &nbsp; exit;<br>&nbsp; &nbsp; }<br>}</tt></p></div> </blockquote> mpeters 2008-05-15T23:05:53+00:00 journal VMWare's Jerky Mouse This isn't the least bit Perl related, but I figured this was an easy way to get some useful info out to Google. I searched in vain for a solution and managed to hit it only by accident. Maybe I can save someone else the trouble. <p> In short, I installed Fedora 8 as a guest in VMWare running on Windows XP. Everything was fine but the mouse movement was jerky. Not jerky like a video performance problem, but more like wiggly and jittery. It looked a little like my hand was shaking on the mouse, but I was calm as can be. </p><p> Reading through the logs, I realized that X had somehow detected my mouse twice. I guess it was getting conficting signals from each movement. I fixed it by opening<nobr> <wbr></nobr>/etc/X11/xorg.conf and commenting out the entire mouse configuration section and the reference to the mouse config in the screen section. I restart X and my mouse was auto-detected fine. </p><p> Most of the advice I could find on the topic of funky mouse movement in VMWare advises you to install VMWare tools. Don't bother - it won't work under Fedora 8 and you'll waste a lot of time trying. You don't need it anyway - Fedora 8 comes with working VMWare display and mouse drivers. </p><p> -sam</p> samtregar 2008-04-23T23:15:04+00:00 journal Oslo QA Workshop and Smolder <p>Smolder 1.21 has just been released thanks to the Oslo Perl QA Workshop and my<br>trip sponsored by and Linpro (Linpro was an amazing host for this event. They gave us free reign of their offices for the whole weekend, free food, drinks and travel). This releases fixes several installation<br>bugs on various platforms and other small bugs. It also adds several new<br>features. The highlights are:</p><p>+ Project categories have now been replaced by the more flexible smoke report tags<br>+ TAP archives can now be downloaded through the UI for a given report<br>+ Graphs can now be done on test run duration if that info was provided in the<br>archive. This was inspired by Ovid's work on the BBC's test suite's run time.<br>+ The default DB-Platform is now SQLite instead of MySQL</p><p>See the changle log for full details.<br></p><p>I was also pretty heavily involved in the group working on the next version of TAP. I have to say that this gave me a new appreciation for standards committees. It's really hard to balance extensibility, backwards compatability and ease of use. Especially with TAP since one of it's goals is being extremely human readable. We definitely made some progress and all of the new, slightly less readable features are completely optional so if authors don't want to use them, they don't have to. But Smolder is going to have fun with all this new information coming it's way!</p> mpeters 2008-04-07T22:02:04+00:00 journal Sharing a DBI handle between Class::DBI and Rose::DB::Object At my current job we've got a large existing code-base built on Class::DBI. For my current project I decided to experiment with using Rose::DB::Object instead, hoping to see less need for hand-written SQL. So far it's been a success, but one issue was quite difficult to get right - getting the two systems to share a DBI handle. If I didn't do this then I was going to see a doubling of total DBI connections when I deployed my new app to the web cluster, which is unacceptable. <p> I started with the most obvious solution, an over-ridden init_db() in my Rose::DB::Object subclass: </p><blockquote><div><p> <tt>sub init_db {<br>&nbsp; &nbsp; my ($pkg, @args) = @_;<br> <br>&nbsp; &nbsp; My::Rose::DB-&gt;new_or_cached(dbh =&gt; My::Class::DBI-&gt;db_Main(), @args);<br>}</tt></p></div> </blockquote><p> That worked great at first - when Rose needs a DB connection it gets one pre-loaded with my Class::DBI handle. (And as a side-note, the fact that the Class::DBI handle uses DBIx::ContextualFetch doesn't cause Rose problems.) </p><p> However, sometimes for unknown reasons Rose will decide it needs to reconnect after the initial connection is established. To intercept these calls I added an overridden dbh() method to my Rose::DB sub-class: </p><blockquote><div><p> <tt>sub dbh {<br>&nbsp; &nbsp; my $self = shift;<br>&nbsp; &nbsp; unless (@_) {<br>&nbsp; &nbsp; &nbsp; &nbsp; $self-&gt;{dbh} ||= My::Class::DBI-&gt;db_Main();<br>&nbsp; &nbsp; }<br>&nbsp; &nbsp; return $self-&gt;SUPER::dbh(@_);<br>}</tt></p></div> </blockquote><p> Another useful hint which helped me notice very quickly when Rose decided to reconnect - I gave Rose::DB an invalid password in my call to register_db. That way Rose would have all the correct information about the connection, but wouldn't be able to initiate new connections. </p><p> I hope this helps other suffering Class::DBI users to give Rose a try! </p><p> -sam</p> samtregar 2008-04-03T20:12:31+00:00 journal HTML::FIllInForm patch to mark invalid fields I've been working on producing yet another form validation system, this time for MasonX::WebApp. I modeled my work on the very nice CGI::Application::Plugin::ValidateRM, with a bit of the Krang::Message system thrown in to help get errors out to the user. <p> One thing I wanted to do this time that I'd never done before was automate marking invalid form fields. I'd done this in the past using manual template changes, but that's a lot of grunt work. </p><p> Instead, my coworker Perrin Harkins suggested we use HTML::FillInForm to do the job. We're already using it to re-fill forms with errors, so why not use it to setup a CSS class on invalid fields too. It turns out HTML::FillInForm has a very similar facility already to disable selected fields, so adding an invalid fields feature was pretty simple. </p><p> I've already sent along the patch to the maintainer, but if you want to get it now here it is: </p><blockquote><div><p> <a href="">fillinform_invalid2.diff</a></p></div> </blockquote><p> -sam</p> samtregar 2007-12-03T19:57:21+00:00 journal Get your Geo::Coder::US DB file here! <p> I've put up a freshly built <a href="">Geo::Coder::US</a> DB file for download here: </p><blockquote><div><p> <a href=""></a></p></div> </blockquote><p> It's 384MB compressed (845MB uncompressed), but believe me, you'd much rather download this than download the 6.6GB of <a href="">TIGER/Line source files</a> needed to build it! </p><p> Supposedly I've got 6TB of downstream bandwidth each month at my ISP (<a href="">Dreamhost</a>). I guess we'll see if they meant it! I should hook up something to take down the file automatically if I go over my limit... </p><p> -sam </p> samtregar 2007-07-06T20:19:53+00:00 journal [smolder grant] progress update <p> I found some time this weekend to do some more work on my TPF grant to improve Smolder. I'll be doing a presentation to the Raliegh Perl Mongers on Smolder the first week of June, so I'll need to find some more time pretty soon too. Plus I'll need to definitely finish this round of work before YAPC and OSCON. </p><p> The first milestone (ditching the old XML format for plain TAP) is now feature complete (with tests of course). The accepted format is now a TAR archive containing TAP files with one YAML file with META information about the test run. To see more documentation see <a href=""><nobr>.<wbr></nobr> pod?view=markup</a>. </p><p> The next 2 milestones have to do with setting up a system to easily automate CPAN style distributions for automatic testing from svn, etc. I've put this off for a few days but will get to it soon. </p><p> The 4th milestone was to add project specific RSS feeds. I was originally planning on using <a href="">XML::RSS</a>, but I've decided to go a simpler route. I don't need to parse RSS or do anything fancy other than emit XML using RSS tags. Using a template would be much easier. It also means I don't need to include the dependencies of XML::RSS inside of Smolder either, so it's that much smaller. </p><p> This milestone is now feature complete as well. That just leaves 2 left. </p> mpeters 2007-05-20T16:31:25+00:00 journal [Smolder Grant Update] more work with TAP::Parser <p> I was a little delayed in my work this week ($work and fun stuff) because my SVK repository got really screwy and wouldn't let me push changes back for certain branches. So I took a whole lot of time troubleshooting that and then finally just deleting everything and starting my svk mirrors all over again. </p><p> After that I was able to jump right back into the gutting of the XML stuff from smolder in favor or plain ol' TAP. This went pretty smoothly and I was able to get a nice an intial implementation working, so I committed it back to the svn repo. I was a little pressed for time, so the test suite is still broken, but that shouldn't take too much time to get into order. </p><p> While working with the code that I haven't touched in a while, I noticed a lot of other areas (not related to my grant work) that could use some sprucing up too. Plus there's lot of new tricks and tips I've picked up from my work on other stuff (mostly the AJAXification of <a href="">Krang</a>) that would be useful for smolder to. But I guess that's the way it goes. There's always something to learn and never enough time to backport those learnings to earlier projects. </p> mpeters 2007-04-28T21:44:40+00:00 journal Review of Programming Erlang, the short version <p> In short, <a href="">fantastic book</a>, absolutely recommended to any progammer interested in learning about a fascinating and very novel progamming language. It's very well written and I was engaged from start to finish. </p><p> I think it may be of particular interest to Perl programmers because Erlang offers a great solution to a problem which is very hard to tackle in Perl - reliable, simple paralell processing. I'm definitely excited to use Erlang in the future - I've already got ideas for more projects than I could possibly find time to complete. I think its utility will only grow as processors continue to add more cores. </p><p> As a side note, I read the book as a beta PDF, which I enjoyed a lot more than I thought I would. It was very convenient to be able to cut-and-paste code from the book into a running Erlang interpreter to see it work. I got several updated versions of the book as I was reading, but the changes weren't too disruptive. </p><p> I may write a longer review, perhaps after I've written a moderate amount of Erlang so I can get a sense for what was left out. But don't wait for that - buy it now and thank me later! </p><p> -sam</p> samtregar 2007-04-26T18:46:01+00:00 journal [Smolder Grant Update] XML = TAP <p> After battling some sickness and a little bit of yak shaving, I've finally started work on my <a href="">grant for smolder</a>. </p><p> The first deliverable is to replace the custom XML schema that I used (which was based on the internal representation of a test run used by <a href="">Test::TAP::Model</a>. I know, I know... this was a really bad thing to do, but so it goes (and yes, Test::TAP::Model has changed their internal representation). I want to replace the XML with straight TAP, but there are a few complications. TAP doesn't currently support multiple files. Each TAP "stream" correlates to one test file and the harness keeps track of the results to show the viewer at the end. Since I want smolder users to upload 1 file per test run I need something to contain them. I could wrap all those TAP streams in some container format (YAML, JSON) but then I'm no better of than the XML route I currently have. </p><p> So I've decided that smolder will take a simple archive file (tar/tar.gz initially) which contains the various TAP files. Since we sometimes want metrics like "How long did this test run take" or "What order where these files run in" (since it doesn't have to be alphabetical) then there will also be an optional YAML (or JSON I haven't decided) file in there with this meta information. </p><p> So the first step was ripping out all the XML stuff currently in Smolder and it's reliance on Test::TAP::Model and friends. Then to replace it with a TAP parser (using <a href="">TAP::Parser</a>). This is currently about half way done, and I hope to find the time to finish it this week. </p> mpeters 2007-04-23T13:50:28+00:00 journal Article about Data Warehousing with MySQL and Perl An article I wrote about building a data warehouse with Perl and MySQL just went up on O'Reilly's database site:<blockquote><div><p> <a href=""><nobr>w<wbr></nobr> ith-mysql-and-perl.html</a></p></div> </blockquote><p> I'd intended it for <a href=""></a>, but as the editor correctly pointed out it doesn't have a lot of Perl content. Take a look and let me know what you think (|grep -v 'mysql hate'). </p><p> -sam</p> samtregar 2007-04-20T19:18:52+00:00 journal TPF Grant for Smolder and OSCon <p> I'm pretty excited that my TPF Grant for improving <a href="">Smolder </a> has <a href="">been accepted</a>. Some of the improvements are minor, but others are pretty big. I've been watching the progress of <a href="">TAP::Parser</a> with enthusiasm since it means I can get rid of the YAML/XML transport hack that Smolder currently uses to get test results into the server. Hopefully this will also give some evangelism for TAP as a nice, simple and really human readable format for testing. </p><p> The grant also comes at a good time because my proposal for OSCon this year was accepted and I'll be talking about an updated testing toolkit for web applications. So Smolder will be all nice and shiny from the grant work before the talk. Which will be a nice chance to show it off and hopefully get more people using it. </p><p> I'll be updating this journal pretty regularly as I work on this grant, so stay tuned if you're interested. </p> mpeters 2007-04-01T15:37:43+00:00 journal Decoding another bash error <p> Again I've been confused by a bash error message. And again, Google was little help, so I figured I'd post it here so perhaps others won't search in vain. </p><p> Here's the error, encountered trying to run a Perl script on a USB drive from my Fedora Core 5 machine: </p><blockquote><div><p> <tt>$ bin/krang_ctl restart<br>-bash: bin/krang_ctl:<nobr> <wbr></nobr>/usr/bin/perl: bad interpreter: Permission denied</tt></p></div> </blockquote><p> The "bad interpreter" part led me on a mission to make sure Perl was installed ok. It was, and scripts in other locations ran fine. Then I spent some time investigating the "Permission defined" angle, but I couldn't find a permissions problem anywhere. </p><p> Finally I looked at how the USB disk was mounted and there it was: </p><blockquote><div><p> <tt># grep Cube<nobr> <wbr></nobr>/etc/mtab<br>/dev/sda1<nobr> <wbr></nobr>/media/Cube ext3 rw,nosuid,noexec,nodev 0 0</tt></p></div> </blockquote><p> Note the "noexec" there! Fedora auto-mounted this disk with "noexec" turned on. I still don't know how to tell Fedora not to do that, but I do know how to fix it after the fact: </p><blockquote><div><p> <tt># mount<nobr> <wbr></nobr>/media/Cube/ -o remount,exec</tt></p></div> </blockquote><p> After that the problem went away. How the heck having "noexec" on a mounted filesystem triggers a "bad interpreter" error in bash, I have no idea... </p><p> Hope that helps someone! </p><p> -sam</p> samtregar 2006-12-08T18:52:36+00:00 journal Mac line-endings and Text::CSV_XS <p> To say that <a href="">Text::CSV_XS</a> has trouble with Mac line-endings (\015) is somewhat of an understatement. Not only will it not parse a file that uses them to end lines, it won't even allow them inside a field in binary-mode. Binary-mode is advertised as allowing any character as long as it's in a quoted string, so this is clearly (in my opinion) a bug. </p><p> I dug into the code intending to solve both problems, but only managed to fix the latter. Actually supporting \015 as a line-ending character looks like it would be hard. For my purposes it wouldn't help unless it was automatic - if I have to tell Text::CSV_XS that a file has Mac line-endings then I might as well just translate them. That's the way Unix and Windows line-endings work now - you don't have to tell the module what to expect and you can even mix them in a single file. The way it accomplishes this feat doesn't extend well though, at least as far as I can tell. </p><p> In any case, here's the bug fix: <a href="">mac.diff</a>. After you apply it you should find that stray \015 characters work just fine in binary mode. I also sent it to the maintainer, but since the module hasn't had a release in 5 years I'm not exactly holding my breath! </p><p> I came very close to going on an optimization mission while I was in the code. The state machine looks like it could benefit from some tweaking and the way lines are read looks like it could be improved. This would be pretty foolish though - Text::CSV_XS is already so fast that I've never seen it show up in a profile on a serious app. Usually I'm reading CSVs so I can load data into a database via DBI, by which point Text::CSV_XS is unlikely to be a bottleneck. </p><p> -sam</p> samtregar 2006-10-29T22:44:29+00:00 journal Smolder Slides <p> I gave the first presentation on <a href="">smolder</a> at the first Pittsburgh Perl Workshop on Saturday. It was a nice little conference that was well worth the money (<em>and not just because it was free to speakers</em><nobr> <wbr></nobr>:) </p><p> In case others are interested and missed it, my slides are <a href="">found here</a>. </p> mpeters 2006-09-25T18:37:47+00:00 journal Safely timeout DBI with DBIx::Timeout Recently I needed to find a way to timeout a DBI request. I found the state of the art less than satisfying, involving unsafe signals and a chance of memory corruption deep in Perl's guts:<blockquote><div><p> <a href=""><nobr>s<wbr></nobr> </a></p></div> </blockquote><p> This led me to create <a href="">DBIx::Timeout</a> which instead of using unsafe signals: </p><blockquote><div><p> <tt>&nbsp; - Forks a child process which sleeps for $timeout seconds.<br> <br>&nbsp; - Runs your long-running query in the parent process.<br> <br>&nbsp; - If the parent process finishes first it kills the child and<br>&nbsp; &nbsp; returns.<br> <br>&nbsp; - If the child process wakes up it kills the parent's DB thread and<br>&nbsp; &nbsp; exits with a code so the parent knows it was timed out.</tt></p></div> </blockquote><p> Tim Bunce suggested a possible optimization - fork just one child process and have it watch any number of slow queries simultaneously. It would accept assignments via a pipe interface. Seems like a good idea, although it's likely overkill for my usage. The queries I need to timeout are very likely to be long-running, and when they're not don't need to be very fast. The overhead of forking a process that exits almost immediately won't cause any problems, I'm betting. </p><p> So, please give it a try! And if you're not a MySQL user, please port it to your DB and send me a patch. It should be an easy job - all you have to do is implement a call to kill another process in the DB (MySQL does it with "KILL $thread_id"). (<b>UPDATE:</b> actually, it's a little more work - you also need to write new tests. The tests I wrote for MySQL use GET_LOCK() to test timeouts - you'll need to do something analgous for your DB.) </p><p> -sam </p><p> PS: I should note that the mechanism this module uses was suggested by my co-worker <a href="">Perrin Harkins</a>. I'll add that to the module's POD for the next release.</p> samtregar 2006-09-17T19:28:29+00:00 journal