inkdroid's Journal inkdroid's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:11:53+00:00 pudge Technology hourly 1 1970-01-01T00:00+00:00 inkdroid's Journal troll Me talk troll one day. inkdroid 2004-11-04T02:27:36+00:00 journal boston Hey who put that butt ugly Boston Red Sox icon in the use.perl logo? inkdroid 2004-11-04T02:23:15+00:00 journal wishlists on the cheap <p>I finally got around to documenting and making <a href="">available</a> my Amazon wishlist tool. If you maintain a wishlist and want to periodically see if any items on it are available for cheap you can put this in your cron:</p><blockquote><div><p> <tt>0 * * * *<nobr> <wbr></nobr>/usr/local/bin/wishlist --token=MYTOKENID --wishlist=MYWISHLISTID</tt></p></div> </blockquote><p>Which means at the top of every hour wishlist will go and look at my wishlist and email me if any items on it are available for cheap: cheap being less than 50% of the coverprice and/or less than $5. Cheapness is configurable. It uses Amazon's Web Services, so you'll need to get a token (free). I imagine that there's some CPAN module that does this, but there wasn't at the time when I wrote this about a year ago (I don't think).</p><p>If you use it you'll get lots of hits from used bookstores around the country. But I don't mind used when the quality is good, and I've been really happy with the quality measures on Amazon.</p> inkdroid 2004-03-05T17:06:47+00:00 journal HTTP::Server Gotcha, so this is python:<blockquote><div><p> <tt>import BaseHTTPServer<br> <br>class HelloWorldServer( BaseHTTPServer.BaseHTTPRequestHandler ):<br>&nbsp; &nbsp; def do_GET( self ):<br>&nbsp; &nbsp; &nbsp; &nbsp; self.send_response( 200 )<br>&nbsp; &nbsp; &nbsp; &nbsp; self.send_header( "Content-type", "text/html" )<br>&nbsp; &nbsp; &nbsp; &nbsp; self.end_headers()<br>&nbsp; &nbsp; &nbsp; &nbsp; self.wfile.write( "hello world" )<br> <br>server = BaseHTTPServer.HTTPServer( ( '', 8080 ), HelloWorldServer )<br> <br>while 1:<br>&nbsp; &nbsp; try: server.handle_request()<br>&nbsp; &nbsp; except: break</tt></p></div> </blockquote><p>Creates a non-forking HTTP server which responds to requests with "Hello World". I could get used to that. Right tool for the right job and all.</p> inkdroid 2004-03-05T16:32:11+00:00 journal Open Archives Ariadne just went live with my <a href="">article</a> about using the <a href="">Open Archives Initiative</a> Protocol for Metadata Harvesting (OAI-PMH) from Perl using <a href="">Net::OAI::Harvester</a>. OAI-PMH is basically a protocol for sharing metadata using HTTP/XML. Net::OAI::Harvester aims to be LWP::UserAgent for the OAI-PMH. The module was fun to write since I got to know XML::SAX better, started to grok how SAX filters work, and spent some time coming up with a class hierarchy that mirrors the protocol. Another fun part was finding a way to provide iterators that return objects based on parts of the potentially huge XML documents. At first I serialized objects on disk while SAX parsing, but used YAML, but in the end went with Storable since UTF8 support still isn't there for YAML. inkdroid 2004-02-02T18:13:01+00:00 journal XP pre-history I can't remember if I saw <a href="">this</a> article cited here on use.perl, but I printed it out months ago and finally got around to reading it. I had no idea that there was such a rich history of iterative development that stretches back to the 1950s...way before XP was a glimmer in Kent Beck's eye. The piece kind of reminded me of what Damian said once when visiting about how much of XP is just good common sense, and how it really isn't "extreme" at all<nobr> <wbr></nobr>:-) inkdroid 2004-01-28T22:48:55+00:00 journal chop chop Just ran across Dave Thomas's <a href="">katas</a> courtesy of the <a href="">software MFA</a> list. I haven't practiced any of them yet, but they look interesting. The software-mfa list has been picking up lately due to a trial run of the program down at the University of Illinois. It's basically taking Richard Gabriel's <a href="">ideas</a> about the convergence between art and software and putting them into practice. If you are interested there is a <a href="">wiki</a> for the trial run. inkdroid 2004-01-21T20:15:32+00:00 journal xISBN <p> <a href="">xISBN</a> is a neat service from <a href="">OCLC</a> in libraryland. Feed it an ISBN and get back a list of ISBNs that are "related". Relatedness in this case means associated editions or printings.</p><p>So, the latest camel has an ISBN of 0596000278. This link:</p><blockquote><div><p></p></div> </blockquote><p> Will return this list:</p><blockquote><div><p>0596000278 1565921496 0937175641 1565929748 1565924622</p></div> </blockquote><p> Which are the following: </p><ul> <li> <a href="">1565921496</a> </li><li> <a href="">0937175641</a> </li><li> <a href="">1565929748</a> : no longer at Amazon<nobr> <wbr></nobr>:(</li><li> <a href="">1565924622</a> </li></ul> inkdroid 2004-01-21T17:08:58+00:00 journal wikipedia I saw in Tim Bray's <a href="">journal</a> that the Wikipedia is looking for donations to upgrade their hardware. I donated $25, mainly because it's such a cool project, and I maintain <a href="">WWW::Wikipedia</a>. I hope this little bit helps. inkdroid 2003-12-29T01:15:43+00:00 journal EFF A good time of year to say <a href="">thanks</a> to the <a href="">EFF</a>. inkdroid 2003-12-22T18:32:02+00:00 journal Pragmatic Version Control I emailed the <a href="">Pragmatic Programmers</a> (Dave Thomas and Andy Hunt) to see if they would send copies of the <a href="">Pragmatic Starter Kit</a> to our <a href=""></a> group for review. Next day I got an email from Dave saying that the first two volumes are on the way. I just got Pragmatic Version Control, and Pragmatic Unit Testing last Friday, and I quickly devoured the first. In fact, the <a href="">review</a> is finished as well. inkdroid 2003-11-17T22:02:17+00:00 journal electronic voting <a href=";item=2821">EFF's Action Center</a> gives you a quick way to send your House Rep a fax or email letting them know that you want to cast your votes using an <i>open</i> system, not a blackbox that could be riddled with bugs and security holes. On the EFF site there is more information about the issue. Included is a link to a <a href="">paper</a> by some folks at Johns Hopkins and Rice who review the <a href=";script=410&amp;layout=-6&amp;item_id=433030">Diebold </a> system that was used in Georgia state wide elections in 2002, and is getting more and more business since the last Presidential election. Diebold's source code was <a href=",1848,59925,00.html">obtained</a> and posted to the net by a hacker who exploited a vulnerability in a webserver (any guesses) and gained access to their corporate "intranet".<blockquote><div><p> <i> Our analysis shows that this voting system is far below even the most minimal security standards applicable in other contexts. We highlight several issues including unauthorized priveledge escalation, incorrect use of cryptography, vulnerabilities to network threats, and poor software development processes. For example, common voters, without any insider privileges, can cast unlimited votes without being detected by any mechanisms within the voting terminal. </i></p></div> </blockquote><p>I like the idea of voting electronically...but if we're going to do it, let's do it <a href=",1272,61045,00.html?tw=wn_tophead_9">right</a>.</p> inkdroid 2003-11-12T15:22:15+00:00 journal de-transliterate I've been <a href="">working</a> off and on with Jane Jacobs at <a href="">Queens Public Library</a> in New York City to help them with some data munging. QPL has lots of patrons who speak Russian, and a large collection of Russian materials. Unfortunately their catalog data has transliterated titles, authors, subjects etc. This means the original Cyrillic has been romanized so that cards (and more recently records in a database) can be sorted and displayed. Now that there's Unicode, these Cyrillic characters can be sorted and displayed pretty easily. So I used Perl with <a href="">MARC::Record</a> to detransliterate their data, which was then imported back into their online catalog. Now patrons who read Russian can search for and read bibliographic records for Russian titles <b>in Russian</b>. It's kind of neat to see your name in the news, even if it is just a tiny blip in the days events. I just wish they mentioned Perl in the press release<nobr> <wbr></nobr>:) inkdroid 2003-11-11T23:56:30+00:00 journal chicago/perl Last night's <a href=""></a> meeting went very well. Lots of new faces, and people with interesting backgrounds...and I learned a thing or two about command line switches that I had forgotten after reading <a href="">Data Munging with Perl</a>. Andy's presentation should be available soon <a href="">here</a>. I spent some time after the meeting encouraging some people to volunteer to talk at a future meeting: in particular I'd really like to hear <a href="">hachi</a> talk about <a href="">POE</a> sounds like he's involved with some interesting work with it at <a href=""></a>. Also, there was a guy (I'm terrible with names) from <a href="">Northwestern</a> who is doing some interesting genetics work with <a href="">BioPerl</a>, <a href="">GenBank</a> and some slime. Steven Lembark volunteered to talk about debugging Perl next month, which should be interesting as well. Jason, Steven and I had some beers at a nearby microbrewery. They had a nice India Pale Ale, which helped since Steven got talking about the differences between swapping/paging until my brain hurt<nobr> <wbr></nobr>:) All good fun. Having the new space with the projector (and pool table) really helped alot. I think this marks the beginning of good things to come. inkdroid 2003-11-05T16:34:43+00:00 journal lanl <p>I got an email from Patrick Hostenbach at <a href="">Los Alamos National Laboratory</a> asking for me to tweak <a href="">Net::OAI::Harvester</a> a bit. His OAI-PMH repository is particularly strict and it turns out N::O::H was ocassionally passing a name/value pair that caused problems.</p><p>I'm really happy to hear LANL is using the module since Herbert Van de Sompel at LANL was one of the chief architects of the <a href="">OAI-PMH</a>. It was devised partly as a way for people to harvest metadata from the <a href="">arXiv</a> eprint archive.</p> inkdroid 2003-10-24T19:24:06+00:00 journal planck I've been working with Martin Emmerich at the <a href="">Max Planck Society</a> to get the <a href="">Net::OAI::Harvester</a> docs up to snuff. We've even turned up a few not so minor bugs, and added some missing unit tests. It turns out their <a href="">eDoc</a> server has an OAI-PMH interface. eDoc is "an easy to use electronic platform to document, share, manage, archive, disseminate, publish scientific information" of the Max Planck society. Thanks CPAN. inkdroid 2003-10-22T20:51:06+00:00 journal album-a-day I didn't manage to make it to last night's meeting. But I did happen to visit Michael Graham's website. Michael was visiting from I learned about the <a href="">Album-a-Day</a> project, which is kind of a neat idea. If you are looking to listen to some random/free music it's worth a visit. I didn't see it, but Spin did a <a href=";n=23">writeup</a> on the project. I also happen to use Michael's Palm::Progect, which works pretty well for me. Thanks Michael. inkdroid 2003-09-16T16:13:45+00:00 journal psh <p>On a tip from autrijus I took a look at <a href="">psh</a> again (perl shell). Last time I looked it didn't catch my fancy...but my how things have changed. The full power of a familiar shell, (cmd line completion etc) plus all the power of Perl at your fingertips. I feel like psh is going to make the amount of one off programs that are scattered all over the file system rapidly diminish.</p><p> Just as an example, I wanted to look at the comments inside some image files I had.</p><blockquote><div><p> <tt>% use Image::Magick;<br>% $i = Image::Magick-&gt;new();<br>% forfiles *.jpg ( $i-&gt;read( $_ ); print $i-&gt;get( 'Comment' ),"\n"; }</tt></p></div> </blockquote><p>That's all from the command line, and this is just the tip of the iceberg! psh is the most fun I've had in Perl for a while.</p> inkdroid 2003-09-14T01:36:07+00:00 journal eff &amp; sco Thanks EFF for making <a href=";item=2775">this</a> easy. inkdroid 2003-08-29T11:29:51+00:00 journal mjd on patterns I just discovered mjd's piece <a href="">"Design Patterns" Aren't</a>....I need to really read the GoF book before I can make an educated comment, but it seems to me that MJD is right on about the larger meaning of Alexander's work. inkdroid 2003-08-28T14:21:15+00:00 journal timeless way of building (2) <p>In my last journal entry I found it difficult to summarize what Alexander meant by the "quality without a name" because he expresses the idea so well himself, and the idea (by definition) kind of defies description. The same can be said for the second part of the <a href="">Timeless Way of Building</a> entitled "The Gate". This section of the book outlines the idea of a pattern language, which is the main ingredient in creating places, buildings, towns (software perhaps) which have the quality without a name. He does such a masterful job at describing what a pattern language is, that it is hard to review it here...but here it goes anyway.</p><p>The basic idea is that during the act of building a builder will assemble component pieces to create a new work. These component pieces are themselves made up of component pieces. The new work is itself just a part of a larger whole, which is being built in components. The components and their relationships together make up the patterns of a 'pattern language', which is shared by a community of builders. These components or elements of the language can include both physical objects (town square, religious center, main street), and also events that take place in these places (markets, weddings, walking). In fact the places and events are inextricably bound together. The 'pattern language' is the complete set of options that a builder has available when building. The languages vary from place to place, time period from time period. And as you might expect pattern languages aren't always used explicitly, but are often used implicitly all the time by builders. Choices are always made, patterns are copied from various places, and are incorporated in new ways. Most importantly some patterns (and hency pattern languages) are able to generate and sustain life, and others which inhibit and end up destroying life.</p><p>This is where things get interesting, the difference which makes a pattern (and the language it is a part of) alive. Alexander argues that the specialization which makes much of modern life possible also encourages a dislocation in the components of a pattern language:</p><blockquote><div><p> <i> If I build a fireplace for myself, it is natural for me to make a place to put the wood, a corner to sit it, a mantel wide enough to put things on, an opening which lets fire draw.<br> <br> But, if I design fireplaces for other people--not for myself--then I never have to build a fire in the fireplaces I design. Gradually my ideas become more and more influenced by style, and shape, and crazy notions--my feeling for the simple business of making fire leaves the fireplace altogether.<br> <br> So, it is inevitable that as the work of building passes into the hands of specialists, the patterns which they use beocme more and more banal, more willful, and less anchored in reality. </i></p></div> </blockquote><p> It's extremely important that patterns remain useful and rooted to their purpose, or else they risk becoming abstracted, useless and dead. </p><p>Another point Alexander makes reminded me of the Perl philosophy, and open source in general, since pattern languages must be shared if they are to be alive.</p><blockquote><div><p> <i> The acts of design which have been thought of as central are acts which use the structure already present in these underlying languages to generate the structure of specific buildings.<br> <br> In this view, it is the structure of the underlying language which is doing most of the hard work. If you want to influence the structure of your town, you must help to change the underlying languages. It is useless to be innovative in an individual building, or an individual plan, if this innovation dos not become part of a living pattern language which everyone can use.<br> <br> And we may conclude, even more strongly, that the central task of 'architecture' is the creation of a single, shared, evolving, pattern language, which everyone contributes to, and everyone can use.<br> <br> </i></p></div> </blockquote><p>Alexander goes on to outline how to explicitly describe a pattern as a rule so that it may be shared. Essentially, all patterns have the form:</p><blockquote><div><p> <i> Context --&gt; System of Forces --&gt; Configuration </i></p></div> </blockquote><p> Where context is the arena in which the pattern operates, the system of forces is the dynamic which the pattern is attempting to resolve, and the configuration is the resolution to the conflict. This method of description will make more sense if you read the book. It's not really relevant, but I couldn't help but be reminded of the idea of a tuple in RDF where each statement has the form:</p><blockquote><div><p> <i> Resource -&gt; Property -&gt; Value </i></p></div> </blockquote><p> I think this is because I'm enjoying O'Reilly's new <a href="">RDF</a> book at the moment too<nobr> <wbr></nobr>:-) At any rate after Alexander provides details about how to describe a pattern it is very clear what he means by it. He stresses that it is hard work finding these deep life sustaining patterns, but he provides instructions on how to go about it. What's more these instructions use architecture as the model, but they seem to abstract so well to software development. It's no wonder that this book inspired others too look for patterns in software. In building construction patterns had been shared implicitly for a long time, but the specialization and professionalization of architecture have lead to these patterns being dropped in favor of ones that could be expressed as blueprints etc. What's more these blueprints are guarded as business secrets, and are not shared. All this leads to a breakdown in the patterns, and the overarching pattern language that they make up. The process of discovering and sharing patterns in an unambiguous way is a technique for countering this trend. "The Gate" was every bit as fascinating as "The Quality", and I've still got one more section to go...up next "The Way". </p> inkdroid 2003-08-20T11:26:26+00:00 journal POD in PHP I work with PHP and Perl alot during the day. I've been internally bewailing the fact that there is no POD like documentation format for PHP. At least none that comes standard, as POD and javadoc do for Perl/Java. Then I was looking closely at one of the PHP class files I was editing and noticed that someone (it had to be <a href="">petdance</a>) had done this:<blockquote><div><p><nobr> <wbr></nobr><tt>/*<br> <br>=head1 NAME<br> <br>Page - A base class for all HTML pages.<br> <br>=cut<br> <br>*/<br> <br>class Page {<br> <br>/*<br> <br>=head2 Page()<br> <br>The constructor for the Page object. Pass in the args, etc, etc.<br> <br>=cut<br> <br>*/<br> <br>function Page() {<br>&nbsp; &nbsp; &nbsp;... do PHP stuff here...<br>}<br> <br>...</tt></p></div> </blockquote><p>So it's PHP but it has POD embedded in it within PHP multiline comments. So then you can run perldoc on the PHP file, and kazaam you get a nice POD document on the screen. I have no idea why this never occurred to me before, the solution is so simple it's pure genius<nobr> <wbr></nobr>:) Thanks Andy, this made my day.</p> inkdroid 2003-07-31T18:43:51+00:00 journal timeless way of building (1) <p> Recently I've been reading about design patterns. The thing I really like about them is that they provide guidance in finding successful ways to solve problems with workable models. In my reading so far I kept running across the name Christopher Alexander, and I decided to pick up his book <a href="">The Timeless Way of Building</a>. Alexander is an architect and mathematician, whose ideas about pattern languages largely inspired the design pattern movement in computer programming. I really enjoyed reading jplindstrom's <a href="">notes</a> about the Mythical Man Month, so I thought I could post some of my notes about the Timeless Way of Building here. </p><p> When I first began seeing myself as a professional programmer I was working in New York City. On my way to and from work I would often pass all sorts of construction projects. Sometimes I would see workmen assembling scaffolding to make structural repairs to a building, or drilling into the road revealing a vast network of cables and pipes. The act of creating new structures, and fixing existing ones in the city's complexity somehow reminded me of programming. For some reason I enjoyed thinking about programming as a type of building...especially all the craft that goes into it. So it wasn't a stretch for me to pick up a Timeless Way of Building. </p><p> The Timeless Way of Building has an interesting format. It is broken down into three parts: The Quality, The Gate, The Way. Each section is then broken down into chapters, and each chapter is made up of brief sections, each section having it's own 'headline'. Alexander designed the book this way so you can read large chunks quickly to get the feeling of the book as a whole. </p><p>The title says the book is about building, but as you might guess from three parts, and the "timeless" adjective there is a lot of philosophy thrown into the mix. This is especially true in the first few chapters where Alexander starts out talking about the elusive "quality without a name"; which is present in those moments we experience when things just seem "right". He is able to talk about this quality by approximating it with words like "alive", "whole", "comfortable", "free", "exact", and "eternal"; while showing how the approximations don't completely describe the quality without a name. He goes on to examine how places can exhibit this same quality, and how each place and building is actually a collection of elements and relationships among those elements...which he calls patterns. For example:</p><blockquote><div><p>Consider a typical mid-twentieth-century American metropolitan region. Somewhere towards the center of the region, there is a central business district, which contains a very hig density office block; near these there are high density apartments. The overall density of the region slopes off with distance from the center, according to an exponential law; periodically there are again peaks of higer density, but smaller than the central ones; and subsidiary to these smaller peaks, there are still smaller peaks. Each of theses peaks of density contains stores and offices surrounded by higher density housing. Towards the outer fringe of the metropolis there are large areas of freestanding one family houses; the farther out from the center they are, the larger their gardens. The region is served by a network of freeways. These freeways are closer together at the center. Independent of the freeways, there is a roughly regular two dimensional network of streets. Every five or ten streets, there is a larger one, which functions as n artery. A few other arteries are even bigger than the others: these tend to be arranged radially, branching out from the center in a star-shaped fashion. Where an artery meets a freeway, there is a characteristic cloverleaf arrangement of connecting lanes. Where two arteries intersect, there is a traffic light; where a local street meets an artery, there is a stop sign. The major commercial areas, which coincide with the high density peaks in the density distribution, all fall on the major arteries. Industrial areas all fall within half a mile of a freeway; and the older ones are also close to at least one major artery.</p></div> </blockquote><p> In much of the early chapters Alexander is training the reader to look for patterns. He also begins to examine patterns that are 'alive', or self sustaining, and those that are 'dead' or destructive. What's more the patterns of a place are inextricably bound up with the activities that take place there.</p><blockquote><div><p>The town which is alive, and beautiful, for me, shows, in a thousand ways, how all its institutions work together to make people comfortable, and deep seated in respect for themselves.<br> <br> Places outdoors where people eat, and dance; old people sitting in the street, watching the world go by; places where teenage boys and girls hang out, within the neighborhood, free enough of their parents that they feel themselves alive, and stay there; car places where cars are kept, shielded, if there are many of them, so that they don't oppress us by their presence; work going on among the families, children playing where work is going on, and learning from it.<br> <br> And finally the quality without a name appears, not when an isolated pattern lives, but when an entire system of patterns, interdependent at many levels, is stable and alive.<br> <br> A building or town becomes alive when every pattern in it is alive: when it allows each person in it, and each plant and animal, and every stream, and bridge, and wall and roof, and every human group and every road, to become alive in its own terms.<br> <br> And as that happens, the whole town reaches the state that individual people sometimes reach at their best and happiest moments, when they are most free.</p></div> </blockquote><p> It is very difficult for me to describe the effect that this first section "The Quality" had on me. Near to the end of the section I found myself being reminded of what I enjoy so much about the Perl programming language: the vibrant community, diversity of interests, the wealth of CPAN where programmers can share work together...all of these just seem very full of life. I'm looking forward to the next section "The Gate" where Alexander asks:</p><blockquote><div><p>Is there a fluid code, which generates the quality without a name in buildings, and makes things live? Is there some process which takes place inside a person's mind, when she allows herself to generate a building or a place which is alive? And is there indeed a process which is so simple too, that all the people of society can use it, and so generate not only individual buildings, but whole neighborhoods and towns? It turns out there is. It takes the form of language.</p></div> </blockquote><p>The most refreshing thing about this book so far is that it provides a lens for thinking about programming and design that is outside of the ordinary realm of computer science. I can't wait to get on to the next section.</p> inkdroid 2003-06-22T16:07:27+00:00 journal my $day = YAPC::NA-&gt;new( day =&gt; 3 ); $day-&gt;print(); <ul> <li>My last day in Boca started out listening to <b>Piers Cawley</b> talk about refactoring. Refactoring is the process of improving the design of some code, without changing its functionality. Refactoring also involves the art of looking for repetition in your code, and eliminating it. Piers spoke a bit about refactoring in general, and then set out to refactor some of his own code (I think it was Class::Builder). He used the audience as his colleague pair programmer, and people were more than happy to point out missing parens, or quotes. Piers used <a href="">Test::Class</a> to write his tests. Apparently Test::Class is very much like the JUnit framework (which my colleague <a href="">Mike O'Regan</a> really likes alot). I don't know much about JUnit, so I was interested to learn that the original was SUnit written by Kent Beck for Smalltalk. I need to check out Test::Class now. It seemed to me that Piers was talking more about testing than refactoring, but perhaps the two are so intertwined it's impossible to talk about one without the other.</li><li>Next I headed off to hear <b>Peter Chines</b> talk about exceptions. Unfortantely I missed a significant chunk of the beginning of the talk. Peter went over the various standard functions for throwing exceptions and warnings in Perl (croak, carp, confess) and provided guidance on why they were important to use. He also showed how signals could be used to automatically add information to all die messages (a handy trick) and talked about CGI::Carp which automatically logs the name of the generating program, and a timestamp to STDERR. As <a href="">2shortplanks</a> pointed out, it would've been nice to see some examples of throwing objects as exceptions. And someone piped up at the end about Damian's Coy module which translates Perl's regular messages into soothing haikus.</li><li>Directly following Peter was <b>Mark Fowler</b> talking about extending Template::Toolkit. Mark motored through an amazing amount of material about TT. The essentials that I took away is that it's remarkably easy to extend TT by subclassing Template::Plugin. For some reason I find the TT framework much easier to grok than Mason. Perhaps it is because the writing of the templates seems like a new language, totally independent of Perl. Perhaps the comparison doesn't hold since they really are quite different in some ways: Mason is an Apache application development environment, and TT is a more generalized templating system. I'm really looking forward to seeing the new ORA book on TT.</li><li>After lunch I headed over to hear <b>Ken Williams</b> talk about Machine Learning. This was a huge topic, and Ken really could've had a whole day to talk about this fascinating stuff. Ken summarized ML as any system that improves (or changes) as it receives training examples. Clustering, categorization, recognition, and filtering are all examples of ML systems. It becomes feasible to use ML when you have too much data to sift through, people are too slow at doing the sifting, and when you can afford to be wrong ocassionally. Perl is a useful ML language since you've got CPAN at your fingertips, it is a quick prototyping language (and you may not throw away the prototype<nobr> <wbr></nobr>:), and if you need it you can drop down to C via XS for speed. As an example Ken described how the use of decision trees could improve Spam Assassin. SA has over 600 attributes that it uses to identify a spam message. The attributes all have (+/-) weights associated with them, which when summed together will mark an email as spam if it exceeds a certain threshold. The problem with 600 attributes is that they all need to have rules associated with them, and these rules must individually be applied to come up with the final sum...which requires a fair amount of time and processing. Ken wrote a program that uses the 600 SA attributes, and a collection of spam/ham available from SA, to generate a decision tree to identify spam. The benefit of a decision tree being that it doesn't need to process each rule, but only a subset of rules that are tuned to each email as it comes in. Ken used the AI::DecisionTree module to do the hard stuff, and plugged the tree into GraphViz for displaying the nodes. Pressed for time Ken quickly went through an example of collaborative filtering, which most people experience when they go on Amazon and are presented with a list of recommendations. Ken wrote another program which used Search::ContextGraph to analyze the CPAN non-core modules that 30 people had installed. This allowed him to say if you have LWP::UserAgent installed, you are likely to also like URI<nobr> <wbr></nobr>:-) which makes sense. It would've been cool to 1/2 a day or even a full day of this stuff...I felt like we were just scratching the surface.</li><li>I managed to stay awake with the help of some sugar-rich cookies and hear <b>Michael Rodriguez</b> talk about XML Modules. He provided a really nice summary of a large chunk of the XML modules that are available on CPAN. They all descend from two C ancestors: expat, and libxml2. They are genearlly divided into two camps, event based parsers (SAX) which process the XML as it comes in and throw events (callbacks); and DOM based parsers which read the entire file into an in memory data structure and then allow you to query it. One of the things that Michael said which really stuck with me is that XML should really only be used as an interchange format, and not as a live data format. What he meant was that XML is really good for exchanging data with other people, but once you've got it you should parse it and get the data into your relational database ASAP. Another thing that I liked was his joke that XML was really the revenge of Java programmers who need angly brackets around everything so that they could parse text easily. Of course Java now has Perl-like regular expressions, so things have probably improved (or have they?).</li><li>Unfortunately I wasn't able to make it to Damian's closing talk because I had an early flight. I took the bus from FAU to the tri-rail, got off at the wrong station, had to take a bus into Ft. Lauderdale, and then another bus to the airport. It was good to get to see the city at any rate. It was cool to hear what I thought was French on the buses, and everyone seemed so relaxed and friendly...perhaps it's the fine weather. I finally got to the airport and my flight was delayed a couple hours, so I probably could've heard Damian speak afterall! Overall it was a great/informative time. Thanks Perl Foundation, and Follett for sending me this year.</li></ul> inkdroid 2003-06-19T16:18:31+00:00 journal my $day = YAPC::NA::-&gt;new( day =&gt; 2 ); $day-&gt;print(); <ul> <li>I spent the AM listening to <b>Arthur Bergman</b> talk about ithreads, Perl's threading mechanism. Arthur was a measured speaker, clearly comfortable with his material, and with being in front of a packed room full of hackers. I didn't know much about threads apart from what I skimmed in the latest camel. Apart from the basics here are the things I took from the talk. If you are using ithreads you must compile perl with them built in. If you want to use threads you should use the latest snapshot of perl (5.8.1 should be out "any day now"). Compiling in threads will cost you a 10% performance hit...even if you are writing a program that doesn't use threads explictly. mod_perl2 is using ithreads big time in support of Apache2's Multi Process Modules (MPMs)...and RT3.0 supposedly runs under MPM. Arthur got a Perl Foundation grant to get POE to work with ithreads, and also has plans to create a which will make all sorts of information about the running Perl interpreter available to Perl programs. So much stuff was covered, even a few scary dips into the Perl source. Best of all Arthur ended the session by asking the audience for suggestions on a simple threading application to write as an example. Several ideas were thrown around: a MUD server, a card shuffler, a customer request broker. Finally Ken Williams suggested that we write a voting application that would allow people to connect to a port on Arthur's presentation machine and make votes for the sample code to write. No sooner said than done, Arthur had is hacked out in 15 minutes, and then we could see as people connected and placed their votes. Unfortunately the fix was in, since one of the POE hackers wrote a program to cast a ton of votes for the MUD server.</li><li> After lunch I headed over to hear <b>Brian Ingerson</b> talk about extreme programming tools for module authors. Brian's talk was really refreshing since he clearly has alot of experience implementing extreme programming ideas, but he didn't seem to be sold on it as a religion...just as a set of practical techniques. He stresed agile strategies more than extremeness. But man, his presentation was extremely extreme. He presented his slides using CGI::Kwiki, using some extension he wrote for slideshows (which I think I heard him creating just the night before in the dorm lounge with some other folks). It was a trip, since folks in the audience were able to modify his slides before he got to them! At one point he hit next and came to <a href=";page_id=ExtremeProgrammingTools&amp;slide_num=13&amp;line_num=0">this</a> page (hit ENTER to reveal the various states of the page). There were so many good things in this talk, you can see the full contents <a href="">here</a>. Here are a few of the things that stuck in my head. The best things about extreme programming are: testing, saw sharpening (having good tools), refactoring, and collaboration. When writing modules you should: make it ridiculously easy to use, scale to any task, easy to install, few pre-requisites, have a good descriptive/catchy name, and the first release should be good enough, but not complete. Ingy also commented in an offhand way that you should imagine that you are writing Perl itself when you are writing a module for CPAN. I thought that was a simple and neat idea, in that you should try to emulate the flexibility, utility and friendliness of Perl when you write modules to extend it. He had some time to demonstrate the CGI::Kwiki module, and Module::Build which illustrated the concepts he covered in the beginning. I'd really like to set up a CGI::Kwiki for when I get back. Ingy graciously cut his session short so we could go to the lightning talks.</li><li>The lightning talks were as usual very humorous. Highlights for me were 3D madness in Perl by <b>Pierre Denis</b>, WWW::Mechanize by <b>Uri Guttman</b> (he mentioned my colleague Andy Lester), and of course the <b>Allison's</b> Restaurant sing along. You may have had to be there...but you can because some of the slides are <a href="">up</a> already. I just hope someone captured it on video<nobr> <wbr></nobr>:)</li></ul> inkdroid 2003-06-18T01:43:14+00:00 journal my $day = YAPC::NA-&gt;new( day =&gt; 1 ); Day 1 YAPC was great and am looking forward to more today. In a nutshell here's what I got to hear about yesterday: <ul> <li> <b>Damian Conway</b> talking about the Perl6 language. I've been subscribed to the perl6-lang list for a bit, but hearing Damian talk about the new stuff that is coming along is really helpful. He has so much enthusiasm and energy for the new language constructs, and it is contagious. No word yet on *when* Perl6 is going to be in alpha or beta, but perhaps this will come at his closing talk at the end.</li><li> <b>Dave Rolsky</b> talking about dates/times in Perl. This talk was fascinating, mainly because of the breadth of Dave's knowledge of dates, and the <a href="">date/time</a> Perl project which he organized (and which is coming along really well). I particularly liked the overview of the various time standards and history of calendars that Dave presented. And of course the comparison (strengths and weaknesses) and summary of existing date/time modules on CPAN (over 35 of them!) was invaluable. The main thing I took from the talk is that you never want to store offsets in the database ( -5:00 GMT, store "America/New York" instead), and that you want to use the date/time project (and help them extend it). I also had never heard of Olson's timezone database before, and never really thought how the 2038 32bit time bug will start to take effect rather soon in mortgage calculations<nobr> <wbr></nobr>:)</li><li>After lunch it was more <b>Damian</b> talking about how he has customized his personal work environment to achieve the goal of more laziness. First he talked about some cool macros that he wrote for optimizing his key strokes in Vi. Damian's point was not to use Vi or his macros, but to get in the mindset where you become aware of where you are spending your time, and then taking the time to write a tool, or an optimization. Also, he stressed that mantra that the computer should adapt to people, and that people shouldn't have to adapt to the computer. I need to check out Text::Autoformat (which he said people should look at), and he recommended OSX users take a look at a simple/free app called XShelf.</li><li>After that I headed over to hear <b>Chris Winters</b> talk about generating Java w/ Perl. He is working at a utility company that must use Java (for the usual reasons), and they have a huge database (500 tables), and needed a bunch of rudimentary classes. He was primarily a Perl programmer, and learning Java on the job, so he wrote Perl programs that wrote all the repetitive Java classes from metadata. I learned about the difference between passive code generators (for example Visual C++) whose output you can edit, and active code generators which have output you should not edit.</li><li>After that it was more <b>Rolsky</b> talking about advanced Masonry. I started fading near the end of the day, and was thinking about my <a href="">Open Archives</a> project a bit; so I only woke up enough near the end to ask how he saw Mason relating the TemplateToolkit</li><li>After that I went back to the dorms, did a little coding, and then went off to the Boston party to have a few drinks. Met some cool people, and fell asleep pretty late<nobr> <wbr></nobr>:)</li></ul> inkdroid 2003-06-17T13:05:21+00:00 journal wants <p>I want two CPAN modules, which may or may not exist:</p><p> <b>XML::Filter::Simple</b></p><p>An XML::SAX handler that does what XML::Simple does. So if I have some XML like this:</p><blockquote><div><p> <tt>&nbsp; &nbsp; &lt;foo a=1&gt;<br>&nbsp; &nbsp; &lt;bar&gt;baz&lt;/bar&gt;<br>&nbsp; &nbsp; &lt;bar&gt;bez&lt;/bar&gt;<br>&nbsp; &nbsp; &lt;/foo&gt;</tt></p></div> </blockquote><p>I would be able to do this:</p><blockquote><div><p> <tt>&nbsp; &nbsp; my $foo = XML::Filter::Simple();<br>&nbsp; &nbsp; my $parser = XML::SAX::ParserFactory-&gt;parser( Handler =&gt; $foo );<br>&nbsp; &nbsp; $parser-&gt;parse_string( $xml );<br> <br>&nbsp; &nbsp; ## same kind of data structure as XML::Simple<br>&nbsp; &nbsp; print $foo-&gt;{ a };&nbsp; &nbsp; &nbsp; &nbsp; # prints 1<br>&nbsp; &nbsp; print $foo-&gt;{ bar }[0];&nbsp; &nbsp; # prints baz<br>&nbsp; &nbsp; print $foo-&gt;{ bar }[1];&nbsp; &nbsp; # prints bar</tt></p></div> </blockquote><p> <b>Politics::US</b> </p><p>I want to be able to keep up to date with the goings on of my congressman, senators, and I want Perl to help me.</p><blockquote><div><p> <tt>use Politics::US::Senator;<br>use Politics::US::Bill;<br> <br>my $senator = Politics::US::Senator-&gt;new( 'Durbin, Richard' );<br>foreach my $vote ( $senator-&gt;votes() ) {<br> <br>&nbsp; &nbsp; &nbsp;my $decision = $vote-&gt;decision();<br>&nbsp; &nbsp; &nbsp;my $billNum = $vote-&gt;billNumber();<br>&nbsp; &nbsp; &nbsp;my $bill = US::Politics::Bill-&gt;new( $billNumber);<br> <br>&nbsp; &nbsp; &nbsp;print<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "Today Durbin cast a vote of $decision regarding $billNum\n",<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "The bill's title is: ", $bill-&gt;title(), "\n"<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; "And here is the content of the bill:\n",<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $bill-&gt;content();<br>}</tt></p></div> </blockquote><p>Between the <a href="">Senate</a> website, and <a href="">Thomas</a> and WWW::Mechanize this isn't so far fetched at all.</p> inkdroid 2003-06-14T21:07:00+00:00 journal pacman (eclipsed) Today's <a href="">Astronomy Picture of the Day</a> of the solar eclipse last week proves that the eclipse was really a video game simulation...just as I suspected. inkdroid 2003-06-06T18:01:05+00:00 journal attack of the killer hashes <p>By now you've probably already seen the news articles about this new form of DoS, which essentially makes the attacked computer do lots of work without bombarding it at the network layer. I didn't realize that the attacks were demonstrated on Perl regexes and hashes. I haven't read the article yet, but here's an excerpt from the <a href="">original</a>.</p><blockquote><div><p>We present a new class of low-bandwidth denial of service attacks that exploit algorithmic deficiencies in many common applications' data structures. Frequently used data structures have ``average-case'' expected running time that's far more efficient than the worst case. For example, both binary trees and hash tables can degenerate to linked lists with carefully chosen input. We show how an attacker can effectively compute such input, and we demonstrate attacks against the hash table implementations in two versions of Perl, the Squid web proxy, and the Bro intrusion detection system. Using bandwidth less than a typical dialup modem, we can bring a dedicated Bro server to its knees; after six minutes of carefully chosen packets, our Bro server was dropping as much as 71% of its traffic and consuming all of its CPU. We show how modern universal hashing techniques can yield performance comparable to commonplace hash functions while being provably secure against these attacks.</p></div> </blockquote> inkdroid 2003-06-03T15:50:35+00:00 journal data collection <p>The Wallstreet Journal has a nice <a href="">article</a> about data collection by the US Federal Government after 9/11. Interesting to note that the Total Information Awareness program has been renamed to the <a href="">Terrorism Information Awareness</a> program. Among the database initiatives covered is this story:</p><blockquote><div><p>Examples of how local police records can live on in federal databases are surfacing in Denver, where the police department recently released documents in response to a lawsuit by the American Civil Liberties Union. They show the police intelligence unit had secretly built a computer database full of personal details about people active in political groups. Included were a Quaker peace-advocacy group called the American Friends Service Committee and right-wing causes such as the pro-gun lobby. </p><p> The Denver department is purging people not suspected of a crime from the records. But last summer, when a man listed in the Denver files as a gun-rights group member got into a fender bender, a police officer checking VGTOF found him described as "a member of a terrorist organization" and part of a "militia." According to a Denver police memo, the officer reported the stop to the FBI as a "terrorist contact." The Denver police and the FBI decline to comment on how the man ended up in VGTOF.</p></div> </blockquote><p>Sure mistakes happen, but the problem is that restrictions on the accuracy of the data are being loosened as the amount of data is being increased. The loosening is probably so that TIA is legally allowed to link together these new data sources. Bruce Schneier has a nice piece <a href="">debunking</a> that whole idea with a bit of math. The <a href="">EFF</a> is doing a great job fighting this stuff in Congress. They have their work cut out for them, because it's hard to speak rationally when large amounts of fear are involved.</p> inkdroid 2003-05-23T11:44:12+00:00 journal