rats's Journal http://use.perl.org/~rats/journal/ rats's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-02-09T01:34:23+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 rats's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~rats/journal/ Bertrand Meyer: 7 Principles of Software Testing http://use.perl.org/~rats/journal/36404?from=rss I can't find a link for a really useful article I received today via email on software testing. (<a href="http://archive.eiffel.com/webforms/subscribe.html">Subscribe to Eiffelworld newsletter</a>) (No attempt to insert whole article here, but here's the 7 principles. Read the article for explanations.) <p>Principle 1: Definition of software testing <br>To test a program is to try to make it fail. </p><p>Principle 2: Tests versus specs <br>Tests are no substitute for specifications. One can derive tests from specifications, not the other way around. </p><p>Principle 3: Regression testing <br>Any failed execution must yield a test case, to remain permanently part of the project's test suite. </p><p>Principle 4: Applying oracles <br>Determining success or failure of tests must be an automatic process. </p><p>Principle 5: Manual and automatic <br>An effective testing process must include both manually and automatically produced test cases. </p><p>Principle 6: Empirical assessment of testing strategies <br>Evaluate any testing strategy, however attractive it may appear in principle, through objective assessment based on explicit criteria and a reproducible testing process. </p><p>Principle 7: Assessment criteria <br>The most important property of a testing strategy is the number of faults (or, if faults are not directly measurable, failures) it uncovers as a function of testing time.</p> rats 2008-05-13T12:29:20+00:00 journal Why I'm Passionate About Perl http://use.perl.org/~rats/journal/36403?from=rss <a href="http://use.perl.org/~brian_d_foy/journal/36356">brian d'foy</a> asks people to submit answers/comments to above topic. Here's my take: <ul> <li> <strong>The person who introduced me to Perl showed me that...</strong> No one person introduced me to Perl. I was attempting to install a manufacturing package using the Unify RDBMS and was finding the C API hair-tearingly difficult to use. A developer for the package mentioned there was a new language called Perl and there was Unify-Perl available. (This was in 1990.)</li><li> <strong>I first started using Perl to...</strong> talk to a Unify RDBMS.</li><li> <strong>I kept using Perl because...</strong> Perl did garbage collection and I'd spent 15 years fighting C's malloc and free. I didn't do anything but manage memory for my C programs and Perl came along and I suddenly had 90% more time to write real code, not manage memory.</li><li> <strong>I can't stop thinking about Perl...</strong> because the syntax seems to align with how my ideas form (or maybe my thinking is now so warped towards Perl I can't think any other way).</li><li> <strong>I'm still using Perl because...</strong> I haven't found anything sufficiently compelling in other languages to make me want to switch. I spent a few months wrangling Python for a living but felt only relief when I was able to switch back to Perl.</li><li> <strong>I get other people to use Perl by...</strong> teaching in-house courses, always being ready to answer any questions about Perl wherever I work, writing new apps in Perl if language is not specified.</li><li> <strong>I also program in<nobr> <wbr></nobr>...</strong> Python <strong>and<nobr> <wbr></nobr>...</strong> JavaScript, <strong>but I like Perl better as...</strong> I can write whole solutions to problems in a few lines in Perl. (Ever tried writing a 'one-liner' in Python? It ain't pretty<nobr> <wbr></nobr>:-)</li></ul> rats 2008-05-13T12:01:08+00:00 journal Programmer flow http://use.perl.org/~rats/journal/24061?from=rss <p>Yesterday was one of those days where I was in 'flow'. I was rewriting one of my early attempts. A changed requirement now forced me to generalise the process. Previously I used procedural code, copying an even earlier version of the script written by my predecessor here. </p><p> But yesterday I decided it had to be implemented with a reasonable design (and that means OO for me) instead of more ad-hocery on top of existing ad-hocery. I still wasn't sure how to implement my design but I started coding and put a stub class and/or method for everything which was still unclear. As I proceeded the design crystallised in front of me. The script is small enough to allow me to rewrite the whole class hierarchy within a couple of hours if necessary so I had no fear of refactoring and refactoring again. </p><p> When my alarm went off to remind me to go home, I realised I'd spent the whole day in 'flow', totally absorbed in the code. </p><p> It doesn't happen often but when it does it reminds me why I love programming. </p> rats 2005-04-08T01:10:50+00:00 journal OO as roadmap http://use.perl.org/~rats/journal/23559?from=rss <p>I remember a comment Bertrand Meyer made in <a href="http://archive.eiffel.com/doc/oosc/page.html">OOSC2</a> that OO design simply provides an orderly way of arranging all the bits and pieces of a program that get scattered all over the place in traditional/procedural programming. OOD gives a roadmap of where to find things.</p><p>A refactoring exercise yesterday is a classic example. The existing code seemed to be written by someone who was reluctant to use anything more complex than single index arrays and single key hashes to store the quite complex data being retrieved from the database. I replaced the single keys/indexes with equivalent [array]{hash}[array]{hash} structures. But then it became obvious I really needed to replace the structures with objects. The clincher was a 50-line sort routine which used splice()!! All the sort was doing was separating the jobs into today's and yesterday's and then sorting the two lists from latest to earliest but because of the tangled up structures it was really difficult to do this using 'sort grep @array'.</p><p>I created a Job class and then the sort reduced to two lines: one to select and sort the previous day's jobs; one to select and sort the current day's jobs. I then noticed six lines of code at the end of the old sort routine which calculated column breaks for displaying the jobs (and held the results in global variables!). Why were they there? Well I supposed they had to be somewhere and the sort routine was as good a place as any. Turns out the lines and global variables weren't even needed but you wouldn't realise this until the sort routine shrunk so dramatically.</p><p>I occasionally hear or read dismissive comments about OO. "I can write faster code using paradigm X or paradigm Y" sort of thing. But I use OO as a tool to help me order and arrange my thoughts. Thinking is hard and I'll use any help I can get. If, as a side-effect, I can reduce a 50-line chunk of code to 2 lines, well that's nice too.</p> rats 2005-03-09T04:20:45+00:00 journal devel heaven for curmudgeons http://use.perl.org/~rats/journal/23526?from=rss <p>My client has given developers a lot of liberty to set up the devel environment they like. Aside from the obvious Debian Linux and Apache, I've always used vi/vim for Perl code devel. I've tried some IDEs over the years but the learning curve was too low (i.e. too long to learn fully). I remember using ptkdb years ago and it was great but the next place I worked at wouldn't let me install Tk. So I've always tried to use the bare minimum toolset with Perl: vi/vim and perl -d.</p><p>However last year I decided that if I was going to use vim because it, like Perl, is available on every system I work with, I had better use it as efficiently as possible.</p><p>The first thing I looked at was the <a href="http://www.rayninfo.co.uk/vimtips.html">Best of Vim Tips</a>. Oh! if only I had read it years ago... </p><p>Then I decided to search <a href="http://www.vim.org/">http://www.vim.org/</a> for Perl plugins. Jackpot! I found <a href="http://vim.sourceforge.net/scripts/script.php?script_id=556">perl-support.vim</a> and <a href="http://vim.sourceforge.net/scripts/script.php?script_id=215">tagmenu.vim</a>. Now I could churn out Perl code as fast as I could think.</p><p>The only annoyance for me was that the keyboard shortcut for tagmenu clashed with one in perl-support. So I opened tagmenu.vim with gvim (Escher's drawing of the hand drawing the hand comes to mind) and changed the shortcut. This meant I had to maintain my own version of tagmenu and I never got around to troubling the author to change it but tagmenu hadn't changed for years (note that: I could have used it *years* ago) so it wasn't urgent. </p><p> Well, frabjous day!, as part of this recent install I've installed a more recent version of perl-support and it no longer clashes with tagmenu. All those years of using vi and vim now pay off because I can write and test my Perl code and my HTML code with the minimum of keystrokes (think 'RSI') and I rarely have to think which keys to use anyway. And I don't have to take my hand off the keyboard to use the mouse.</p><p>Of course, this only benefits crusty old curmudgeons like me who've built up the keyboard habits.</p> rats 2005-03-06T23:41:21+00:00 journal Scraping HTML with XML::LibXML http://use.perl.org/~rats/journal/23303?from=rss Writing a test script to hit a webpage and scrape out enough from the HTML response to verify it is correct... <p> First test is to (stop and) start my fake xmlrpc server with the response file I want and confirm it's alive. Hmmm. <a href="http://search.cpan.org/~rjray/RPC-XML-0.57/">RPC::XML</a> t/* tests do lots of that so let's steal/borrow some code. Hmmm. Net is down (firewall machine again probably). <a href="http://search.cpan.org/~rjbs/CPAN-Mini-0.36/">minicpan</a> to the rescue. Minicpan has saved my bacon so many times I've lost count... </p><p> Well that was relatively painless. <a href="http://search.cpan.org/~rjray/">Randy J Ray</a> writes nice clean <b>intelligible</b> Perl code. My script loads an RPC::XML::Server with the canned methods and forks it to a background process then gets a page from my web app to confirm the xmlrpc server is running correctly. </p><p> Now comes the fun. I hate HTML scraping but if I have to do it, I really like to use <a href="http://search.cpan.org/~phish/XML-LibXML-1.58/">XML::LibXML</a>. Aside from being very fast at parsing (which isn't important for this app), I can use XPath notation to navigate the DOM tree and, even better, there's <a href="http://search.cpan.org/~pajas/XML-XSH2-2.0.2/"> <b>xsh</b> </a> to let me try out my XPaths interactively. Yes it's possible to read the HTML code and keep track of how many levels of table/tr/td you are down by hand but why waste hours when with xsh you can do this in minutes. </p><p> Ouch! A small problem. LibXML expects xhtml and crashes all over the place when I ask xsh to parse the HTML output of my webapp. Lucky(!) for me (another reason for choosing CGI::Application) I have moved all the HTML from the old webapp into <a href="http://search.cpan.org/~samtregar/HTML-Template-2.7/">HTML::Template </a> templates. So it's really easy to rewrite it as xhtml using <a href="http://www.vim.org/">Vim</a>. (I discovered after rewriting by hand that one of the options for <a href="http://tidy.sourceforge.net/"> <b>tidy</b> </a> is --asxhtml. It outputs HTML as xhtml. Double d'oh!) </p><p> So now I've got clean xhtml output I can use xsh to navigate through the parsed tree and find the fields I expect to see in the page if the webapp is working correctly. The first one I want has an XPath of </p><blockquote><div><p><nobr> <wbr></nobr><tt>/html/body/table/tr[2]/td[2]/form/a[6]</tt></p></div> </blockquote><p> thanks to xsh. Glad I didn't have to work that one out by hand. It's a link to expand the tree. So I'll use WWW::Mechanize to click the link and grab the response and verify it returned the required number of tables in the correct order with the correct contents. And then on to the next test...</p> rats 2005-02-22T02:31:05+00:00 journal Displaying a directory tree in a webpage http://use.perl.org/~rats/journal/23298?from=rss The client's webapp appearance is modelled on directory tree displays. Clicking on icons expands the branch and clicking on icons in the branch expands the subbranch and so on. Likewise clicking on the icon at the top of an expanded branch closes/collapses the branch. Each icon is a button and places a call to the webserver. I initially thought I would replace the icons/buttons with Javascript because the response would be much faster and (I thought) there was no need to bother the webserver. So I looked at half a dozen examples available on the Net. Some people even want you to pay for their scripts. The <a href="http://www.webreference.com/programming/javascript/trees/index.html">neatest one</a> I found sets up a tree in the browser's DOM and then changes an attribute so the browser can see or not see the branch as it repaints the page. This is (largely) browser independent. <p> But this turned out to be all irrelevant. I didn't realise at first but have now discovered that the client's app refreshes its data via a web service call every time an icon is clicked. So I might as well process the expand/collapse calls on the webserver. </p><p> This is where the session handling becomes critical and why I'm glad I chose CGI::Application. I created a four-level array of hashes (Days/Branches/Jobs/Items) to hold the state of the display then having use'd CGI::Application::Plugin::Session earlier on I simply added two lines of code:</p><blockquote><div><p> <tt>$self-&gt;session-&gt;param('DAYS', $self-&gt;{Days});</tt></p></div> </blockquote><p>to save the array between calls and</p><blockquote><div><p> <tt>$self-&gt;{Days} = $self-&gt;session-&gt;param('DAYS');</tt></p></div> </blockquote><p> to retrieve it next time the page is called. Cookies and data serialisation and storage are all handled by the module. Ah laziness! Let me count thy ways!</p> rats 2005-02-21T23:22:05+00:00 journal Faking a web service for testing http://use.perl.org/~rats/journal/23297?from=rss So how to write a dummy xmlrpc server? Obviously look in CPAN. Two modules grabbed my attention: <a href="http://search.cpan.org/~byrne/SOAP-Lite-0.60a/">SOAP::Lite </a> and <a href="http://search.cpan.org/~rjray/RPC-XML-0.57/">RPC::XML</a>. <p> The old webapp uses XMLRPC::Lite (a subclass of SOAP::Lite) to access the web service so why not use XMLRPC::Lite in server mode? But RPC::XML::Server has a nice feature: add_method. Allows me to package my canned server responses as methods in external files and restart the server and it will read the external file and add the method to its available services. Very nice for automated testing. Test merely(!) has to start the xmlrpc server with a known response file, run the test on the webapp, print pass or fail then restart server with a different response file and so on. </p><p> Time passes... </p><p> RPC::XML::Server is definitely the way to go! Works well and I now have a controllable/configurable web service server for my tests.</p> rats 2005-02-21T22:34:55+00:00 journal Testing a webapp http://use.perl.org/~rats/journal/23296?from=rss Another complication for me is that there are no automated tests for the existing app. Seems they tested by sitting one guy at one terminal to edit some test data and another guy at another terminal clicked the various buttons on the webpage to see that it responded appropriately. Not very efficient in my view (but I wasn't there when the deadline was a couple of hours away and reputations were on the line...) <p> So I need some repeatable automated tests to verify the rewrite at least does everything the old app did as well as being able to check the new stuff. I decided to write a controllable xmlrpc server to fake the real server. Two reasons for this: one is that I need to be able to repeat the information that the server hands my webapp (the real server data changes constantly (is that an oxymoron?)); and secondly the team handling the rewrite for the new processes are behind schedule and they can't give me the new data yet (at least the call names have been finalised) so I need a way to fake it. </p><p> I also want to have tests which automatically configure the server to supply known data then make a call on the webapp and scrape the results out of the HTML and verify the webapp has responded correctly. This is a job for Test::More, <a href="http://search.cpan.org/~petdance/WWW-Mechanize-1.10/">WWW::Mechanize</a>. and <a href="http://search.cpan.org/~petdance/Test-WWW-Mechanize-1.02/">Test::WWW::Mechanize</a></p> rats 2005-02-21T22:16:34+00:00 journal A generic process control webapp? http://use.perl.org/~rats/journal/23295?from=rss The story continues... <p> I won't say who my client is in case I say something libellous (they're actually very nice people and I want to keep it that way) and the app could be used in a number of different situations. The app simply gives the current status of customers' jobs in a manufacturing process and allows an operator to redirect jobs from one machine to another or to another branch if total disaster has hit. Off the top of my head I can think of half a dozen uses: a quick copy or business card printer; various custom building products like garage doors or window frames; auditting a major organisation; an online grocery store; a fast food or table service restaurant chain. The list goes on and on. </p><p> The complication here is the client is moving from making/selling a single product to making/selling multiple products and they're also franchising the business and each franchisee will be able to customise the appearance of their web screens. </p><p> I've got the simplest part of the deal: modifying the webapp. The app uses xmlrpc calls to the manufacturing system and designing and modifying the interface to the machines is the hard part in my view. </p><p> Other developers here have raised the issue of making the system open source but the senior management hasn't made a decision yet.</p> rats 2005-02-21T21:51:44+00:00 journal Rewriting an old webapp http://use.perl.org/~rats/journal/23281?from=rss Client wants me to rewrite/cleanup/extend a webapp written in Perl a couple of years ago by another developer. Almost no use of CPAN modules and HTML is embedded in code i.e. almost impossible to change. And it appears original developer didn't know or want to use OO-Perl which makes extensibility very difficult (for me anyway). Client also wants to use app for another website so stylesheets would be useful. <p> Obviously I'm replacing everything I can with CPAN modules. Spent some time trying to decide which framework to use. Looked at Maypole and Catalyst but didn't feel confident enough to use in production. Looked at Mason which I've used before for production app but found embedded HTML too messy. Even looked at AxKit which I've used for my own apps before. But client doesn't run mod_perl and website is very low traffic so unlikely in the future. So this pretty well made CGI::Application the way to go. Plus I like the clean separation of code and HTML templates. I looked at Template::Toolkit for a very long time but eventually decided TT is overkill for simple web pages client wants. </p><p> I discovered large chunks of obscure (to me) code which I eventually realised was all about session handling. Replaced it all with the single line 'CGI::Application::Plugin::Session'. </p><p> Interesting how much my Perl style has changed in the last few years. Found myself replacing C-style for-loops with more compact Perl list operators (e.g. map, grep) just so I could understand what the original code was trying to do. This allowed me to reduce a couple of 100-line subroutines to 10-line ones and then to realise two of them were duplicates so threw one away. </p><p> At this stage I have a working version of original app. Now to add the new stuff...</p> rats 2005-02-21T06:40:44+00:00 journal