tsee's Journal http://use.perl.org/~tsee/journal/ tsee's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:16:02+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 tsee's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~tsee/journal/ Long time no hacking! http://use.perl.org/~tsee/journal/39893?from=rss <p>It's been a while since I could do some proper hacking on fun code. Whenever I had<br>spare time recently, I had to spend it on maintenance work. It's been even longer<br>since I could hack on new features for <a href="http://padre.perlide.org/">Padre</a>. Over five months, to be exact. In<br>such a long time span, it's easy to forget lots of details about the code and its<br>design. That is the case particularly for such a quickly evolving project.</p><p>After my first glance at the code today, I felt like I was starting from zero.<br>But in the end, I managed to implement a couple of new features that I'm quite satisfied<br>with. This gentle slope of a learning curve demonstrates one of the enormous<br>strengths of the project: Most of the code is quite straightforward and easy to understand[1].<br>Newbies get up to speed quickly and adding features is a breeze. It's simply fun.</p><p>Within a few hours, I improved on <a href="http://szabgab.com/blog/">Gabor</a>'s recent "Find method declaration" feature<br>in that it can now use the "perltags"[2] file that is already used for class name<br>and method auto-completion. This makes the determination of the file in which<br>the selected method was declared a lot more robust. Too bad we can't look into<br>Perl scalars with static analysis. This currently works only for class method syntax<br>a la <i>class-&gt;method</i>. It's still an order of magnitude faster to use than<br>searching files manually.</p><p>Next on my list was adding an XS document type with "calltips" support for the<br><a href="http://perldoc.perl.org/perlapi.html">perl interpreter API</a>. Now, when you open an <a href="http://perldoc.perl.org/perlxs.html">XS</a> file in Padre, you will, among other<br>horrors, get shitty syntax highlighting because it's only recognized as a C source<br>file. But at least Padre can now show you the documentation of the perlapi macro<br>that you're hovering your cursor over. Trust me, that can be a salvation for<br>misremembering, constantly perlapi-rereading and misreading XS neophytes like<br>myself. We have a long way to go to make Padre a useful editor for XS and C,<br>but I'm optimistic. It just so easy to get started that I can relearn the code<br>base once every few months [3].</p><p>--Steffen</p><p>[1] "Easy to understand" doesn't necessarily mean that it's all great or elegant<br>code. The code quality in the project varies greatly. Usually, the initial, hackish<br>implementations are redesigned and refactored into good APIs as the need arises.</p><p>[2] Think "exuberant ctags", but for Perl. Generated with the <a href="http://search.cpan.org/dist/Perl-Tags">Perl::Tags</a> module.<br>Thanks to osfameron for promptly uploading a new version.</p><p>[3] Next up, I'd like to add multiple copies of the perlapi docs. One for each<br>release of perl. The user would then select the version of perl he's developing<br>for and get the correct documentation automatically. Eventually including all the<br>backported constructs from <a href="http://search.cpan.org/dist/Devel-PPPort">ppport.h</a>. Help welcome.</p> tsee 2009-11-13T20:00:27+00:00 journal Users, speak up! http://use.perl.org/~tsee/journal/39619?from=rss <p>A few days ago, <a href="http://szabgab.com/">Gabor Szabo</a> has put out a <a href="http://szabgab.com/blog/2009/09/1252571011.html">request for help packaging Padre</a>. Since then, a few people stopped by the #padre channel on irc.perl.org willing to help with packaging <a href="http://padre.perlide.org/">Padre</a> using <a href="http://par.perl.org/">PAR</a>. After a short introduction, some stated something like "I can help with packaging Padre because we packaged a very large and complex $work application using PAR::Packer. It required a few hacks for the worst external dependencies, but apart from that it was as easy as pie." </p><p> This is great for several reasons. It means people are considering Padre important enough that they are willing to help out and share their expertise in packaging applications with PAR. But it is also one of the few occasions that I have seen folks publicly acknowledge the usefulness of the PAR toolkit. I regularly get private mail from people asking for help with packaging their big-ass $work applications because it's grown over their head. Unfortunately, those people generally also ask for a bit of confidentiality as well. So I know PAR is being used <i>all over the place</i>, in large and small applications alike, scaling up to tens of thousands of computers. But I mustn't point at a list of companies who use it for deployment and do the "this is enterprisey software, worship it" dance. </p><p> So dear PAR users. Please share with the world that you use it and maybe how you do. You're enjoying a good, free tool. The least you can do is give (public) feedback and share your experience with others. The same applies if you're not enjoying it at all. Let us know so we can improve it! </p><p>Cheers,<br>Steffen</p><p>PS: Most of this post would probably equally apply to many other successful modules. If you have stories about those, please share them as well!</p> tsee 2009-09-13T08:09:21+00:00 journal Invaluable advice http://use.perl.org/~tsee/journal/39569?from=rss <p>Here's some invaluable advice:</p><p><b>If you have been hired to fill a non-technical position, do not let anyone know that you possess any kind of technical skills.</b></p><p>Do not let your laziness lure you into giving up clues. For example, do not offer to your admin that you can just fix something yourself given the appropriate permissions. Just bite the bullet and wait like the next guy.</p><p>Seems selfish? Say that when they start to queue for help. Even if you like to help and manage to serve all your friendly colleagues well within a reasonable amount of time and effort, the hidden cost shows up only later. It's when people higher up in the food chain realize that it is most efficient for the division to have the most technically skilled person carry most of the technical maintenance (and/or development) burden. Prepare for a drag if that happens.</p><p>Of course, I don't expect anyone to take this at face value, but it's sad that there's some sort of twisted truth to it.</p> tsee 2009-09-02T11:09:26+00:00 journal Reusable packaged applications with PAR::Packer http://use.perl.org/~tsee/journal/39313?from=rss <p> <a href="http://search.cpan.org/dist/PAR-Packer">PAR::Packer</a> is a tool that can (among other things) create stand-alone binary executables from Perl scripts. It scans the code for dependencies and includes them in the executable. </p><p> Until now, it wasn't possible to reuse these perl-installation-in-one-file packages to run Perl code that wasn't part of the package. This proved to be a bit of a problem in some cases because many Perl applications expect to be able to simply use $^X or a verbatim "perl" executable to execute another instance of the same perl. For this reason, I just implemented the <i>--reusable</i> option to <a href="http://search.cpan.org/perldoc?pp">pp</a>, the PAR Packager command line tool. Since this feature may have dubious security implications, it is disabled by default. To use it, you do this:</p><blockquote><div><p> <tt>&nbsp; pp --reusable -o myapp myapp.pl<br>&nbsp; # works normally:<br>&nbsp;<nobr> <wbr></nobr>./myapp<br> <br>&nbsp; # runs otherapp.pl providing the module library that's part of the executable:<br>&nbsp;<nobr> <wbr></nobr>./myapp --par-options --reuse someOtherApp.pl</tt></p></div> </blockquote><p> If you try to use the <tt>--par-options --reuse</tt> option with an application that wasn't packaged to be <tt>--reusable</tt>, it will refuse to run. </p><p> The new feature requires a new PAR and PAR::Packer release. <a href="http://search.cpan.org/dist/PAR-0.993/">PAR 0.993</a> has been uploaded to CPAN. For PAR::Packer, you need the development version <a href="http://search.cpan.org/dist/PAR-Packer-0.992_01/">0.992_01</a>. If either one of those isn't available from your favourite mirror yet, you can find them <a href="http://steffen-mueller.net/tmp">here</a> temporarily. </p><p> Cheers,<br> Steffen </p> tsee 2009-07-19T17:02:10+00:00 journal In defense of Perl ithreads http://use.perl.org/~tsee/journal/39225?from=rss <p> People like to point out the problems with Perl's threading, say they're <a href="http://search.cpan.org/dist/Coro/Coro.pm#DESCRIPTION">simply the Windows fork-emulation ported to other operating systems</a> and conclude that they're of no use otherwise. They generally omit mentioning the cases in which Perl ithreads are the only viable solution for concurrency in Perl. </p><p> First, you have to understand the <i>i</i> in ithreads. Read: <i>interpreter threads</i>. Each ithread in your Perl program has its own copy of the whole interpreter. Nothing is shared between the interpreters by default*. Most other threading implementations work the other way around: By default they share everything and the user has to deal with locking of any shared resources. This has many advantages over ithreads. Most obviously, an ithread takes a lot more memory. Furthermore, passing data between ithreads is rather painful and very, very slow. But there is, unfortunately, a big downside to shared-by-default: </p><p> Real concurrency (i.e. multiple threads executing concurrently on multiple CPUs) doesn't seem to be feasible with the shared-by-default approach <i>in a language such as Perl</i>. This is because almost all operations -- including those who seem to be entirely read-only -- can potentially modify the data structures. Use a scalar that contains an integer in a string-context and the scalar will be modified to contain a char*. Marc Lehmann explained this in more detail in his talk "Why so-called Perl threads should die" at the German Perl Workshop 2009. (Couldn't find his slides online, sorry.) As far as I know, the typcial dynamic programming languages <i>other than Perl</i> only have (non-concurrent) cooperative multi-threading to start with. </p><p> Now, some will be quick to point out that ithreads are a mere fork() reimplementation with quite a few disadvantages. For a real fork, the kernel can do COW and non-thread-safe XS modules aren't a problem. But if your software has to run on Windows, the fork's a non-starter. As mentioned earlier, threads are used for the emulation of fork() on Windows. That means if you use fork(), you'll get multiple processes on systems which support it natively and multiple (i)threads on Windows with all the associated problems regarding memory use and thread-safety. If you're writing software predominantly on Linux, would you rather debug problems in your development environment or on your customer's (or generally user's) machine? I thought so. There is a case to be made for consistency. </p><p> The other big contender is the excellent <a href="http://search.cpan.org/dist/Coro">Coro</a> module (or an event loop). I suggest you have a look at its documentation to understand what it does exactly**. The downside? It's <i>cooperative</i> multi-threading. It doesn't <i>really</i> run concurrently. The code in each Coro has to cede control to the other coroutines regularly. If there is some code that's not directly under your control and takes a long time, your GUI or what not will be blocked. If you think about it a bit, you'll realize that heavy re-use of code from CPAN and cooperative multi-threading is a non-starter. In my case, I needed to parse Perl code using PPI. That can take seconds... </p><p> I'm all ears for suggestions on how to do concurrency right in a dynamic language. (here: Perl, but if another language does it better, that'd be interesting, too.) </p><p> The requirements are: </p><ul> <li>Real concurrency, i.e. using multiple cores.</li><li>Non-cooperative multi-threading due to code that is not under my control</li><li>Portability</li><li>A point I haven't touched in my rant: Ability to recover from crashes. You can recover from crashing ithreads.</li></ul><p> * This statement may become painfully untrue if you use a non-thread-safe XS module, unfortunately. </p><p> ** I'm not restating what it does in my own words because I'd expect them to be slightly inaccurate thus provoking eternal damnation. </p> tsee 2009-07-04T11:29:06+00:00 journal More refactoring goodness http://use.perl.org/~tsee/journal/39121?from=rss <p>Writing code that modifies code is a difficult task. Writing code that modifies Perl code is a horrible task. Thankfully, writing <i>Perl</i> code that modifies Perl code is not quite as horrible as it could be, thanks to <a href="http://use.perl.org/~Alias/journal">Adam Kennedy</a>'s <a href="http://search.cpan.org/dist/PPI">PPI</a>.</p><p>One stated goal of the <a href="http://padre.perlide.org/">Padre</a> project is to provide refactoring tools for Perl code as well as reasonably possible. So far, there's shortcuts for replacing a variable in its lexical scope, finding a variable's declaration (be it a lexical, file-scoped (our), or package variable declared with 'use vars'), finding unmatched braces, and aligning code blocks on operators. These features are all useful, but they're only a subset of what more mature projects like Eclipse provide. A recent <a href="http://perlmonks.org/?node_id=769387">post on perlmonks</a> discusses some examples of refactoring tools (or strategies) and their applicability to different languages. One of these is the <i>Introduce Explaining Variable</i> pattern. It's now implemented in <a href="http://svn.perlide.org/padre">Padre trunk</a>. It's really quite simple, let me explain with an example:</p><p>The following code implements the derivative of the atan2 function. The code is from the <a href="http://search.cpan.org/~smueller/Math-Symbolic-0.603/lib/Math/Symbolic/Derivative.pm">Math::Symbolic::Derivative</a> module. (I wrote it, so I'm complaining about my own cruft.) This basically implements the equation that is shown in the highlighted comment.</p><blockquote><div><p> <code>sub _derive_atan2 {<br> &nbsp; &nbsp; &nbsp; &nbsp; my ( $tree, $var, $cloned, $d_sub ) = @_;<br> &nbsp; &nbsp; &nbsp; &nbsp; <b># d/df atan(y/x) = x^2/(x^2+y^2) * (d/df y/x)</b><br> &nbsp; &nbsp; &nbsp; &nbsp; my ($op1, $op2) = @{$tree-&gt;{operands}};</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; my $inner = $d_sub-&gt;( $op1-&gt;new()/$op2-&gt;new(), $var, 0 );<br> &nbsp; &nbsp; &nbsp; &nbsp; # templates<br> &nbsp; &nbsp; &nbsp; &nbsp; my $two = Math::Symbolic::Constant-&gt;new(2);<br> &nbsp; &nbsp; &nbsp; &nbsp; my $op = Math::Symbolic::Operator-&gt;new(&#39;+&#39;, $two, $two);</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; my $result = $op-&gt;new(&#39;*&#39;,<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(&#39;/&#39;,<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(&#39;^&#39;, $op2-&gt;new(), $two-&gt;new()),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;+&#39;, $op-&gt;new(&#39;^&#39;, $op2-&gt;new(), $two-&gt;new()),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(&#39;^&#39;, $op1-&gt;new(), $two-&gt;new())<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $inner<br> &nbsp; &nbsp; &nbsp; &nbsp; );<br> &nbsp; &nbsp; &nbsp; &nbsp; return $result;<br>}<br></code></p></div> </blockquote><p>Now, this is pretty hard to read. The <tt>$op1</tt> and <tt>$op2</tt> variables correspond to the function operands <i>y</i> and <i>x</i> respectively. <tt>$d_sub</tt> is a closure that can derive recursively. The two <i>templates</i> are simply a shorthand so I didn't have to write <tt> <i>someclass</i>-&gt;new(...)</tt> repeatedly. To make <i>x</i> and <i>y</i> more apparent and to name <tt>$d_sub</tt> more fitting to its purpose, I open up the file in Padre, right-click each of those variables, select <i>Lexically Replace Variable</i> from the context menu, and provide the new names. Similarly, I replace <tt>$inner</tt>. This yields:</p><blockquote><div><p> <code>sub _derive_atan2 {<br> &nbsp; &nbsp; &nbsp; &nbsp; my ( $tree, $var, $cloned, <b>$derive</b> ) = @_;<br> &nbsp; &nbsp; &nbsp; &nbsp; # d/df atan(y/x) = x^2/(x^2+y^2) * (d/df y/x)<br> &nbsp; &nbsp; &nbsp; &nbsp; my (<b>$y</b>, <b>$x</b>) = @{$tree-&gt;{operands}};</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; my <b>$inner_derivative</b> = <b>$derive</b>-&gt;( <b>$y</b>-&gt;new()/<b>$x</b>-&gt;new(), $var, 0 );<br> &nbsp; &nbsp; &nbsp; &nbsp; # templates<br> &nbsp; &nbsp; &nbsp; &nbsp; my $two = Math::Symbolic::Constant-&gt;new(2);<br> &nbsp; &nbsp; &nbsp; &nbsp; my $op = Math::Symbolic::Operator-&gt;new(&#39;+&#39;, $two, $two);</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; my $result = $op-&gt;new(&#39;*&#39;,<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(&#39;/&#39;,<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(&#39;^&#39;, <b>$x</b>-&gt;new(), $two-&gt;new()),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;+&#39;, $op-&gt;new(&#39;^&#39;, <b>$x</b>-&gt;new(), $two-&gt;new()),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(&#39;^&#39;, <b>$y</b>-&gt;new(), $two-&gt;new())<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; )<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <b>$inner_derivative</b><br> &nbsp; &nbsp; &nbsp; &nbsp; );<br> &nbsp; &nbsp; &nbsp; &nbsp; return $result;<br>}<br></code></p></div> </blockquote><p>Of course, that leaves the giant expression intact which actually calculates the result. It makes sense to add a few more temporary variables with descriptive names. I select <tt>$op-&gt;new(&#39;^&#39;, $x-&gt;new(), $two-&gt;new())</tt> in the above version of the code, right-click, and select <i>Insert Temporary Variable</i>. Then I type the name of the new variable <tt>$x_square</tt>. Padre finds the beginning of the current statement for me and inserts a temporary variable declaration for <tt>$x_square</tt> at that point. It also replaces the selected text with <tt>$x_square</tt>. I manually replace another occurrance of the new temporary and then select <tt>$op-&gt;new(&#39;^&#39;, $y-&gt;new(), $two-&gt;new())</tt> and have it replaced with <tt>$y_square</tt> accordingly. There's more that can be cleaned up, but this handful of clicks and practically no typing has improved the code's readability considerably:</p><blockquote><div><p> <code>sub _derive_atan2 {<br> &nbsp; &nbsp; &nbsp; &nbsp; my ( $tree, $var, $cloned, $derive ) = @_;<br> &nbsp; &nbsp; &nbsp; &nbsp; # d/df atan(y/x) = x^2/(x^2+y^2) * (d/df y/x)<br> &nbsp; &nbsp; &nbsp; &nbsp; my ($y, $x) = @{$tree-&gt;{operands}};</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; my $inner_derivative = $derive-&gt;( $y-&gt;new()/$x-&gt;new(), $var, 0 );<br> &nbsp; &nbsp; &nbsp; &nbsp; # templates<br> &nbsp; &nbsp; &nbsp; &nbsp; my $two = Math::Symbolic::Constant-&gt;new(2);<br> &nbsp; &nbsp; &nbsp; &nbsp; my $op = Math::Symbolic::Operator-&gt;new(&#39;+&#39;, $two, $two);</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; my <b>$x_square</b> = $op-&gt;new(&#39;^&#39;, $x-&gt;new(), $two-&gt;new());<br> &nbsp; &nbsp; &nbsp; &nbsp; my <b>$y_square</b> = $op-&gt;new(&#39;^&#39;, $y-&gt;new(), $two-&gt;new());</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; my $result = $op-&gt;new(&#39;*&#39;,<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $op-&gt;new(<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &#39;/&#39;, <b>$x_square</b>, $op-&gt;new(&#39;+&#39;, <b>$x_square</b>, <b>$y_square</b>)<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $inner_derivative<br> &nbsp; &nbsp; &nbsp; &nbsp; );<br> &nbsp; &nbsp; &nbsp; &nbsp; return $result;<br>}<br></code></p></div> </blockquote><p>Thus Padre helps me refactor crufty code with ease. Many more of these tiny helpers are planned. Stay tuned!</p><p><i>PS</i>: If this didn't convince you, maybe you should just give it a shot. I had to wrestle use.perl for hours to get it to add the highlighting in the example code. If I could add screenshots of the real thing...</p> tsee 2009-06-14T16:50:40+00:00 journal Expanding products of sums http://use.perl.org/~tsee/journal/39083?from=rss <p> A few years ago, when I started studying physics, I wrote a set of modules for representing and dealing with algebraic expressions in Perl: <a href="http://search.cpan.org/dist/Math-Symbolic">Math::Symbolic</a>. It's not a beauty, but it can be quite useful. </p><p> Occasionally, I'm getting mail from people who want it to perform the tasks of a full computer algebra system such as Mathematica. The short answer is: It's not even close to such a thing, it never will be, and was, in fact, never meant to be. One of the most frequent questions I get is a variation of: </p><blockquote><div><p> <i>"How can I expand this product of sums into a sum of products using Math::Symbolic?"</i></p></div> </blockquote><p> Here again, the answer is it can't do that out of the box. But since I've been asked so many times, I wrote two implementations of that which you'll find below. This is to prevent anyone asking me ever again<nobr> <wbr></nobr>:) </p><p> The first implementation is really simple and I'd almost call it elegant. It is, however, also quite slow. </p><blockquote><div><p> <tt>use strict;<br>use warnings;<br>use Math::Symbolic qw/:all/;<br>use Math::Symbolic::Custom::Transformation qw/:all/;<br> <br>my $function = parse_from_string(&lt;&lt;'HERE');<br>(a + b)*(d + e + f)<br>HERE<br>#(b + c + d + e + f)*(a + b)*(d + e + f)*(a + b + c + d)*(a + b + c + d + e)<br> <br>print "Before: $function\n";<br> <br>my $pattern = Math::Symbolic::Custom::Pattern-&gt;new(<br>&nbsp; parse_from_string('(TREE_x+TREE_y) * TREE_z'),<br>&nbsp; commutation =&gt; 1,<br>);<br> <br>my $expand = new_trafo(<br>&nbsp; &nbsp;$pattern =&gt; 'TREE_x*TREE_z + TREE_y*TREE_z',<br>);<br> <br>while (1) {<br>&nbsp; my $result = $expand-&gt;apply_recursive($function);<br>&nbsp; last if not defined $result;<br>&nbsp; $function = $result;<br>}<br> <br>print "After: $function\n";</tt></p></div> </blockquote><p> It uses the Math::Symbolic syntax itself to define the logic. Most of the work is actually done by the pattern matching and transformation modules <a href="http://search.cpan.org/dist/Math-Symbolic-Custom-Pattern">Math::Symbolic::Custom::Pattern</a> and <a href="http://search.cpan.org/dist/Math-Symbolic-Custom-Transformation">Math::Symbolic::Custom::Transformation</a>. The Pattern class defines search rules that can be matched against the expression's tree. The Transformation specifies rules to replace it with. Kind of like regular expressions. Just not as good (or fast). </p><p> The second implementation is likely much more useful and certainly a lot faster (but not optimized). It implements almost all of the logic manually and is based somewhat on Mark Jason Dominus wonderful iterator from <i>Higher Order Perl</i>. (Go, buy the book if you haven't. It's an utterly enjoyable read.) </p><blockquote><div><p> <tt>use strict;<br>use warnings;<br>use Math::Symbolic qw/:all/;<br> <br>my $function = parse_from_string(&lt;&lt;'HERE');<br>(a + b)*(d + e + f)<br>HERE<br>#(b + c + d + e + f)*(a + b)*(d + e + f)*(a + b + c + d)*(a + b + c + d + e)<br> <br># First, split the product into sums<br>my @sums = split_formula( B_PRODUCT, $function );<br> <br>#print "$_\n" foreach @sums;<br> <br># Split each sum into its sub-terms<br>my @terms = map {<br>&nbsp; [ split_formula( B_SUM, $_ ) ]<br>} @sums;<br> <br>my $n_terms = 1;<br>$n_terms *= @$_ for @terms;<br>print "Calculating all $n_terms terms...\n";<br>print "@$_\n" foreach @terms;<br> <br># This calculates the full formula in memory and stores it in $function<br># $function = multiply(\@terms);<br># print $function, "\n";<br> <br># This calculates each term and then prints it to STDOUT, but doesn't<br># store it because memory is scarce<br>multiply_print(\@terms);<br> <br># We have to keep in mind that the formula is really a tree.<br>sub split_formula {<br>&nbsp; my $optype = shift;<br> <br>&nbsp; my @formulas = @_;<br> <br>&nbsp; my @split;<br>&nbsp; while (@formulas) {<br>&nbsp; &nbsp; my $f = shift @formulas;<br>&nbsp; &nbsp; if ($f-&gt;term_type == T_OPERATOR and $f-&gt;type == $optype) {<br>&nbsp; &nbsp; &nbsp; push @formulas, @{ $f-&gt;{operands} };<br>&nbsp; &nbsp; }<br>&nbsp; &nbsp; else {<br>&nbsp; &nbsp; &nbsp; push @split, $f;<br>&nbsp; &nbsp; }<br>&nbsp; }<br> <br>&nbsp; return @split;<br>}<br> <br># all of the following is based on the iterator<br># pattern of Mark Jason Dominus' "Higher Order Perl", p. 128ff<br>sub multiply {<br>&nbsp; my $terms = shift;<br>&nbsp; my ($max, $count) = make_pattern($terms);<br> <br>&nbsp; my $func = make_product($terms, $count);<br>&nbsp; return $func unless increment($max, $count);<br> <br>&nbsp; while (1) {<br>&nbsp; &nbsp; my $prod = make_product($terms, $count);<br>&nbsp; &nbsp; $func += $prod;<br>&nbsp; &nbsp; last unless increment($max, $count);<br>&nbsp; }<br> <br>&nbsp; return $func;<br>}<br> <br>sub multiply_print {<br>&nbsp; my $terms = shift;<br> <br>&nbsp; my $iter = make_term_iterator($terms);<br> <br>&nbsp; my $first = 1;<br>&nbsp; while (1) {<br>&nbsp; &nbsp; my $prod = $iter-&gt;();<br>&nbsp; &nbsp; last if not defined $prod;<br>&nbsp; &nbsp; if ($first) {<br>&nbsp; &nbsp; &nbsp; $first = 0;<br>&nbsp; &nbsp; &nbsp; print $prod;<br>&nbsp; &nbsp; } else {<br>&nbsp; &nbsp; &nbsp; print " + " . $prod;<br>&nbsp; &nbsp; }<br>&nbsp; }<br> <br>&nbsp; print "\n";<br>&nbsp; return;<br>}<br> <br>sub make_term_iterator {<br>&nbsp; my $terms = shift;<br>&nbsp; my ($max, $count) = make_pattern($terms);<br> <br>&nbsp; my $empty = 0;<br>&nbsp; return sub {<br>&nbsp; &nbsp; return() if $empty;<br>&nbsp; &nbsp; my $func = make_product($terms, $count);<br>&nbsp; &nbsp; $empty = !increment($max, $count);<br>&nbsp; &nbsp; return $func;<br>&nbsp; };<br>}<br> <br>sub make_product {<br>&nbsp; my $terms = shift;<br>&nbsp; my $count = shift;<br> <br>&nbsp; # Note: One *could* save some CPU cycles by not cloning here (new).<br>&nbsp; # BUT that may lead to fun debugging and interesting memory cycles<br>&nbsp; # if you intend to actually use the tree.<br>&nbsp; my $prod = $terms-&gt;[0][ $count-&gt;[0] ]-&gt;new;<br>&nbsp; foreach my $i (1..$#$terms) {<br>&nbsp; &nbsp; $prod *= $terms-&gt;[$i][ $count-&gt;[$i] ]-&gt;new;<br>&nbsp; }<br>&nbsp; return $prod;<br>}<br> <br>sub increment {<br>&nbsp; my $max = shift;<br>&nbsp; my $count = shift;<br> <br>&nbsp; my $i = $#$count;<br>&nbsp; while (1) {<br>&nbsp; &nbsp; if ($count-&gt;[$i] &lt; $max-&gt;[$i]) {<br>&nbsp; &nbsp; &nbsp; $count-&gt;[$i]++;<br>&nbsp; &nbsp; &nbsp; return 1;<br>&nbsp; &nbsp; }<br>&nbsp; &nbsp; else {<br>&nbsp; &nbsp; &nbsp; $count-&gt;[$i] = 0;<br>&nbsp; &nbsp; &nbsp; $i--;<br>&nbsp; &nbsp; }<br>&nbsp; &nbsp; if ($i &lt; 0) {<br>&nbsp; &nbsp; &nbsp; return();<br>&nbsp; &nbsp; }<br>&nbsp; }<br>}<br> <br>sub make_pattern {<br>&nbsp; my $terms = shift;<br>&nbsp; my @max;<br>&nbsp; my @pattern;<br>&nbsp; foreach my $set (@$terms) {<br>&nbsp; &nbsp; push @max, $#$set;<br>&nbsp; &nbsp; push @pattern, 0;<br>&nbsp; }<br>&nbsp; return \@max, \@pattern;<br>}</tt></p></div> </blockquote><p> I bet you can see why that second implementation doesn't give me as much of a warm, fuzzy feeling. </p><p> Cheers,<br> Steffen </p> tsee 2009-06-05T17:18:51+00:00 journal Padre usability improvements http://use.perl.org/~tsee/journal/38955?from=rss <p>On the week-end, I finally implemented a few simple usability improvements for <a href="http://padre.perlide.org/">Padre, the Perl IDE</a>.</p><p>With the recently released version 0.35, Padre supports a context (right click) menu that's actually context sensitive! (<i>No shit!</i>) If you right-click on a variable, Padre will offer additional options in the context menu that let you jump to the declaration of the variable or replace all occurrences of a lexical(!) variable.</p><p>Additionally, we stole a nice feature from Eclipse/Perl: If you hold Ctrl while left-clicking on a variable or subroutine name, the focus will jump to the respective definition. All this only works in the current file, unfortunately, but eventually, this will be a project-wide feature.</p><p>Today, a <a href="http://padre.perlide.org/ticket/321">ticket in the bug tracker</a> from <a href="http://peter.makholm.net/">Peter Makholm</a> prompted me to implement an API for plugin authors. Your plugin can now provide different context menus depending on the document type, code at the cursor, additional modifier keys, or moon phase.</p><p><i>Update:</i> Here's a simple example that you can copy&amp;paste to your plugin to extend the context menu with a simple item</p><p><code><br>sub event_on_context_menu {<br> &nbsp; &nbsp; &nbsp; &nbsp; my ($self, $doc, $editor, $menu, $event) = @_;</code></p><p><code> &nbsp; &nbsp; &nbsp; &nbsp; $menu-&gt;AppendSeparator();<br> &nbsp; &nbsp; &nbsp; &nbsp; Wx::Event::EVT_MENU(<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $editor,<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $menu-&gt;Append( -1, Wx::gettext("Fun") ),<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; sub { warn "FUN!" },<br> &nbsp; &nbsp; &nbsp; &nbsp; );<br> &nbsp; &nbsp; &nbsp; &nbsp; return();<br>}<br></code></p><p>Cheers,</p><p>Steffen</p> tsee 2009-05-11T18:35:13+00:00 journal How Padre saved my day http://use.perl.org/~tsee/journal/38714?from=rss <p> Every moderately proficient Perl programmer will eventually be faced with the horror that is old code written by people who still thought golf was good programming style. Very recently, my worst experience until now was with code that had the friendly warning "use 5.002;" in the first line. As if that wasn't enough to scare the hell out of me, I was told the code had been written only in 2006 or so. Not just that: It had been devised by somebody I highly respect and know to be extremely intelligent. But an individual who simply hadn't known Perl when he wrote the program. </p><p> Here's one of the biggest downsides of the language in action. Somebody who isn't proficient but smart and creative will be able to craft complicated programs that (kind of) serve their complicated purpose and won't be readable by anyone but their inventor. Hackers who know the language well could do the same, but they know better ways to solve the problem at hand than resorting to unnecessary cleverness. </p><p> At this point, you already know the program in question doesn't use strictures. Instead, it does interesting stuff like using the file handles (GLOBs!) of literal name "0" to N to process N+1 files synchronously or using $_ implicitly in a scope that spans way over 100 lines. Variables are named appropriately as <code>$a1, $a2, $a3, $a4</code> and <code>$aa1, $aa2, $aa3, $aa4</code>. But I mustn't forget my favorite: <code>$hwww</code>! </p><p> If you've ever had to deal with a complicated program that uses only globals, you will most certainly agree that the first step to understanding it is to declare those variables lexically in the tightest scope possible. That isolation of contexts makes it a damn sight easier to grok what's happening. </p><p> But I digress. This is about how <a href="http://padre.perlide.org/">Padre</a> helped me fix this. I'd love to say that I simply opened the document in the editor, positioned my cursor on one of those pesky pseudo-globals and hit the "convert this global to a lexical in the tightest sensible scope" action in the Perl-specific-features menu. You know, it would walk the scope tree from the occurrence of the variable I highlighted and find the tightest scope that contains all occurrences of the variable and declare it there. Furthermore, if it's used as a loop variable a la <code>for($i=0;$i&lt;...;++$i)</code>, it'd detect that its value likely not depended on outside the loop and declare it there for me, too. But I haven't had the time to actually write that feature yet*. </p><p> I still had to figure out the scope of each variable manually. Instead, once I had declared a variable *somewhere*, I could simply hit "replace this lexical variable" in the aforementioned Perl menu and have all occurrences (including <code>"${aa1}"</code> in strings) replaced with a less meaningless name. This was particularly useful for loop variables which tend to be reused in different scopes and thus meanings. A normal search/replace would require user interaction to stop it after the current section of the code. One distraction less while trying to understand some complicated piece of code. </p><p> But this isn't really how Padre saved my day. It's that when this heavy use of the lexical replacement feature triggered a couple of bugs in it, I was able to dive into the implementation head-first and simply fix it. It's just Perl and most of it is actually quite accessible! That's how Padre made my day less miserable. it helped my fix that ugly code and gave me the warm, fuzzy feeling of being in full control of my tools and particularly being able to improve them when I need to. </p><p> * The key here is: I could! So could you or any other Perl programmer. </p> tsee 2009-03-28T10:32:35+00:00 journal PAR::Repository with static dependency resolution http://use.perl.org/~tsee/journal/38409?from=rss <p>My <a href="http://use.perl.org/~tsee/journal/38343">previous journal entry</a> was about the <a href="http://search.cpan.org/dist/PAR-Repository">PAR::Repository</a> auto-upgrading feature. That, however, was just a precursor to the big news. Here's (approximately) what I posted to the <a href="http://nntp.perl.org/group/perl.par">PAR mailing list</a> a few days ago: </p><p> Let me provide some context. Ever since I wrote PAR::Repository::*, people mistook it for a PAR-based <a href="http://ppm4.activestate.com/">PPM</a> replacement. It was never intended to be a package manager/installer like PPM but instead as a sort of application server that could be comfortably and centrally managed, maintained, and upgraded. Even having separate staging and production repositories is quite simple as a PAR repository is just a directory on an ordinary web server or file system. Heck, you can even import one into git and switch branches as your heart desires. Since the clients simply fetch the most current packages for their specific needs, they are always be up to date when launching a new application from the repository. </p><p> After I gave a <a href="http://www.yapceurope2008.org/ye2008/talk/1389">talk about PAR and the repository concept</a> at <a href="http://www.yapceurope2008.org/">YAPC::EU 2008 in Copenhagen</a>, people again asked whether they could use a PAR repository in place of PPM. I said they couldn't and that the fundamental difference is that PAR::Repository finds dependencies dynamically, recursively, at run-time, whenever a module is required as opposed to PPM's static dependency information. But at the time, I already had a secret scheme for adding static dependency information to PAR repositories. Since the work on PAR is done purely in my not so copious spare time, I didn't spill the beans just yet in case I'd never get around to finish it. Seems I was lucky. </p><p> Since a couple of days ago, there are development releases of <a href="http://search.cpan.org/~smueller/PAR-0.985_01/">PAR</a>, <a href="http://search.cpan.org/~smueller/PAR-Dist-0.44/">PAR::Dist</a>, <a href="http://search.cpan.org/~smueller/PAR-Repository-0.18_01/">PAR::Repository</a>, <a href="http://search.cpan.org/~smueller/PAR-Indexer-0.91/">PAR::Indexer</a> and <a href="http://search.cpan.org/~smueller/PAR-Repository-Client-0.23_02/">PAR::Repository::Client</a>[1] that sport <b>support for static dependency extraction from<nobr> <wbr></nobr>.par files, storage thereof in the repository, and resolution and application of it in the client!</b> </p><p> Getting to this point required a bit of Yak Shaving.</p><ul> <li>PAR::Dist's merge_par routine formerly simply copied the first package's META.yml. Now, it also merges the "provides" as well as the various requires-like sections. </li><li>The tests of both PAR::Repository and<nobr> <wbr></nobr>::Client were in dire need of improvement because... </li><li>... both modules needed some refactoring to make way for the rest of the changes. </li><li>The PAR::Repository aquired a new index file for dependencies. </li><li>The PAR file injection routines use the information from META.yml to fill it. The removal routines correctly remove the information again. </li><li>To make up for the extra bandwidth required for the dependency information, a checksum-scheme has been implemented to check for updates. </li><li>The client has a new option "static_dependencies" to enable recursive resolution of dependencies as found in the new dependencies index. </li><li>The "use PAR {...}" interface now has a "dependencies" option that enables the client's static dependency processing. </li></ul><p> All involved modules have new releases on CPAN. They are mostly developer releases, since there <i>must</i> be serious bugs. </p><p> Thanks for reading! </p><p> Best regards,<br> Steffen</p><p> [1] To give the new releases a whirl, you can simply install PAR::Repository (for the server side) or PAR::Repository::Client (for the client, doh). No need to manually install all the distributions, they'll be picked up as dependencies. </p> tsee 2009-02-04T18:50:14+00:00 journal PAR evilness to make remote auto-upgrading work http://use.perl.org/~tsee/journal/38343?from=rss <p>When I think about telling people about PAR internals, a reply from a colleague readily comes to mind, when he was asked about an icky detail of his analysis:</p><blockquote><div><p> <i>You don't want to know how sausages are made!</i></p></div> </blockquote><p> But then I can't resist grossing out people with some details anyway... </p><p> Two years ago, I wrote <a href="http://search.cpan.org/dist/PAR-Repository-Client">PAR::Repository::Client</a> as an interface for loading <a href="http://search.cpan.org/dist/PAR">PAR</a>s and thus arbitrary modules from a remote server. If the client is installed, all you need to do to auto-load missing modules from the server is: </p><blockquote><div><p> <tt>&nbsp; use PAR { repository =&gt; 'https://foo.com/myapp' };<br>&nbsp; use Foo; # will be loaded from remote if necessary</tt></p></div> </blockquote><p> But since this may become expensive, and caching the binaries only removes part of that, the "install" option was part of the interface almost from the start: </p><blockquote><div><p> <tt>&nbsp; use PAR { repository =&gt; 'https://foo.com/myapp', install =&gt; 1 };<br>&nbsp; use Foo; # will be loaded AND INSTALLED if necessary</tt></p></div> </blockquote><p> Back then, I also added most of the code necessary for an "upgrade" option. </p><blockquote><div><p> <tt>&nbsp; use PAR { repository =&gt; 'https://foo.com/myapp', upgrade =&gt; 1 };<br>&nbsp; use Foo; # will be loaded AND INSTALLED OR UPGRADED if necessary</tt></p></div> </blockquote><p> Unfortunately, it was missing a few critical details until today. The repository client is normally only invoked when all other sources fail. But that's a problem if you're trying to check for upgrades. Thus, repositories in upgrade-mode are now checked early in the module-loading process. </p><p> The real bummer was that in order to check for upgrades, the locally installed version has to be determined. Since this is hard to do reliably without loading the module, that's what PAR has to do. But that means require()ing module X from within an early @INC hook that ran due to a "require X;". There's so many things wrong with that idea, it's not even funny. It seems that creating an infinite recursion in an @INC hook segfaults perl 5.8.9. Regardless, it can be (and was) made to work: </p><ul> <li>Before running the client's upgrade_module method, dynamically override the set of active (via PAR.pm) PAR::Repositories to be empty.</li><li>Run the current repository client's upgrade_module method which will attempt to require the module for checking its version.</li><li>Afterwards, check whether the module was loaded using %INC.</li><li>If not, continue normally, probably ending up failing to load the module from anywhere or loading the freshly installed copy.</li><li>If the module was loaded, prevent an additional loading with an evil trick in the @INC hook:</li></ul><blockquote><div><p> <tt>&nbsp; my $line = 1;<br>&nbsp; return \*I_AM_NOT_HERE, sub { $line ? ($_="1;",$line=0,return(1)) : ($_="",return(0)) };</tt></p></div> </blockquote><p> Even disregarding the slight obfuscation, can you figure out how this works? </p><p> One obscure feature of @INC and the module loading is the return value(s) of a subroutine @INC hook. It normally simply returns a file handle that the module code is then read from. But if it returns a code ref as its second return value, that code ref is called repeatedly until it returns false. After each invocation, $_ is assumed to contain the next line of the module code. If the first argument was a file handle nonetheless, $_ is initialized to a new line from the file handle before calling the subroutine. </p><p> The motivation here is mostly that we want to set the file contents to "1;". Unfortunately, passing undef as the file handle resulted in the subroutine not being called. This smells like a bug in perl to me, but I'll have to check that more closely with blead. Furthermore, it's not wise to load any unnecessary modules in PAR.pm as they would have to be included verbatim in an uncompressed part of PAR::Packer created executables. Therefore, instead of simply passing a IO::Handle-&gt;new(), I'm supplying an arbitrary GLOB ref. </p><p> Finally, the subroutine itself simply sets $_ to "1;" in the first invocation and returns zero on the second to stop the evaluation, thus essentially short-circuiting require()'s loop through @INC. </p><p> After going through this considerable pain, I got the auto-upgrading feature of PAR::Repository::Client to work. There's probably still bugs and testing it as part of the test suite is no fun (but still feasible). </p><p>Stay tuned for a new release of the involved modules.</p><p> Cheers,<br> Steffen </p> tsee 2009-01-25T22:01:42+00:00 journal Padre 0.22 highlights and checks Perl6 code http://use.perl.org/~tsee/journal/38141?from=rss <p>Padre version 0.22 has just been uploaded to the PAUSE. That means it will propagate to the CPAN mirrors without a few hours. Like the previous release, the list of changes is quite long, but one particular achievement is support for highlighting Perl6 code and checking its syntax if <a href="http://search.cpan.org/dist/Padre-Plugin-Perl6">Padre::Plugin::Perl6</a> is installed and enabled. Christmas is close.</p><p>Once the distribution has reached your CPAN mirror, you will be able to access the <a href="http://padre.perlide.org/browser/tags/Padre-0.22/Changes">full Change Log here</a>.</p><p>The next Padre release is very preliminarily scheduled for December 28th or 29th and we're still looking for a new release engineer.</p><p>Best regards,</p><p>Steffen</p> tsee 2008-12-23T19:14:38+00:00 journal On a similarity between C++ and Perl popularity http://use.perl.org/~tsee/journal/38116?from=rss <p>Today, I read an interesting <a href="http://www.devx.com/SpecialReports/Article/38813/0/page/1">interview with Bjarne Stroustrup</a>, the father of C++, on <a href="http://www.devx.com/">DevX</a> from August of this year. It's a good read, so if you're a C++ user, you should have a look. But even if you never touched any C++ code, there's a very interesting bit of information:</p><p>On <a href="http://www.devx.com/SpecialReports/Article/38813/0/page/6">page 6</a>, the interviewer, Danny Kalev, asks Stroustrup:</p><blockquote><div><p> <i>Is C++ usage really declining, as some biased analysts and journalists have been trying to convince us for years (often not without ulterior motives), or is this complete nonsense?</i></p></div> </blockquote><p>Does that ring a bell? The "Perl is dead" crap that's been splashing down the gutters of the interweb waste disposal system, anyone? I urge you to read the full answer. Much of it applies to Perl as well. Please note that Stroustrup doesn't simply dismiss the issue. Here's an excerpt from his reply:</p><blockquote><div><p> <i>[...] C++ use appears to be declining in some areas and appears to be on an upswing in other areas. [...] Most of the popular measures basically measures noise and ought to report their findings in decibel rather than "popularity." Many of the major uses are in infrastructure (telecommunications, banking, embedded systems, etc.) where programmers don't go to conferences or describe their code in public. Many of the most interesting and important C++ applications are not noticed, they are not for sale to the public as programming products, and their implementation language is never mentioned. [...]</i></p><p><i>It's a really big world "out there" and the increase in the number of users of one language does not imply the decrease in the numbers of another. [...]</i></p><p><i>One simple thing that confuses many discussions of language use/popularity is the distinction between relative and absolute measures. For example, I say that C++ use is growing when I see user population grow by 200,000 programmers from 3.1M to 3.3M. However, somebody else may claim that "C++ is dying" because it's "popularity" has dropped from 16 percent to 10 percent of the total number of users. Both claims could be simultaneously true as the number of programmers continues to grow and especially as what is considered to be programming continues to change. [...]</i></p><p><i>Most of the popularity measures seem to measure buzz/noise, which is basically counting mentions on the web. That's potentially very misleading. Ten people learning a scripting language will make much more noise than a thousand full time programmers using C++, especially if the thousand C++ programmers are working on a project crucial to a company&#8212;such programmers typically don't post and are often not allowed to. My worry is that such measures may actually measure the number of novices and thus be an indication of a worsening shortage of C++ programmers. Worse, managers and academics may incautiously take such figures seriously (as a measure of quality) and become part of a vicious circle.</i></p></div> </blockquote><p>I know first hand about large C++ systems that don't produce the slightest bit of publicity for the language they're implemented in. It's what I deal with every day. Dito for large Perl code bases. Stroustrup hits the nail on the head about this issue (and C++). It's exactly what I think about Perl in the same context. There may be an issue with not generating as much noise as others (not enough new blood), but it's by no means an indication of stagnation or decline. People simply use it do what they always did. People also use it to do new stuff. But they don't blather about it all day. They earn their salary and at the end of the day, they go home to their families and spend their spare time on more interesting things than blogging about their favourite new toy language*. You have to realize: This applies to easily more than 95% of all professional programmers.</p><p>* That reminds me of something... got to go.</p> tsee 2008-12-18T17:09:34+00:00 journal Padre 0.21 - More improvements than can fit in this margin! http://use.perl.org/~tsee/journal/38086?from=rss <p>I'm quite proud to announce the release of Padre 0.21.</p><p>It features the biggest list of changes of any Padre release so far. The menu code has received a major overhaul, the editor has become multi-threaded, and we see more "advanced" Perl-specific features like reliably finding the location of a variable declaration or the experimental feature of replacing a lexical variable. The full list of changes can be found in the <a href="http://search.cpan.org/src/SMUELLER/Padre-0.21/Changes">Changes</a> file of the distribution.</p><p>The list of developers has grown to thirteen, but that list is probably not even complete. The program has been translated to nine languages at this point.</p><p>With this release, we are starting to rotate the release duty so the weekly or bi-weekly releases don't block on the availability of Gabor at those rare points in time when no major refactoring or feature implementation is going on.</p><p>You can find more information and the Padre mailing list, irc information, bug and issue tracking, etc. on the <a href="http://padre.perlide.org/">Padre site</a> as usual.</p><p>Cheers,<br>Steffen</p> tsee 2008-12-14T19:54:41+00:00 journal Class::XSAccessor with constructors = Object::Tiny::XS http://use.perl.org/~tsee/journal/37985?from=rss <p>Since <a href="http://search.cpan.org/dist/Class-XSAccessor">Class::XSAccessor</a> now supports constructors (see previous journal entry), there's everything in place to implement <a href="http://search.cpan.org/dist/Object-Tiny">Object::Tiny</a> in XS. Hence you can find <a href="http://search.cpan.org/dist/Object-Tiny-XS">Object::Tiny::XS</a> 1.01 on CPAN very soon. It's been uploaded to PAUSE.</p><p>The following benchmarks can be found in the distribution. They're comparing O::Tiny::XS to O::Tiny and Class::Accessor::Fast.</p><blockquote><div><p> <tt>Benchmarking constructor plus accessors...<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Rate accessor&nbsp; &nbsp; &nbsp;tiny&nbsp; tiny_xs<br>accessor 107325/s&nbsp; &nbsp; &nbsp; &nbsp;--&nbsp; &nbsp; &nbsp;-40%&nbsp; &nbsp; &nbsp;-57%<br>tiny&nbsp; &nbsp; &nbsp;177837/s&nbsp; &nbsp; &nbsp; 66%&nbsp; &nbsp; &nbsp; &nbsp;--&nbsp; &nbsp; &nbsp;-29%<br>tiny_xs&nbsp; 248931/s&nbsp; &nbsp; &nbsp;132%&nbsp; &nbsp; &nbsp; 40%&nbsp; &nbsp; &nbsp; &nbsp;--<br> &nbsp; <br>Benchmarking constructor alone...<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Rate accessor&nbsp; &nbsp; &nbsp;tiny&nbsp; tiny_xs<br>accessor 168659/s&nbsp; &nbsp; &nbsp; &nbsp;--&nbsp; &nbsp; &nbsp;-49%&nbsp; &nbsp; &nbsp;-57%<br>tiny&nbsp; &nbsp; &nbsp;330831/s&nbsp; &nbsp; &nbsp; 96%&nbsp; &nbsp; &nbsp; &nbsp;--&nbsp; &nbsp; &nbsp;-17%<br>tiny_xs&nbsp; 396844/s&nbsp; &nbsp; &nbsp;135%&nbsp; &nbsp; &nbsp; 20%&nbsp; &nbsp; &nbsp; &nbsp;--<br> &nbsp; <br>Benchmarking accessors alone...<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Rate accessor&nbsp; &nbsp; &nbsp;tiny&nbsp; tiny_xs<br>accessor 331/s&nbsp; &nbsp; &nbsp; &nbsp;--&nbsp; &nbsp; &nbsp;-16%&nbsp; &nbsp; &nbsp;-54%<br>tiny&nbsp; &nbsp; &nbsp;395/s&nbsp; &nbsp; &nbsp; 19%&nbsp; &nbsp; &nbsp; &nbsp;--&nbsp; &nbsp; &nbsp;-45%<br>tiny_xs&nbsp; 715/s&nbsp; &nbsp; &nbsp;116%&nbsp; &nbsp; &nbsp; 81%&nbsp; &nbsp; &nbsp; &nbsp;--</tt></p></div> </blockquote> tsee 2008-11-30T12:07:33+00:00 journal Class::XSAccessor - now doing XS constructors http://use.perl.org/~tsee/journal/37974?from=rss <p>So far, when using <a href="http://search.cpan.org/dist/Class-XSAccessor">Class::XSAccessor</a> or <a href="http://search.cpan.org/dist/Class-XSAccessor-Array">Class::XSAccessor::Array</a> to generate fast accessors implemented in XS for you, you'd still have to write a simple <i>new</i>:</p><p><code><br> &nbsp; &nbsp; sub new {<br> &nbsp; &nbsp; &nbsp; &nbsp; my $class = shift;<br> &nbsp; &nbsp; &nbsp; &nbsp; my $self = bless { @_ } =&gt; $class;<br> &nbsp; &nbsp; &nbsp; &nbsp; return $self;<br> &nbsp; &nbsp; }<br></code></p><p>So I just added an XS implementation of the above constructor which you can optionally import into your class. The gains are modest compared to the large speedup of the accessors. You gain only something like 40% (without arguments). However, I haven't compared this with any of the heavier method generators out there.</p><p>Now, if you think that this kind of micro-optimization is pointless: You're right. Besides being an exercise in XS, this module means that <b>you</b> don't have to spend your time micro-optimizing your simple methods. Everybody knows it's bad, but almost all succumb to the inner voice that makes them do it regardless eventually. With this being done, you can't get much faster, so you don't even have to try!</p><p>Cheers,<br>Steffen</p><p> &nbsp; &nbsp; &nbsp; &nbsp;</p> tsee 2008-11-29T20:56:10+00:00 journal Marcel's accessor generator benchmarking http://use.perl.org/~tsee/journal/37791?from=rss <p> So Marcel has been <a href="http://hanekomu.at/blog/dev/20081101-1054-dissecting_the_moose_3-benchmarking_accessor_generators.html">benchmarking accessor generators</a>. He used Class::Accessor::Fast::XS which is a verbatim copy of Class::XSAccessor with the Class::Accessor interfacing code added. But he missed out on the fastest ones bar XS accessor hand-rolling. It's a combination of using Object::Tiny's constructor and Class::XSAccessor's accessor generation. (Let's call it Object::Tiny::XS!) Here's the adapted code and then the results: </p><p> <a href="http://hanekomu.at/blog/dev/20081101-1258-dissecting_the_moose_4-benchmarking_accessor_generators-accessors_only.html"> <b>Benchmarking attribute access only</b> </a> </p><blockquote><div><p> <tt>#!/usr/bin/env perl<br> <br>use warnings;<br>use strict;<br>use Benchmark qw(cmpthese timethese<nobr> <wbr></nobr>:hireswallclock);<br> <br>package WithMoose;<br>use Moose;<br>has myattr =&gt; ( is =&gt; 'rw' );<br> <br>package WithMooseImmutable;<br>use Moose;<br>has myattr =&gt; ( is =&gt; 'rw' );<br>__PACKAGE__-&gt;meta-&gt;make_immutable;<br> <br>package WithMouse;<br>use Mouse;<br>has myattr =&gt; ( is =&gt; 'rw' );<br> <br>package WithClassAccessor;<br>use base qw(Class::Accessor);<br>__PACKAGE__-&gt;mk_accessors(qw/myattr/);<br> <br>package WithClassAccessorFast;<br>use base qw(Class::Accessor::Fast);<br>__PACKAGE__-&gt;mk_accessors(qw/myattr/);<br> <br>package WithClassAccessorFastXS;<br>use base qw(Class::Accessor::Fast::XS);<br>__PACKAGE__-&gt;mk_accessors(qw/myattr/);<br> <br>packag e WithClassAccessorComplex;<br>use base qw(Class::Accessor::Complex);<br>__PACKAGE__-&gt;mk_new-&gt;mk_scalar_accessors(qw/myat<nobr>t<wbr></nobr> r/);<br> <br>package WithClassAccessorConstructor;<br>use base qw(Class::Accessor::Constructor Class::Accessor::Complex);<br>__PACKAGE__-&gt;mk_constructor-&gt;mk_scalar_accessors(qw<nobr>/<wbr></nobr> myattr/);<br> <br>package WithMojo;<br>use base qw(Mojo::Base);<br>__PACKAGE__-&gt;attr('myattr');<br> <br>package WithClassMethodMaker;<br>use Class::MethodMaker<br>&nbsp; &nbsp; [ scalar =&gt; [ qw/myattr/ ],<br>&nbsp; &nbsp; &nbsp; new&nbsp; &nbsp; =&gt; [ qw/-hash new/ ],<br>&nbsp; &nbsp; ];<br> <br>package WithAccessors;<br>use accessors qw(myattr);<br> <br>sub new { bless {}, shift }<br> <br>package WithObjectTiny;<br>use Object::Tiny qw/myattr/;<br>sub set_myattr { $_[0]-&gt;{myattr} = $_[1] }<br> <br>package WithSpiffy;<br>use Spiffy -base;<br>field 'myattr';<br> <br>package WithClassSpiffy;<br>use Class::Spiffy -base;<br>field 'myattr';<br> <br>package WithClassXSAccessor;<br>use Class::XSAccessor accessors =&gt; { myattr =&gt; 'myattr' };<br>sub new {my $class = shift; bless {@_} =&gt; $class}<br> <br>package WithClassXSAccessorArray;<br>use Class::XSAccessor::Array accessors=&gt; { myattr =&gt; 0 };<br>sub new {my $class = shift; my %args = @_; bless [$args{myattr}] =&gt; $class}<br> <br>package WithObjectTinyXS;<br>use Object::Tiny qw/myattr/;<br>use Class::XSAccessor accessors =&gt; { myattr =&gt; 'myattr' }, replace =&gt; 1;<br> <br>package main;<br> <br>my $moose&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; = WithMoose-&gt;new;<br>my $moose_immutable&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; = WithMooseImmutable-&gt;new;<br>my $mouse&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; = WithMouse-&gt;new;<br>my $class_accessor&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= WithClassAccessor-&gt;new;<br>my $class_accessor_fast&nbsp; &nbsp; &nbsp; &nbsp; = WithClassAccessorFast-&gt;new;<br>my $class_accessor_fast_xs&nbsp; &nbsp; &nbsp;= WithClassAccessorFastXS-&gt;new;<br>my $class_accessor_complex&nbsp; &nbsp; &nbsp;= WithClassAccessorComplex-&gt;new;<br>my $class_accessor_constructor = WithClassAccessorConstructor-&gt;new;<br>my $mojo&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= WithMojo-&gt;new;<br>my $class_methodmaker&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; = WithClassMethodMaker-&gt;new;<br>my $accessors&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; = WithAccessors-&gt;new;<br>my $object_tiny&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; = WithObjectTiny-&gt;new;<br>my $spiffy&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= WithSpiffy-&gt;new;<br>my $class_spiffy&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= WithClassSpiffy-&gt;new;<br>my $direct_hash&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; = {};<br>my $class_xsaccessor&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= WithClassXSAccessor-&gt;new;<br>my $class_xsaccessor_array&nbsp; &nbsp; &nbsp;= WithClassXSAccessorArray-&gt;new;<br>my $object_tiny_xs&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= WithObjectTinyXS-&gt;new;<br> <br>cmpthese(timethese(-5,{<br>&nbsp; &nbsp; moose =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $moose-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $moose-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; moose_immutable =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $moose_immutable-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $moose_immutable-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; mouse =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $mouse-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $mouse-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_accessor-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_accessor-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_fast =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_accessor_fast-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_accessor_fast-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_fast_xs =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_accessor_fast_xs-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_accessor_fast_xs-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_complex =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_accessor_complex-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_accessor_complex-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_constructor =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_accessor_constructor-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_accessor_constructor-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; mojo =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $mojo-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $mojo-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_methodmaker =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_methodmaker-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_methodmaker-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; accessors =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $accessors-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $accessors-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; object_tiny =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $object_tiny-&gt;set_myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $object_tiny-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; spiffy =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $spiffy-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $spiffy-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_spiffy =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_spiffy-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_spiffy-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; direct_hash =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $direct_hash-&gt;{myattr} = 27;<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $direct_hash-&gt;{myattr};<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; object_tiny_xs =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $object_tiny_xs-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $object_tiny_xs-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_xsaccessor =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_xsaccessor-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_xsaccessor-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_xsaccessor_array =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; $class_xsaccessor_array-&gt;myattr(27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $class_xsaccessor_array-&gt;myattr;<br>&nbsp; &nbsp; },<br>}));</tt></p></div> </blockquote><p> <b>Results:</b> </p><blockquote><div><p> <tt>class_accessor_constructor&nbsp; &nbsp;2454/s<br>moose&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 4989/s<br>mouse&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;11187/s<br>class_methodmaker&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;35946/s<br>mojo&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 52058/s<br>class_accessor_complex&nbsp; &nbsp; &nbsp; 54325/s<br>moose_immutable&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;60575/s<br>class_accessor&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 75348/s<br>class_spiffy&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 79642/s<br>class_accessor_fast&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;89620/s<br>class_accessor_fast_xs&nbsp; &nbsp; &nbsp;107349/s<br>spiffy&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;134988/s<br>class_xsacessor_array&nbsp; &nbsp; &nbsp; 155608/s<br>class_xsacessor&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 173799/s<br>object_tiny&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 234721/s<br>object_tiny_xs&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;287949/s<br>direct_hash&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 449616/s</tt></p></div> </blockquote><p> <a href="http://hanekomu.at/blog/dev/20081101-1054-dissecting_the_moose_3-benchmarking_accessor_generators.html"> <b>Benchmarking object creation and attribute access</b> </a> </p><blockquote><div><p> <tt>#!/usr/bin/env perl<br> <br>use warnings;<br>use strict;<br>use Benchmark qw(cmpthese timethese<nobr> <wbr></nobr>:hireswallclock);<br> <br>package WithMoose;<br>use Moose;<br>has myattr =&gt; ( is =&gt; 'rw' );<br> <br>package WithMooseImmutable;<br>use Moose;<br>has myattr =&gt; ( is =&gt; 'rw' );<br>__PACKAGE__-&gt;meta-&gt;make_immutable;<br> <br>package WithMouse;<br>use Mouse;<br>has myattr =&gt; ( is =&gt; 'rw' );<br> <br>package WithClassAccessor;<br>use base qw(Class::Accessor);<br>__PACKAGE__-&gt;mk_accessors(qw/myattr/);<br> <br>package WithClassAccessorFast;<br>use base qw(Class::Accessor::Fast);<br>__PACKAGE__-&gt;mk_accessors(qw/myattr/);<br> <br>package WithClassAccessorFastXS;<br>use base qw(Class::Accessor::Fast::XS);<br>__PACKAGE__-&gt;mk_accessors(qw/myattr/);<br> <br>packag e WithClassAccessorComplex;<br>use base qw(Class::Accessor::Complex);<br>__PACKAGE__-&gt;mk_new-&gt;mk_scalar_accessors(qw/myat<nobr>t<wbr></nobr> r/);<br> <br>package WithClassAccessorConstructor;<br>use base qw(Class::Accessor::Constructor Class::Accessor::Complex);<br>__PACKAGE__-&gt;mk_constructor-&gt;mk_scalar_accessors(qw<nobr>/<wbr></nobr> myattr/);<br> <br>package WithMojo;<br>use base qw(Mojo::Base);<br>__PACKAGE__-&gt;attr('myattr');<br> <br>package WithClassMethodMaker;<br>use Class::MethodMaker<br>&nbsp; &nbsp; [ scalar =&gt; [ qw/myattr/ ],<br>&nbsp; &nbsp; &nbsp; new&nbsp; &nbsp; =&gt; [ qw/-hash new/ ],<br>&nbsp; &nbsp; ];<br> <br>package WithAccessors;<br>use accessors qw(myattr);<br> <br>sub new { bless {}, shift }<br> <br>package WithObjectTiny;<br>use Object::Tiny qw/myattr/;<br>sub set_myattr { $_[0]-&gt;{myattr} = $_[1] }<br> <br>package WithSpiffy;<br>use Spiffy -base;<br>field 'myattr';<br> <br>package WithClassSpiffy;<br>use Class::Spiffy -base;<br>field 'myattr';<br> <br>package WithClassXSAccessor;<br>use Class::XSAccessor accessors =&gt; { myattr =&gt; 'myattr' };<br>sub new {my $class = shift; bless {@_} =&gt; $class}<br> <br>package WithClassXSAccessorArray;<br>use Class::XSAccessor::Array accessors=&gt; { myattr =&gt; 0 };<br>sub new {my $class = shift; my %args = @_; bless [$args{myattr}] =&gt; $class}<br> <br>package WithObjectTinyXS;<br>use Object::Tiny qw/myattr/;<br>use Class::XSAccessor accessors =&gt; { myattr =&gt; 'myattr' }, replace =&gt; 1;<br> <br>package main;<br> <br>cmpthese(timethese(-5,{<br>&nbsp; &nbsp; moose =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithMoose-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; moose_immutable =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithMooseImmutable-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; mouse =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithMouse-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassAccessor-&gt;new({ myattr =&gt; 27 });<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_fast =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassAccessorFast-&gt;new({ myattr =&gt; 27 });<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_fast_xs =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassAccessorFastXS-&gt;new({ myattr =&gt; 27 });<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_complex =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassAccessorComplex-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_accessor_constructor =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassAccessorConstructor-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; mojo =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithMojo-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_methodmaker =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassMethodMaker-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; object_tiny =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithObjectTiny-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; spiffy =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithSpiffy-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_spiffy =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassSpiffy-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; direct_hash =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $h = {};<br>&nbsp; &nbsp; &nbsp; &nbsp; $h-&gt;{myattr} = 27;<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $h-&gt;{myattr};<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; object_tiny_xs =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithObjectTinyXS-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_xsacessor =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassXSAccessor-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>&nbsp; &nbsp; class_xsacessor_array =&gt; sub {<br>&nbsp; &nbsp; &nbsp; &nbsp; my $obj = WithClassXSAccessorArray-&gt;new(myattr =&gt; 27);<br>&nbsp; &nbsp; &nbsp; &nbsp; my $x = $obj-&gt;myattr;<br>&nbsp; &nbsp; },<br>}));</tt></p></div> </blockquote><p> <b>Results:</b> </p><blockquote><div><p> <tt>class_accessor&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 159366/s<br>mojo&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 240146/s<br>class_spiffy&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 250628/s<br>mouse&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;270458/s<br>spiffy&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 277170/s<br>moose&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;288516/s<br>moose_immutable&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;293403/s<br>class_accessor_fast&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;295518/s<br>class_accessor_constructor&nbsp; 302447/s<br>accessors&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;318795/s<br>class_accessor_complex&nbsp; &nbsp; &nbsp; 352492/s<br>class_methodmaker&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;389809/s<br>object_tiny&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;435876/s<br>class_xsaccessor&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 537101/s<br>class_accessor_fast_xs&nbsp; &nbsp; &nbsp; 572424/s<br>class_xsaccessor_array&nbsp; &nbsp; &nbsp; 620285/s<br>object_tiny_xs&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 638328/s<br>direct_hash&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1183721/s</tt></p></div> </blockquote><p>I'll leave the analysis of the results to the reader.</p> tsee 2008-11-03T09:22:53+00:00 journal Gabor's post on Perl application distribution http://use.perl.org/~tsee/journal/37759?from=rss <p>Since Gabor has disabled comments in his blog, I'll have to write in my own.</p><p>This is about <a href="http://www.szabgab.com/blog/2008/10/1225187432.html">Gabor Szabo's post about Perl application distribution</a>.</p><p>There is a whole lot of things that need taking care of with a scheme like the one that Gabor outlines. The whole notion of managing dependencies gives me headache. In my experience, there are very few distribution schemes which work well (e.g. dpkg, CPAN) and many which don't work quite as well (various CPAN-alikes for other languages, PPM). Sometimes, it's a conceptual issue, sometimes just that such a beast carries a large technical and maintenance burden.</p><p>For example, I once created a PAR::Repository with (almost) the whole of CPAN in it. Took a horribly long time to generate... and was slow as molasses because of some technical issues. (In this case, it was a combination of large DBM::Deep databases and the dynamic dependency resolution of PAR::Repository::Client. It would be somewhat faster today.)</p><p>But Gabor suggests reducing this whole nightmare to a relatively "flat" dependency tree. It shifts parts of the problem to the developer of the app (need to include all dependencies which are not part of the platform) and eliminates some.</p><p>I could go on for ages about this and Gabor has had to suffer through that already, but what I really wanted to point out is that a tiny part is solved already: Soon, I'll release a tool that can install an arbitrary<nobr> <wbr></nobr>.par into an arbitrary perl installation on the system. That may sound simple enough, but I dare you to try install into a non-running perl using CPAN(PLUS). This means that you can just ship a single, binary executable that can install stuff if you just select the perl executable and the URI of the package to install. I hate bootstrapping.</p> tsee 2008-10-28T16:28:49+00:00 journal Tired of ignorant users http://use.perl.org/~tsee/journal/37706?from=rss <p>Warning. A bit of a frustrated rant ahead.</p><p>I have many, many distributions on the CPAN and have been struggling very hard to support them as well as I possibly can. Unsurprisingly, providing user support and fixing bugs is not the only thing I do and certainly not the only thing I <i>like</i> to do. In order to be a good CPAN citizen, I have adopted various unmaintained modules and applied tiny changes or made large overhauls resulting in a total of 488<nobr> <wbr></nobr>.tar.gz files in my BACKPAN directory. Sorry if I'm coming across as a narcissistic, self-righteous bastard, but I'm trying to get across that the time it takes to maintain all that is not negligible.</p><p>One of those distributions is PAR, the Perl Archive Toolkit, and many are related. If you happen not to know what that is, it's a set of tools for packaging, managing, and distributing binary packages of Perl code and related resources and was written by Audrey Tang. It's a complex and very system dependent piece of software and I'm very, very glad that unlike many other packages in my CPAN directory, I'm not the only one who works on it. There is an active mailing list with quite a few incredibly nice, competent, and patient subscribers. In particular, there are various people who have a better overview over the code and its peculiarities on some platforms than I do. (I don't do Windows nor MacOS, for example.)</p><p>Now, each and every manual page of a PAR-related module states explicitly that support requests shall go to the PAR mailing list and bugs to the request tracker (which also sends a copy to the list, thanks to Jesse Vincent's tireless work). The PAR homepage prominently displays that same information and nowhere does it as much as mention me.</p><p>Regardless, I regularly receive support requests for the PAR modules from users to my personal e-mail address. I'm glad to hear they're using PAR, but they're usually very badly written and do not even give enough information for me to help them.<br>So far, I think I have answered all of these mails except maybe a handful that were either extremely impertinent or just unlucky because I was much busier than usual.<br>I started replying with an answer to their problem if I had one. Eventually, I grew tired of that and gave them their answer after noting that they should follow up on the mailing list.<br>This, however, has turned out to be highly ineffective. Those who got their answer usually didn't end up participating in the mailing list but just disappeared again.</p><p>How do you solve this dilemma? I still want to try to help wherever I can, but should I really spread my spare time thin between those who read the documentation, do some testing, and write a well-phrased mail to the mailing list and those who just send me a two-line mail saying that my shit doesn't install on their computer with an unspecified operating system? I'm exaggerating here. There's lots of shades in between.<br>Replying simply <i>RTFM</i> or <i>Send it to the mailing list and include your system details and logs</i> feels really rude. Forwarding the terrible thing to the list seems much worse: I'd be rude to those people who spend their spare time supporting the software. I'm actually thinking about not answering those mails at all unless they seem particularly interesting.</p><p>I can't be the only one in this situation. How do you handle this?</p> tsee 2008-10-20T19:25:52+00:00 journal PAUSE on CPAN, indexing $stuff http://use.perl.org/~tsee/journal/37362?from=rss <p> Most likely, everybody who reads this directly or indirectly depends on the operation of the PAUSE indexer (aka mldistwatch). The PAUSE indexer scans new distributions on the CPAN (really on PAUSE at that point) for the packages/namespaces and associated versions they contain, sends the uploader a friendly message with the results, and adds the information to the metadata that's used by our toolchain when people install modules from the CPAN. </p><p>The PAUSE code was written and is still maintained by Andreas K&#246;nig. It's a rather large and unquestionably a rather complex piece of software. </p><p>I don't think I'm giving anybody a big surprise if I say that being able to run this same indexer on a given tarball or zip offline may be useful for some toolchain modules. One example would be generating the META.yml provides section. </p><p>At the social event of YAPC::EU 2008, Andreas and a posse of other PAUSE admins, including me, sat down to talk about the directions our tools are heading as well as policies. I don't think anybody disagreed to that it'd be great to have components of PAUSE available individually from CPAN. But that's one ambitious goal! </p><p> A long time ago, I had spent a significant amount of time on porting the PAUSE indexer code in order to be able to index PAR distributions for injection into a PAR::Repository. I could do all sorts of simplifications for that purpose. For example,<nobr> <wbr></nobr>.par files are always in ZIP format, no tarballs, etc.<br> Last night, I decided to give it a shot at making the PAUSE indexer it's own CPAN module. </p><p> But I failed. </p><p>It turned out to be very, very tightly woven into the whole PAUSE code. I'm really not sure how I got the PAR file scanner to work on the basis of the PAUSE indexer. So I switched to a less ambitious goal: split the PAR indexer out of the PAR::Repository code into the PAR::Indexer module (and distribution) for general consumption. </p><p> That's where it stands today. For the future, I figure adding some code back into the mix and making a more generic indexer distribution would get it up to producing 98% of the same results as the real PAUSE indexer. I can do this, but:<br> Now I'd like to know, would you consider this useful?<br> And a challenge for all the testing gurus: How would you try to exhaustively test this thing agains the PAUSE indexer? </p><p> Cheers,<br> Steffen</p> tsee 2008-09-05T14:19:52+00:00 journal Why our involvement in Summer of Code 2008 was a success... http://use.perl.org/~tsee/journal/37318?from=rss <p>... where "our" means The Perl Foundation organization's, but the subject got too long. Also, keep in mind that this is a very subjective piece of text.</p><p>Initially, I was made aware of that there's a Google Summer of Code going on this year again and that there could/should be a Perl-specific organization in it by <a href="http://www.nntp.perl.org/group/perl.perl5.porters/2008/02/msg134636.html">Eric Wilhelm's short reminder mail to the perl5-porters list</a>. Without re-reading the ensuing thread, I seem to remember that the discussion that started from there was semi-productive and nobody really volunteered. But I guess that's kind of the norm in public discussions.</p><p>So Eric just went ahead and pulled it off regardless. He collected proposals, wrote Wiki pages, and announced to other lists. It turned out that collecting proposals was relatively easy. Once he got the word out, the proposals wiki page filled up quite quickly. He went on to talk to the TPF people and the previous GSOC administrators for the TPF. With their help, he tried to pin down what went well and not so well in the past years and devised a scheme which would deal more gracefully with failures similar to those that happened in the past.</p><p>As far as I can tell, many problems in the past arose from the fact that central figures either underestimated the huge heap of work involved in managing the GSOC administration or were taken aback by other, non-voluntary obligations.<br>Essentially, I think, the three most important pieces to that plan for 2008 were</p><ol><li>an effective way of recovering from the disappearance of any single person involved,</li><li>a good student application template,</li><li>an administrator who put it <b>a lot</b> of effort.</li></ol><p>What Eric did to reduce the insane work load on the central figure was to add an extra layer of people between the mentors and the admin. He split the organization's involvement in different departments and appointed <i>pilots</i> and backup <i>pilots</i> (see point 1) for each of them. For example, for the Perl 6 branch, Jerry Gay and Will Coleda shared the responsibility. This extra layer of people not only reduced the burden on the admin, they were also chosen to be more experienced with their departments than the admin ever could be. Similarly a backup was sought for each mentor.</p><p>Eric et al went on to write the organization's proposal for getting into the program at all. Needless to say, we were accepted, or I wouldn't be writing this. Furthermore, a template for the student applications was prepared. Having this template turned out to be crucial to weed out bad applicants: If they weren't able to read, understand, and answer the questions posed, they likely wouldn't be able to do a GSOC project either. If anything, then because communication is a key ingredient to the success.</p><p>Still, guiding the would-be applicants took a lot of effort. This probably was the largest piece of work Eric off-loaded to the departmental pilots as they were listed as the contact persons for applicants.<br>Next up was finding the right mentors. Here, the Perl6 people really showed their commitment. Finding mentors for the Perl6 and Parrot related projects virtually seemed to take no effort at all.<br>Generally, there were more than enough mentors available, but getting the names nailed down for some of the propsals took some time.</p><p>Once the applications were in shape and ready to be ordered by preference, since we couldn't expect to get them all funded, the most disputatious part of the process started:</p><p><i>Who gets to choose which projects get funding?</i></p><p>Just dwell on that for a bit. It's easy to find good criteria: Probability of success, usefulness to the greatest number of people, probability that the student will stick around after the summer, etc.<br>But those don't really help much. Depending on who you ask, all three of those criteria could be interpreted in favour or against almost every single application. Needless to say, every<br>mentor liked his student's application best!<br>Ideas for a mode of selecting the best applications were batted around IRC for a while and in the end, a somewhat semi-democratic process was chosen.<br>All mentors got a fixed number of votes they had to spread among the proposals. Given the outcome of that, Eric reserved the right to veto an application<br>or to move one up/down a bit.</p><p>While that may sound arbitrary, it really wasn't. There wasn't a good prediction of the outcome. When every voter has a personal agenda, the democratic process doesn't necessarily produce the<br>best ranking. However, it was clear that some applications were very good and needed to be included. Having Eric as a fallback fix for this seemed the least problematic solution.<br>I think this whole selection process was the most fragile step of all, because it had the potential to alienate some of those involved. I guess everybody reading this has an idea of what<br>confusion, disappointment, and anger do to a volunteer!</p><p>We got funding for five projects out of fourteen which (if you ask me) should have been funded. Why not more? Because Google can't fund everything and the core of their algorithm is to spread<br>the slots according to the number of student applications. Through some extra-haggling, we got a special sixth slot for the Bricolage project.</p><p>After the announcement of the accepted projects, a so-called <i>community-bonding</i> period started. During that time, the students were asked to get familiar with<br>the tools, get to know the people involved, and, if at all possible, make themselves visible in the community.</p><p>When the coding period started, things seemed to go reasonably smoothly. Getting reports from the students and mentors was more work that it should have been and Eric, again, had to do a lot of that.<br>Some students (and mentors) were good at communicating their progress, some weren't. Maybe we should have found a way to make the student's work more visible. What was your impression, dear reader, did you<br>follow along?</p><p>In the end, five out of six projects have been successful. I think that is an extra-ordinarily good result.</p><p>Looking towards next year, <i>what needs to be improved over this summer?</i> </p><p>Maybe you caught the problem with the master plan I outlined with three bullet points above. Points 1) and 3) contradict each other.<br>If Eric had disappeared, say, in the student selection process, our involvement in the SOC might have fallen flat on its face. I was told that something<br>like that had happened before. To my information, Eric put in more than a whole month of full-time work. We can't rely on anybody doing that next year.<br>So we need to find a way to share more of the burden among more people and make every single person involved non-crucial if at all possible.</p><p>Furthermore, I think, we need to increase the size of our student pool. We had many more mentors and projects than applications. We have to find a better way to reach out.<br>Ideally to universities. This is a bigger problem than just the TPF GSOC involvement, however.<br>As a minor nit, maybe the project proposals should have been a little more elaborate and glossy. Who wants to rewrite Module::ScanDeps because it's horrid? (That was my proposal, so I can trash-talk it at leasure.) I should have mentioned the shiny glitz of working with the best Perl introspection tool in existence (PPI) or something along those lines. Everybody knows dependency scanning is fun!</p><p>Finally, maybe we can find a more efficient application selection scheme. Maybe we can make it so that it isn't necessary than a single person needs to make the final decision.<br>It's not a fun thing to do.</p><p>I doubt many readers are left at this point. Regardless, I'd like to extend my thanks not only to Eric, whose work I have praised enough in the above text,<br>but also to the mentors for dedicating their spare time and Google for funding the whole program. Most of all, however, I want to congratulate the successful students for<br>having the stamina to stem their projects. You've done great work and I really hope you'll stay involved.</p><p>Thanks for reading.</p><p>Steffen</p> tsee 2008-08-31T11:35:51+00:00 journal YAPC::EU talk on PAR http://use.perl.org/~tsee/journal/36918?from=rss <p>I'm currently preparing the slides for my "Application deployment and dependency management with PAR" talk at YAPC::EU in Copenhagen.</p><p>There's a couple of things on my mind which I want to talk about. However, I realized that my view of what's important and/or interesting may be quite different from what other people think.</p><p>Hence, I'd like to give you the opportunity to tell me what you'd care about in the context of the suite of PAR modules. This way, I might get a somewhat better idea of what people other than myself would expect from a talk on the topic. Feel free to post a reply or send me an email (my PAUSE ID is SMUELLER).</p><p>Thanks for your input!</p><p>Steffen</p> tsee 2008-07-15T20:03:54+00:00 journal Class::XSAccessor - A saner, manual AutoXS with more Fast http://use.perl.org/~tsee/journal/36059?from=rss <p>In the last journal entry, I told a lengthy story about an XS and perl API learning experiment that resulted in <a href="http://search.cpan.org/dist/AutoXS">a module that scans subroutine opcodes at run-time to find candidates for replacing with an XSUB</a>.</p><p>Now, the trouble with that approach is that scanning the op tree for such methods takes a long time. It's a cool idea and maybe even sensible if you have an existing system with a lot of code that is long running and needs to be as fast as possible, but let's face it: No sane person would like to add something as fragile as an op-tree scan to such a system. (Though you have nothing to lose except compilation time.)</p><p>So I ripped out the XS code with the cleverish fake currying that generates the getters, put it in its own module, added an implementation of setters (that was really trivial in XS), and <a href="http://search.cpan.org/dist/Class-XSAccessor">uploaded it to CPAN</a>. The result is potentially the fastest accessors you can get for a Perl hash-based object bar hand-rolled XS. And that would only save a C array access, two C struct accesses and perhaps some slight book-keeping.</p><p>On a related note, I continued hacking on <a href="http://search.cpan.org/dist/B-Utils">B::Utils</a> which I used for the original AutoXS hack.</p><p>For all readers who'd rather jump from the nearest TV broadcasting tower than look at the modules that start with a B: B::Utils provides convenient tools to inspect the op tree that is generated by the perl compiler. Among those, there is a function called <i>opgrep</i> which - you guessed right - matches conditions against an op tree. These conditions are specified as nested hash structures that resemble the op tree itself.</p><p>In the past days, I added a method to the B::OP objects that can dump the op tree as such a pattern for use with op_grep(). This should make it much easier to extend the AutoXS module for more complicated scanning. Additionally, opgrep() can now, while scanning, extract ops from within the op tree that it is traversing. That way, it's no longer necessary to walk the op tree yourself in order to extract information after you've verified that it matches your expectation.</p><p>Cheers,<br>Steffen</p> tsee 2008-04-04T17:42:14+00:00 journal AutoXS.pm: A learning experiment in XS and perlapi http://use.perl.org/~tsee/journal/36025?from=rss <p> I'm a relatively long-time reader of the perl5-porters mailing list. Somewhat recently Nicholas Clark posed a few small challenges intended to draw more people into the Perl core development. I thought it was a great idea, but couldn't follow up on it at the time. I said I liked the concept on the #p5p IRC channel and so I thought I should learn a bit more about the Perl core and XS. While not the same, I presume that having strong knowledge about the XS/Perl API would be a jump start to understanding the core. </p><p> Skip ahead a few weeks. I have since submitted my thesis, went on vacation, and started a new job. But still no progress on my plan to learn XS. Until yesterday. I was idly playing with the <a href="http://search.cpan.org/~rgarcia/perl-5.10.0/ext/B/B.pm">B</a> and <a href="http://search.cpan.org/dist/B-Utils">B::Utils</a> modules when I had a pretty good idea for an interesting learning and experimentation project: <a href="http://search.cpan.org/dist/AutoXS">AutoXS</a>. </p><p> Essentially, the idea started out with using B to scan a running Perl program for subroutines or methods of a particular type. Typically, the simplest and most recurring methods are accessors for hash-based objects. (Just search CPAN for accessor-generators...) The next step is to replace the identified objects with precompiled XSUBs that accomplish the same task but having been written in C, doing so faster. </p><p> For simple accessors, that seems like a simple enough task at first: Write the XS code to access a value in a hash reference which is stored on the stack. Apart from the fact that it took me surprisingly long and a lot of patient help from the friendly people on #p5p to get the XS right (thanks!), this may seem like a simple enough task at first. But where's the hash key coming from? You can't expect the user to pass it in as an argument because that's beside the point. You can't know the key name at XS compile time because that's when the module's built. You currently cannot find the package and name using which the current method/subroutine was called either. So what's the answer? Something like currying. I don't think I need to explain to anyone what that is. But maybe I should mention that it's in C, not Haskell or Perl. C doesn't have currying. </p><p> The solution took some time in coming. The XS <i>ALIAS</i> keyword allows for compile time aliases to a single XSUB. The aliases can be distinguished from within the XSUB by means of an int variable whose value can be associated with the aliases. (Bad explanation, I guess, have a look at perlxs for a better one.) This doesn't get us all the way to currying, though. I had a look at the generated C code and realized that I could just write similar code on my own and assign new values of that magical integer to each new alias of the accessor prototype at run time (CHECK time, really, but run time would work, too). Then, all that was left to do was to put the hash key for the new alias into an array indexed with said numbers. Voila - fake currying for that XSUB. </p><p> By now, it all actually works. The scanner indentifies <a href="http://search.cpan.org/~smueller/AutoXS-0.01/lib/AutoXS/Accessor.pm#RECOGNIZED_ACCESSORS">quite a few typical read-only accessors</a>. The XSUBs are, according to my crude measurements, between 1.6 and 2.5 times faster than the original accessors. If you're calling those accessor methods in a tight loop, that might actually make a bit of a difference. I wrapped it up in a module, <a href="http://search.cpan.org/dist/AutoXS">AutoXS</a>, and gave it the best interface ever. That is, none. You just say </p><p> <code>use AutoXS::Accessor;</code> </p><p> to get the accessor scan for all methods in the current package. More seriously, one could let the user flag eligible methods or even apply the scan globally. But that's not the point. It's just kind of fun that it works <i>at all</i>. </p><p> Cheers,<br> Steffen</p> tsee 2008-04-01T19:32:05+00:00 journal perl 5.12, strict by default http://use.perl.org/~tsee/journal/35213?from=rss <p>So I did it. I proposed to the perl5-porters that we should enable "use strict" by default for some future code. This may be a little less preposterous than it sounds at first, so please wait with hitting the "reply" button until you read the whole of this.</p><p>My proposal basically goes as follows:</p><ul> <li>Add a feature called "strict" to feature.pm.</li><li>Include that feature into the set of default features which are loaded by "use feature ':5.11'" or even just "use 5.11.0;".</li><li>Add a special case for the -E switch to perl so strictures aren't enabled by default for one-liners.</li></ul><p> I'll include my original rationale here: </p><p> <cite> Personally, I've always wanted perl to have strictures on by default for my code. I would think that 95% of all code bases which were written in this century and which are of non-negligible size import "strict". I don't use strictures for one-liners, of course, but for anything else it's a must. It seems to me like others have similar views on this. Try posting some code without "use strict" to some newsgroup or forum and ask for help. Make sure not to give out your email address, though. </cite> </p><p> <cite> "use 5.10.0;" already auto-imports feature.pm and loads the 5.10 specific features. </cite> </p><p> <cite> How about having "use 5.11.0;" (or 5.12.0) automatically import strict along with the 5.10+5.11 feature set? Naturally, the -E switch for one-liners should *not* do that. </cite> </p><p> <cite> This would *not* break backwards compatibility. This would not affect one-liners. This would optimize for the common case: If you write enough code to make importing optional features worthwhile, odds are very high you'd be importing "strict" anyway. The 5% who need to disable strictures again can still add a "no strict;" statement. </cite> </p><p> <cite> strictures-correct code has been best-practice for a long time now. Let's make it the default for *new* code. </cite> </p><p> <cite> Just think of strictures as a feature. It just makes sense. </cite> </p><p> To my surprise, the proposal has been received with very positive feedback. So I wrote the patch. </p><p> With some luck, we'll get strictures by default in 5.12! Flame away! </p><p> Cheers,<br> Steffen</p> tsee 2007-12-27T17:10:24+00:00 journal Evil PAR tricks, issue 0: Binary including a binary http://use.perl.org/~tsee/journal/32140?from=rss <p>In my journal entry from December 22, 2006, I said I'd used a few hacks to package <i>parinstallppdgui</i> into an executable binary using PAR::Packer. I'll explore that further here:</p><p> <i>parinstallppdgui</i> is mostly a GUI front-end to <i>parinstallppd</i>. It doesn't use <i>PAR::Dist::InstallPPD</i> internally, but uses the system to run <i>parinstallppd</i> so if the child process bombs out, the GUI process can show a friendly warning to the user (comprised of the STDERR and STDOUT of the child process) instead of crashing. That's all very simple if you have perl on your system, of course.</p><p>Now, if you want to package <i>parinstallppdgui</i> or any other Perl script that uses another Perl script into an<nobr> <wbr></nobr>.exe, you'll quickly find out that without a perl on the target system, these two scripts cannot share the same interpreter. The obvious "solution" is to package both of them up into an<nobr> <wbr></nobr>.exe separately and ship them both. This has several problems. First, you need to ship two executables instead of one. Second, the first executable won't necessarily know where to look for the second if the user doesn't put them in PATH. Adding a couple of FindBin hacks isn't great at solving this, either! Third, these two executables will have a lot in common - starting with all the core Perl modules.</p><p>So I took a slightly better, yet more complicated route. The process is as follows:</p><ol><li>Add a <b>special case</b> to the parent's source code for execution from inside a PAR binary:<br><code><br>if (defined $ENV{PAR_TEMP}) {<br> &nbsp; &nbsp; &nbsp; &nbsp; require Config;<br> &nbsp; &nbsp; &nbsp; &nbsp; require File::Spec;<br> &nbsp; &nbsp; &nbsp; &nbsp; $ENV{PATH}<nobr> <wbr></nobr>.= (defined $ENV{PATH} ? $Config::Config{path_sep} : '')<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; . File::Spec-&gt;catdir($ENV{PAR_TEMP}, 'inc');<br> &nbsp; &nbsp; &nbsp; &nbsp; $ENV{PAR_GLOBAL_TEMP} = $ENV{PAR_TEMP};<br>}<br></code><br>This forces all PAR'd executables that are started by this process to use the same cache area as this process. This might be a dangerous thing to do if both binaries contain different, incompatible versions of some files. But we control everything here, so it should be okay.</li><li> <b>Package</b> <i>parinstallppdgui</i>, the parent script:<br><code>pp -o parinstallppdgui.exe parinstallppdgui</code></li><li> <b>Package</b> <i>parinstallppd</i>, the child script:<br><code>pp -o parinstallppd.exe -l expat parinstallppd</code><br>(We include the expat library.)</li><li>Remove all files from the child script that are also available from the parent. This makes the child <b>dependent</b> on the parent. Use the <i>pare</i> utility from the PAR::Packer distribution's contrib/ directory for this:<br><code>pare -u parinstallppdgui.exe parinstallppd.exe</code></li><li> <b>Package</b> the parent script again, but this time include the child script as an extra file into the executable:<br><code>pp -o parinstallppdgui.exe -a parinstallppd.exe -l expat parinstallppdgui</code></li><li> <b>Ship</b> the combined binary only.</li></ol><p>Cheers,</p><p>Steffen</p><p>P.S.: A third and even better solution might be to package both of the scripts into the same binary and symlink or copy that binary to <i>parinstallppd.exe</i> and <i>parinstallppdgui.exe</i> and let PAR figure out what to run. This is cool if you have symlinks and sucks if you don't.</p> tsee 2007-01-14T15:50:00+00:00 journal CPANTS Distribution Checker (without PAUSE upload) http://use.perl.org/~tsee/journal/32090?from=rss <p>So CPANTS was down for a couple of weeks. This morning, I noticed it's back, but so far without the cron job that updated the data nightly.</p><p>For me, CPANTS has been a way to check my eighty or so CPAN distributions for packaging shortcomings. Unfortunately, that check happens only after I made the release. I installed Module::CPANTS::Analyse which comes with <i>cpants_lint.pl</i>. Given a distribution file name, it runs the CPANTS metrics against it.</p><p> Since Module::CPANTS::Analyse comes with quite a few prerequisites, I set up a simple web service which you can use to check your distributions before uploading. The interface is a little cumbersome, but I think it's worthwhile. Just upload your distribution at <a href="http://steffen-mueller.net/cgi-bin/cpants-limbo/check.pl">http://steffen-mueller.net/cgi-bin/cpants-limbo/check.pl</a> and then visit the result URL given by that script after a few minutes.</p><p>The reason for the delay is that I didn't want to install all prerequisites on the hosting account and certainly didn't like running <i>cpants_lint.pl</i> from a CGI process. Thus, my local server fetches distributions-to-scan from there and uploads a result text every three minutes.</p><p>Cheers,<br>Steffen</p> tsee 2007-01-06T13:18:21+00:00 journal Windows, the unloved stepchild... (when it comes to Perl) http://use.perl.org/~tsee/journal/31988?from=rss <p>Perl on Windows has always been a bit of a problematic case. It doesn't usually come with (C) development tools and users are, on average, rather clueless. Additionally, a lot of Perl code is written on *nix-like systems and many authors aren't aware of the portability pitfalls. Interestingly, the most common problem are path separators, which is reasonably simple to work around if you know File::Spec[1] or Path::Class[2]. </p><p>I have been doing quite a bit of porting work recently and in the process, I have patched many CPAN modules to work on Windows [3]. Barring downright unportable stuff like Linux::*, the toughest problems stem from modules using signals, fork() or some other form of IPC. For example, it's nigh on impossible to make Test::HTTP::Server::Simple work well on Windows because it needs to kill its forked children. That can make the Perl interpreter dump core on Windows.</p><p>But I'm digressing. I wasn't going to write about portability but about something else entirely:</p><p>Originally, the I-don't-have-a-C-compiler problem on Windows was somewhat solved by ActiveState with the Perl Package Manager (PPM) which comes with their ActivePerl distribution. They provide a variety of CPAN modules in pre-compiled form which you can install using PPM.</p><p>I say "somewhat" because that doesn't really satisfy me. Not that they aren't doing swell work, but it's hard for a single organization to keep up as pointed out by Ovid [4]. This is the reason I jumped at the Strawberry Perl project [5] which goes one step further and delivers a working (and free!) C compiler with the Perl distribution.</p><p>Strawberry Perl enables me to use the CPAN shell on Windows just as I do on Linux and it works with 90% of all modules. Notable exceptions are those which require external libraries, of course. We are working on solving this issue, but it's not pretty. Until then, it might be tempting to want to use ActiveState's PPM tool to install those problematic modules, but it turns out there is only a very old version of it available outside their Perl distribution. The nice GUI which they introduced in PPM4 isn't available outside ActivePerl at all. It's a valid decision to not make the newer PPM's a free CPAN distribution, of course. </p><p>Frankly, this bothered me a little. So I wrote a module to convert PPM packages to PAR distributions [6]. Given the URI of a PPD document, it creates a<nobr> <wbr></nobr>.par file which you can install using PAR::Dist and without involving any PPM-related software at all. It's really quite simple:</p><p> <code> ppd2par --uri http://theoryx5.uwinnipeg.ca/ppms/PDL.ppd -p 5.8 </code> </p><p>It's still a little more tedious than the nice PPM GUI, of course. So I wrote another tool which automatically appends the installation step [7]: </p><p> <code> parinstallppd --uri http://theoryx5.uwinnipeg.ca/ppms/PDL.ppd </code> </p><p>In some cases, it's necessary to tell it which implementation (5.6 or 5.8) to use because the repository maintainers do not provide great meta data. But it works fine for me. If you know how much work it can be to compile the PDL module, you will certainly agree! Still, this is not for the faint of heart, so yesterday, I wrote a GUI front-end [8]. It is <b>not</b> a package manager like PPM, but just the beginnings of a simple GUI front-end to the <i>parinstallppd</i> tool.</p><p>Since these tools require XML::Parser, which requires expat which is an external dependency, I have built a stand-alone executable from <i>parinstallppdgui</i> and put it on my home page [9]. Getting it to work properly with PAR required some trickery. Perhaps I'll explore that further in a separate journal entry.</p><p>So, wrapping it up, you can now enjoy the benefits of a Unixy Perl installation with the added benefit of installing precompiled PPM packages if necessary.</p><p>Cheers,<br>Steffen</p><ol> <li> <a href="http://search.cpan.org/dist/File-Spec">http://search.cpan.org/dist/File-Spec</a> </li><li> <a href="http://search.cpan.org/dist/Path-Class">http://search.cpan.org/dist/Path-Class</a> </li><li> <a href="http://win32.perl.org/wiki/index.php?title=Vanilla_Perl_Problem_Modules">http://win32.perl.org/wiki/index.php?title=Vanilla_Perl_Problem_Modules</a> </li><li> <a href="http://perlmonks.org/index.pl?node_id=587162">http://perlmonks.org/index.pl?node_id=587162</a>, <a href="http://use.perl.org/~Ovid/journal/31899">http://use.perl.org/~Ovid/journal/31899</a> </li><li> <a href="http://win32.perl.org/wiki/index.php?title=Strawberry_Perl">http://win32.perl.org/wiki/index.php?title=Strawberry_Perl</a> </li><li> <a href="http://search.cpan.org/dist/PAR-Dist-FromPPD">http://search.cpan.org/dist/PAR-Dist-FromPPD</a> </li><li> <a href="http://search.cpan.org/dist/PAR-Dist-InstallPPD">http://search.cpan.org/dist/PAR-Dist-InstallPPD</a> </li><li> <a href="http://search.cpan.org/dist/PAR-Dist-InstallPPD-GUI">http://search.cpan.org/dist/PAR-Dist-InstallPPD-GUI</a> </li><li> <a href="http://www.steffen-mueller.net/parinstallppd/parinstallppdgui.exe">http://www.steffen-mueller.net/parinstallppd/parinstallppdgui.exe</a> </li></ol> tsee 2006-12-22T10:19:55+00:00 journal Perl Advent Calendar 2006-12-16 http://use.perl.org/~tsee/journal/31935?from=rss <p>I just read today's "Perl Advent Calendar" (<a href="http://perladvent.pm.org/2006/16/">http://perladvent.pm.org/2006/16/</a>) entry and found out that in today's entry, Jerrad Pierce introduces one of my modules, Number::WithError (<a href="http://search.cpan.org/dist/Number-WithError">http://search.cpan.org/dist/Number-WithError</a>). Yay!</p><p>Now, he certainly has interesting things to say, but to cut down on the dependencies, he suggests commenting out three lines of the module code. Sure, I mean, it's Free Software. You could do that and even release the same module with just this change under a different name! But he goes on to say:</p><p><div class="quote"><p>If you'd rather avoid installing the first three, you can do so by commenting out the following lines in v0.06 of Number::WithError with no apparent side-effects</p></div><p> <i>"With no apparent side-effects"?</i> <br> The test suite is failing all over the place because you have just removed the implementation of the tangent function!</p><p>Now, to be fair, he has a point. The dependencies are, at this stage, just there for tangent, which would otherwise be </p><p> <code> sub my_tan { sin($_[0]) / cos($_[0]) }<br> # or rather in the module's context<br> sub my_tan { CORE::sin($_[0]) / CORE::cos($_[0]) } </code> </p><p> Originally, I intended to leverage the implementation of the various less-common trigonometric and hyperbolic functions found in Math::Symbolic and add them to the repertoire of Number::WithError. I haven't done so because I didn't need it at the time. Then I forgot about it. Doh.</p><p>If it was just for the implementation of those ten or so functions, it would be questionable whether the dependencies (which are all pure-Perl, at least) are worse than duplicating a couple of functions. But there's more: Gaussian error propagation involves derivatives. Other than that, it's a straightforward formula. Math::Symbolic can compute derivatives for you. Got the idea?</p><p>Basically, one would get the implementation of most code in Number::WithError for free by generating the derivatives of the functions using Math::Symbolic and compiling them to Perl subroutines using Math::SymbolicX::Inline during the package BEGIN time. This made particular sense since hand-coding every routine was rather prone to small errors and unless he does the math by hand, it's not necessarily obvious to a programmer reading the code why it does what it does.</p><p>Fact is, I didn't do it back then. I don't even remember why. Probably some time constraint or "let's get this working before I make it elegant" thinking. It doesn't matter. I don't think publicly suggesting users to comment out random code is a good idea. I'll forward the bug reports stemming from this if they surface.<nobr> <wbr></nobr>;)</p><p> Steffen</p> tsee 2006-12-17T11:26:30+00:00 journal What if you find a bug in a module from CPAN? http://use.perl.org/~tsee/journal/31915?from=rss <b>What do you do if you find a bug in a module from CPAN?</b> <p>(Supposing it's not your own.)</p><p> Personally, I think one should take the following steps:</p><ul> <li>Possibly investigate and come up with a fix if it's not too much to ask. </li><li>Check the module documentation for suggested methods of bug-reporting and act accordingly.</li><li>Unless otherwise noted, file a ticket in the RT queue of the module/distribution and take a minute to include all necessary details.</li> </ul><p> This is really where even most experienced people seem to stop. Unfortunately, that is often not enough. Not all module authors use RT. Some deliberately ignore it, some don't like it, some don't know or understand it. Some have disappeared altogether. If one filed a bug against an actively-maintained module and there was evidence that the author used RT, this would be enough. Otherwise: </p><ul> <li>Keep an eye on the ticket. If it remains untouched for a couple of weeks or months and there are no indications that the distribution is maintained, consider contacting the author/maintainer by other means such as email. Of course, it is important to be tactful. Do not harrass these people. They donated their work and time just as you do.</li><li>If the author has completely disappeared and the emails bounce, please consider contacting modules@perl.org about it.</li><li>If it's evident the module is unmaintained, you can then apply your fix (and, if you like, some outstanding tickets) and, with the blessing of modules@perl.org, upload a new distribution to CPAN.</li></ul><p> Now, you might say that you don't have the time to do this. <i>Nobody</i> really does. But if you had the time to write a coherent bug report on RT, it would be a waste of your previous effort not to make sure the module actually gets fixed. </p><p> Getting in touch with authors can be tedious and time consuming, but most are willing but busy and gladly accept help. If you have a fix, consider donating a couple more minutes to apply it and roll a new distribution. </p><p> CPAN is Perl's greatest strength and the abundance and quality of software on CPAN is our best selling point. Let's keep it that way. </p><p> Thanks for reading! </p><p> Steffen</p><p> <i>Update:</i> Now mentioning that it's a good idea to check for the author's preferences concerning bug reports.</p> tsee 2006-12-14T15:41:59+00:00 journal