Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

ethan (3163)

ethan
  tassilo.von.pars ... AMrwth-aachen.de

Being a 25-year old chap living in the western-most town of Germany. Stuying communication and information science and being a huge fan of XS-related things.

Journal of ethan (3163)

Tuesday December 14, 2004
03:11 AM

Benchmarking perls

[ #22299 ]

Partly due to the fact that I didn't have anything interesting to do, I wrote a little set of modules to benchmark perls against each other. There already is perlbench on the CPAN but I found the way tests had to be written inconvenient.

According to the results I get, realworld programs seem to get faster on recent perls. Some other things on the other hand are much slower, most notably regexes (almost by a factor 2 when comparing 5.5.4 with a threaded 5.8.6). The results:

Benchmarks     | perl5.5.4 | perl5.6.2 | perl | perl5.8.6 | perl5.8.6th | Weight
---------------+-----------+-----------+------+-----------+------------- +-------
bench/loops    |   1000    |    906    |  814 |    731    |     782     |     70
bench/regex    |   1000    |   1258    | 1978 |   1457    |    1977     |    116
bench/recurse  |   1000    |    961    |  983 |    956    |     991     |     73
bench/wave     |   1000    |    860    |  994 |    894    |     925     |    236
bench/autoload |   1000    |    968    | 1095 |   1078    |    1159     |    139
bench/substr   |   1000    |    947    |  984 |    986    |     957     |    164
bench/mail     |   1000    |    698    |  903 |    748    |     959     |    198
---------------+-----------+-----------+------+-----------+-------------+-- -----
Overall        |   1000    |    910    | 1085 |    960    |    1082     |   1000

The fourth column is the ordinary Debian testing-perl (perl5.8.4 with threads). The benchmarks can be found here. Some tests I consider almost irrelevant, namely autoload which is an OO version of the Fibonacci-number generator where each recursive instance is autoloaded. Also, loops is not interesting because empty loops are unlikely to show up that often. Real world programs are wave which decreases the volume of a 18meg WAV file, mail which walks through a 28meg mailbox and substr which calculates the length of the longest common substring of perldoc -tT perlfunc. It's 647 by the way, would you have guessed?

The rightmost column is the relative amount of time this test took on the leftmost perl. This number is used for calculating the weighted mean in the bottom row. In case the Weight column doesn't add up to 1000, that's due to some unclever rounding of my code.

Finally, the module which does the timing and that is included from each benchmark script is this:

package Perl::Benchmark::Lib;
 
use strict;
use Time::HiRes;
 
use base qw/Exporter/;
use vars qw/$VERSION @EXPORT/;
$VERSION = '0.01';
 
$SIG{__WARN__} = sub {};
$SIG{__DIE__} = sub {};
 
@EXPORT = qw/bench_adjust/;
 
my $corrective = 0;
 
bench_adjust();
 
sub bench_adjust {
    my ($code, $times) = @_;
    if (defined $code) {
    $times ||= 1;
    $corrective += eval sprintf <<EOEVAL, "$code\n" x $times;
my \@td = Time::HiRes::gettimeofday();
%s
tv_interval(\@td);
EOEVAL
    }
    @Perl::Benchmark::Lib::PB_timer = Time::HiRes::gettimeofday();
}
 
sub report_timing {
    print Time::HiRes::tv_interval(\@Perl::Benchmark::Lib::PB_timer) - $corrective;
    print "\n";
}
 
END {
    report_timing();
}
 
1;

It dumps the number of seconds of the runtime to stdout where it is picked up by another module that does all the bookkeepting and eventually spits out the table you see further above with the help of Text::Table. A very nice module, by the way.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Obviously a column is missing :)
    • I was just lacking the patience to start up buildaperl and from there build a fresh bleadperl. But now that I have a fairly useful benchmark environment, I am quite sure I'll use it the next time I patch around in bleadperl.

      I also would like to release it as Perl::Benchmark to the CPAN. Prior to that, it needs some polishing and maybe some more features. Also, I would like to create a benchmark suite that only relies on core-modules but is still real-world-ish enough to be useful. Maybe I find a way that i
  • Some other things on the other hand are much slower, most notably regexes (almost by a factor 2 when comparing 5.5.4 with a threaded 5.8.6).
    If I may just take one uninformed guess on the cause: Unicode. Internal Unicode support was added starting with 5.6.0.
    • Internal Unicode support was added starting with 5.6.0.

      Ah, that's plausible. I was also thinking of possible causes for this decrease in performance but didn't think of the obvious case.

      And the additional slow-down caused by threads is then also fairly obvious: Perl's regex engine, which is very non-reentrant by nature, has to work more reentrantly now.

      Other than that, I agree with mir's comment that with all the new features and bugfixes, the results are very favourable for perl and its porters. I alw
  • I think these numbers are very impressive. If you consider the increased features (Unicode!) and the bug fixing between 5.5.4 and 5.8.6, then keeping more or less the same speed is quite a feat. Kudos to the porters!

    --
    mirod