Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

acme (189)

acme
  (email not shown publicly)
http://www.astray.com/

Leon Brocard (aka acme) is an orange-loving Perl eurohacker with many varied contributions to the Perl community, including the GraphViz module on the CPAN. YAPC::Europe was all his fault. He is still looking for a Perl Monger group he can start which begins with the letter 'D'.

Journal of acme (189)

Thursday July 06, 2006
08:59 AM

Parallel foreach

[ #30191 ]

Computers are getting faster, that's for sure. However, they are also getting more cores: new laptops thesedays are dual-core and servers are four-or-more-core. Cores are a fancy word for something a bit like another processor. I happen to have lots of things I want to process independently, but if I only use a single process I'll only use one core, a quarter of those available on my server. That's wasting CPU power. The solution is to do more than one thing at a time and common solutions for this are threading and forking. I've found a particularly neat solution which is a very nice idiom too: parallel foreach with Proc::ParallelLoop Something along the lines of the following, which will parallelise the loop by forking 4 workers at a time:

pareach \@todo, \&generate, { Max_Workers => 4 };

The nice thing is that Linux balances each long-lived process on a core, and suddenly my program runs about four time faster! (Okay, so cores aren't complete CPUs - they tend to have dismal floating point performance - but all I am doing are integer calculations so it is mostly the same in my case).

There are many modules on the CPAN which do something similar, but I particularly like the fact that this is a cute idiom: exactly like a foreach, but slightly parallelised. Neat!

See you at the London.pm social meeting tonight!

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • so cores aren't complete CPUs - they tend to have dismal floating point performance

    That may depend upon brand and architecture. But really, Parallel-Each dispatch of floating point operations would be the wrong way, multi-core or single-core SMP.

    • Sun - UltraSparc T1 aka Niagra, T2000 - True.
      * [sun.com], ** [theregister.co.uk].
      Prior USiii dual-core models had 1 FPU per Core, but it wasn't safe to max them. *** [sun.com].
    • IBM/Apple - PowerPC E.g., QuadCore PowerMac G5 - False.
      2xFPU/core = 8 FPUs in the 2 socket, 4 core G5 * [apple.com], ** [ibm.com]
    • IBM - Powe
    --
    Bill
    # I had a sig when sigs were cool
    use Sig;
  • Isn't there a Perl 6 way to do this?
    --
    Bill
    # I had a sig when sigs were cool
    use Sig;