use Perl

Code Profiling

posted by pudge on 2001.10.25 10:11   

jdavidb writes "I've come to a situation where I have a very inefficient search algorithm taking six minutes to accomplish what should be a simple task. I've brought the minds of many coworkers to bear on it, and we've collectively realized that it's difficult to figure out exactly where the performance bottleneck is. I need to optimize something, but what?

Obviously, the solution is to profile the code and find out where the real bottleneck is. But I've never done that before, not even in C. I've heard rumors of DProf and stuff like that, but I don't know if that's the module I should use, or even what my choices are. What options do I have for code profiling, what tutorials, introductions, or other articles exist out there, and what common novice goofs am I about to spend today repeating?"

• DProf: yes (Score:1)

  by jjohn (22)

  When you have a script that is composed of many subroutines, DProf can tell you how much time each subroutine is taking. Programming Perl, 3rd Edition [oreilly.com] has a chapter by Gnat Torkington [frii.com] on this very subject. Before I read that small section, I was a lost about profiling as anyone could be. There's a bit more to it than just using DProf, because that module writes it's findings in a rather cryptic format. Instead, there's a front end, dprofpp, that will help.

  dprofpp -p scriptname

  should do it.
  • Re:DProf and GraphViz? (Score:1)

    by 2shortplanks (968)

    I remember acme [perl.org] talking about graphing using DProf, creating easy to understand diagrams at YAPC::Europe 2000 [yapc.org].

    Tap tap tap

    Ah, here are his slides: http://www.astray.com/graphing_perl/ [astray.com]. Maybe you could convince him to post some source code somewhere....after all we do know how he loves to go on about graphing.
• brian's article (Score:1)

  by acme (189)

  brian d foy has written an article entitled Profiling in Perl [ddj.com] which covers using Devel::SmallProf [cpan.org], a per-line code profiler. It is quite good for small code snippets. The article walks you through how to use the module.

  For larger bits of code, Devel::DProf [engelschall.com] is very good. The documentation walks you through how to use the module simply.

  Oh, and Devel::Cover [cpan.org] is starting to do profiling too...
• Profiling (Score:1)

  by BobGoolsby (1098)

  There is also a chapter in Martin Brown's "Debugging Perl" on DProf (chapter 12 - Automatic Optimizations"

  --
  I go back to sleep now....
• Thanks! (Score:1)

  by jdavidb (1361)

  Thanks for the help, folks. My 6 minute 40 second worst (?) case search is now virtually instantaneous. Actual problems turned out to be in reading from the disk. File::Tail::read to be specific. Not a problem with the module, just my abuse of it.

  For the record, it doesn't help you to read a whole file in and then try to perform an efficient search. You absolutely have to, have to, have to read only parts of the file with seek/tell/sysread type functions. There's no getting around it! But, you'll b

  --
  J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
  • Re:Thanks! (Score:1)

    by hardcode (2374)

    I've had a similar experience here, I need to pass thru a 200000 line text file 3000+ times.
    
    I was converting a date+time in ISO format to epoch seconds every time using Time::Local, now I do it up front (an obvious optimisation, didn't even need a profiler) and that *did* improve the speed dramatically, but it was still slow. The solution? Pre-build an index into the file to avoid having to go thru each line (even tho I was calling next if the line wasn't in the timeframe I was looking for) and lo and beh