Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

jdavidb (1361)

jdavidb
  (email not shown publicly)
http://voiceofjohn.blogspot.com/

J. David Blackstone has a Bachelor of Science in Computer Science and Engineering and nine years of experience at a wireless telecommunications company, where he learned Perl and never looked back. J. David has an advantage in that he works really hard, he has a passion for writing good software, and he knows many of the world's best Perl programmers.

Journal of jdavidb (1361)

Tuesday June 01, 2004
08:29 AM

Big directory

[ #19029 ]

Working in a directory with a large number of files:

$ time ls -1  # GNU ls
real    2m22.169s
user    0m2.080s
sys     0m5.850s
$ time /bin/ls -1  # Solaris ls
real    0m18.013s
user    0m1.230s
sys     0m1.390s
$ time ~/perlls.pl  # Custom Perl program
real    0m22.355s
user    0m1.200s
sys     0m1.260s

Interesting facts. I'd been fretting about how slow using the standard utilities was, until I remembered that I had installed GNU versions of these utilities in my ~/bin, presumably because they were better. I'm sure GNU ls does something like load the entire directory into memory before printing it out or something, in order to use a "different" implementation method and avoid copyright infringement.

Maybe I should install busybox...

For the record, I ended up using a custom Perl program to do what I was trying to do, and it still seemed the fastest solution (even including development time for the program).

Update: it made more sense when I got the proper figures posted, instead of duplicating the figures for Solaris ls. That turns out to be a bug with Cygwin cut and paste...

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • This came up [freebsd.org] recently on the freebsd-current list and the conclusion was that not only do you have to turn off statting each file (no -l), you also have to turn off sorting of the results list. This is ls -f in FreeBSD.

    -Dom

    • That's a -1, not a -l. The reason I added -1 to the test was to make sure none of the ls versions was spending extra time trying to compute the number of filenames that could fit on a line or whatever. (And also because my little hacked-up Perl version couldn't do that.)

      I posted to the GNU coreutils (combination of fileutils, which includes ls, and two other packages) bugs mailing list, and they responded by asking the version of my GNU ls and my OS. Turns out ls from GNU coreutils 5.0 (my version) has

      --
      J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers