All the Perl that's Practical to Extract and Report
use Perl Log In
mako132 (2925)
Repeated Code
As we ended this week's code review session, the facilitator remarked that next week's code (two large CGIs of 2600 and 3900 lines each) would be difficult to get through. I replied that they are actually about a third of the real size because they consist of so much repeated code and HTML.
I got to thinking there must be some way I could find all the repeated code before I start reviewing it and so I wrote a script which gives output like:
Line 309 repeated on line(s) 610, 752, 841, 891, 1100, 1484, 1502, 1546, 1567
Line 310 repeated on line(s) 611, 753, 842, 892, 1101, 1485, 1503, 1547, 1568
Line 311 repeated on line(s) 612, 754, 843, 893, 1102, 1486, 1548, 1569
Line 312 repeated on line(s) 613, 755, 844, 894, 1103, 1487, 1504, 1549, 1570
Line 313 repeated on line(s) 614, 759, 848, 898, 1104, 1488, 1508, 1553, 1574
Line 314 repeated on line(s) 615, 760, 849, 899, 1105, 1489, 1509, 1554, 1575
Line 315 repeated on line(s) 616, 1510, 1555
Line 316 repeated on line(s) 617, 762, 851, 901, 1491, 1511, 1556, 1577
Line 317 repeated on line(s) 618, 763, 852, 902, 1108, 1492, 1512, 1557, 1578
Line 318 repeated on line(s) 619, 764, 853, 903, 1109, 1493, 1513, 1558, 1579
The above is an example of a block of 10 lines of code repeated 8 times, with a minor variation.
So with a report like this, I can take a printout of code (w/line numbers), a highlighter pen and block out the large repeated code blocks.
Naturally, my script detects things like "$i = 0;" or "=cut", but such code is usually not found in large blocks of lines, so it doesn't stand out.
Now I'm thinking how to take this forward - perhaps process the original file and insert footnotes or glyphs like [snip] in place of the repeated code.
an endless discussion thread about what features to include.
-- Jarkko Hietaniemi
Subantive content of Perl programs (Score:2)
A random idea: measuring the degree of redundancy in a Perl source file. I think it'd have to be something other that just simple text redundancy, since you dan't want short symbol names (or hash key) being favored over long ones.
Re:Subantive content of Perl programs (Score:1)
Of course the statistic needs to be weighted by the number of lines and length of the lines.
What is debatable is whether one should toss single-line idioms such as
@out = sort {$a $b} @list;
which might occur in several places, into a single subroutine.
Re:Subantive content of Perl programs (Score:2)
I think a feature to detect the longest sequence of repeated lines would be useful, too.
J. David works really hard, has a passion for writing good software, and knows many of the world's best Perl programmers
Visualizing it (Score:1)
Algorithm::Diff, colored output (Score:1)
I also wanted to mention mjd's Algorithm:::Diff [cpan.org] (which Text::Diff [cpan.org] uses) in case that would help you find longest common sequences.
- Barrie
Re:Algorithm::Diff, colored output (Score:1)
I suspected someone like mjd would have worked this before...but taking a look at the two modules you reference, I can't immediately see how you'd apply them to a single file.
Re:Algorithm::Diff, colored output (Score:1)
You could also use it to diff the lines that were tweaked between the original code chunk and the copy-paste-tweaked code chunk. Lots of visual diffs do that sort of char-by-char diff