use Perl

One of the nasty things about the recent OpenSSH vulnerability is that it affects non-debian systems, too.

Thankfully the script to find the bad keys was written in Perl. With a bit of back-porting, I managed to get it to work with perl 5.6.1, and thanks to the magic of Module::Install, I have made a tarball which includes the dependencies of the debian-published script and uploaded to CPAN as Dowse::BadSSH.

Unlike the published script, the updated dowkd.pl is capable of removing bad keys and checks more places on the system, such as known_hosts files and the system host key.

Portability patches more than welcomed.

Yes, I realise I probably should have based my work off the upstream sources

Also available from utsl.gen.nz. Note there were not one but two brown paper bag releases for this. You want at least version 0.04 to safely use the '-r' option.

That's like ancient history by now. But there's a new gittorrent references file for a distribution if you don't need to be up to date. A shallow clone of blead works out to be a 16MB download. A full clone of the ~37,733 commits in the repository is about a 112MB download.

And for the record,
Com-    Pump-  Release         Date            Notes
mits    king                                   (by no means
                                                comprehensive,
                                                see Changes*
                                                for details)
================================================================================ ===

        Leon     maint-5.005      2007-Oct-02

        Nicholas 5.8.5-RC1     2004-Jul-06
                 5.8.5-RC2     2004-Jul-08
                 5.8.5         2004-Jul-19
                 5.8.6-RC1     2004-Nov-11
                 5.8.6         2004-Nov-27
                 5.8.7-RC1     2005-May-18
23,900           5.8.7         2005-May-30
                 5.8.8-RC1     2006-Jan-20
24,294           5.8.8         2006-Jan-31
25,218           maint-5.8        2008-Mar-03

23,217  Rafael   5.9.2         2005-Apr-01
25,521           5.9.3         2006-Jan-28
                 5.9.4         2006-Aug-15
29,291           5.9.5         2007-Jul-07
                 5.10.0-RC1    2007-Nov-17
                 5.10.0-RC2    2007-Nov-25
30,191           5.10.0        2007-Dec-18
30,256           maint-5.10        2008-Mar-03
30,813           blead        2008-Mar-03



Freshly up - a fresh batch of changes Com-    Pump-  Release         Date            Notes
mits    king                                   (by no means
                                                comprehensive,
                                                see Changes*
                                                for details)
================================================================================ ===
        Leon     5.005_04-RC1  2004-Feb-05
                 5.005_04-RC2  2004-Feb-18
3,943            5.005_04      2004-Feb-23
[...]
        Rafael   5.6.2-RC1     2003-Nov-08
8,089            5.6.2         2003-Nov-15     Fix new build issues
        Jarkko   5.7.0         2000-Sep-02     The 5.7 track: Development.
                 5.7.1         2001-Apr-09
                 5.7.2         2001-Jul-13     Virtual release candidate 0.
15,424           5.7.3         2002-Mar-05
                 5.8.0-RC1     2002-Jun-01
                 5.8.0-RC2     2002-Jun-21
                 5.8.0-RC3     2002-Jul-13
18,560           5.8.0         2002-Jul-18
                 5.8.1-RC1     2003-Jul-10
                 5.8.1-RC2     2003-Jul-11
                 5.8.1-RC3     2003-Jul-30
                 5.8.1-RC4     2003-Aug-01
                 5.8.1-RC5     2003-Sep-22
19,911           5.8.1         2003-Sep-25
        Nicholas 5.8.2-RC1     2003-Oct-27
                 5.8.2-RC2     2003-Nov-03
                 5.8.2         2003-Nov-05
                 5.8.3-RC1     2004-Jan-07
                 5.8.3         2004-Jan-14
                 5.8.4-RC1     2004-Apr-05
                 5.8.4-RC2     2004-Apr-15
20,328           5.8.4         2004-Apr-21

21,401  Hugo    ,5.9.0         2003-Oct-27
22,007  Rafael   5.9.1         2004-Mar-16


Change 9999 already! That's quite a few releases... new GitTorrent references tag is at this link.

For funsies, here's another section of the records:

Com-    Pump-  Release         Date            Notes
mits    king                                   (by no means
                                                comprehensive,
                                                see Changes*
                                                for details)
============================================================================
3460    Sarathy  5.005         1998-Jul-22     Oneperl.

        Sarathy  5.005_01      1998-Jul-27     The 5.005 maintenance track.
                 5.005_02-T1   1998-Aug-02
                 5.005_02-T2   1998-Aug-05
                 5.005_02      1998-Aug-08
        Graham   5.005_03-MT1  1998-Nov-30
                 5.005_03-MT2  1999-Jan-04
                 5.005_03-MT3  1999-Jan-17
                 5.005_03-MT4  1999-Jan-26
                 5.005_03-MT5  1999-Jan-28
                 5.005_03-MT6  1999-Mar-05
3907             5.005_03      1999-Mar-28

        Sarathy  5.005_50      1998-Jul-26     The 5.6 development track.
                 5.005_51      1998-Aug-10
                 5.005_52      1998-Sep-25
                 5.005_53      1998-Oct-31
                 5.005_54      1998-Nov-30
                 5.005_55      1999-Feb-16
                 5.005_56      1999-Mar-01
                 5.005_57      1999-May-25
                 5.005_58      1999-Jul-27
                 5.005_59      1999-Aug-02
                 5.005_60      1999-Aug-02
                 5.005_61      1999-Aug-20
                 5.005_62      1999-Oct-15
                 5.005_63      1999-Dec-09
                 5.5.640       2000-Feb-02
                 5.5.650       2000-Feb-08     beta1
                 5.5.660       2000-Feb-22     beta2
                 5.5.670       2000-Feb-29     beta3
                 5.6.0-RC1     2000-Mar-09     Release candidate 1.
                 5.6.0-RC2     2000-Mar-14     Release candidate 2.
                 5.6.0-RC3     2000-Mar-21     Release candidate 3.
7223             5.6.0         2000-Mar-22

        Sarathy  5.6.1-TRIAL1  2000-Dec-18     The 5.6 maintenance track.
                 5.6.1-TRIAL2  2001-Jan-31
                 5.6.1-TRIAL3  2001-Mar-19
                 5.6.1-foolish 2001-Apr-01     The "fools-gold" release.
8003             5.6.1         2001-Apr-08
The "commits" column is the approximate number of discrete changes in that version of Perl, not the number of changes in the entire repository. At Change 9999, there are 12266 revisions in git.

Will the Parrot team finish Perl 6 before I finish converting the Perl 5 history to git?

Who knows ... but it looks like I will end up missing my original target release date. I just haven't had the energy recently to burn the midnight oil for it.

However, I have managed to knock off one of the reasonably difficult TODO items in my Perforce importer. I think this one was responsible for most of the discrepancies between the preview one and the rsync blead. There's a couple of problems of this scale left, which will culminate in the nastiest queries and techniques employed by the tool being removed.

I'm quite happy that I developed this along the way using the stable development style, otherwise it would have been truly shattered by now :-). At least when progress is slow, it can be steady.

Well, with a bit of luck I'll solve the other two biggies with a couple of those "magic" nights you get every now and then over the next few weeks. I'll post here once I get a decent preview export out of this version, and will continue to update as I work on the project.

The maintenance release 5.004_05 seems a reasonable place to call it a day. Several changes on the maint-5.004 branch were expanded using the same scripts that were used to extract commits from the early 5.004_NN-series patches.

Additionally, there are now new rules that aim to place "soft"-links (commit IDs embedded in the commit message) when change numbers are used in commits.

References File

Ok, so I might have built an exporter which can export 100s of commits per second;

second.maia:~/src/perl.clean$ time git-p4raw export-commits -n 400
git-p4raw: gathering export plan
git-p4raw: exporting commits between 525 and 924
100% [===============================================================]D 0h00m00sgit-p4raw: Now checkpointing.
warning: Not updating refs/heads/p4/perl (new tip ef9eefa4f9098fbcc59c2aba28c73f4e071fe88f does not contain 5ec5d678e03468a8d1d3cb0b3863aacb4ba75233)
warning: Not updating refs/heads/p4/ansiperl (new tip d869d2901d6cf3197af2842a1cd2aca6fcea7024 does not contain c864495f8aa5c60a65eec5f015666b8f3ea5ae96)
warning: Not updating refs/heads/p4/win32/perl (new tip 15d3ab12d6f67a93f4836715f854ade61f9f6030 does not contain 1e696528467d711508235105c2294a1911fc12ad)
git-p4raw: waited 1s for p4raw.8790.marks to be created

real    0m2.528s
user    0m0.636s
sys     0m0.056s
maia:~/src/perl.clean$

However there are some changes which are just begging to be made ... I've added a customization to the part which returns a commit message from a perforce change. This is taking much of the older in-band data, back into its correct header places.

One of the common practices through the history is to refer to other changes by numbers. So, I made the add extra links to the commits. As the model of git-fast-import is to just fire the objects at it and let it calculate the SHA1, this required a hack to git-fast-import. Hack it may well be, but it looks like it works.

Update: GitTorrent references file

git clone git://utsl.gen.nz/perl

(also available via HTTP at http://git.utsl.gen.nz/perl)

Note that the preparation of this history has involved many long toiling hours of correlation of changelog information, searching for and attempting to apply literally thousands of patches from p5p archives and comparing them to the binary releases, etc. I believe it to be a significant achievement into the restoration of the early revisions of Perl. It certainly wouldn't have been possible without the support of my employer, Catalyst IT, and of course the wonderful revision toolkit that is Git.

Here is the release announcement references file.