Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Alias (5735)

Alias
  (email not shown publicly)
http://ali.as/

Journal of Alias (5735)

Tuesday April 17, 2007
02:38 AM

Why is the security section of CPAN such a disaster zone?

[ #33006 ]

What is it about crypto modules that makes them almost universally badly packaged?

Fair enough, crypto can be complicated, but is there some inherent property of that area of programming that makes them so much of a mess?

Many of the classic packaging anti-patterns are displayed in the various depths of the dependency chains of modules like OpenPGP or Net::SSH::Perl, both of which are more or less impossible to install.

Platform Bias

For Windows people, this is huge. The crypto modules are littered with unix assumptions and close-minded authors. In at least one case, when the Strawberry Perl compatibility team tried to contact the author to help him add support for Win32, they were told that "You people go back to your stupid Winblows. There's no way I'm adding support for it".

With the gpg command line generally not available on Windows, and the Perl OpenGPG module unable to install because of people like the above, this completely torpedoed any chance of having CPAN clients do security checks by default, or check signed modules.

Knowledge Bias

Having "options" in your Makefile.PL is almost universally bad.

Now, sometimes Makefile.PL script cannot auto-configure and need to ask for things like library paths, or your ORACLEHOME path or what have you. This is just fine, since it's a last resort.

But have a look at some of the questions for Net::SSH::Perl.

------------------------------------------------------------


As of version 1.00, Net::SSH::Perl supports both the SSH1 and
SSH2 protocols natively. The two protocols have different
module prerequisitives, so you need to decide which protocol(s)
you plan to use. If you use one or the other, only those modules
for your chosen protocol will be installed; if you choose both,
all of the supporting modules will be installed. Please choose
the protocols you'd like to use from the following list ("Both"
is the default).

        [1] SSH1
        [2] SSH2
        [3] Both SSH1 and SSH2

Which protocol(s) do you plan to use? [3]

Some of the Net::SSH::Perl ciphers depend on a Crypt:: module from
CPAN. You may already have the necessary modules installed, in which
case you don't need to bother with this step. Otherwise you'll need
to install at least one cipher to use Net::SSH::Perl. Please choose
at least one from the following list (Crypt::IDEA is the default).

        [1] IDEA
        [2] DES
        [3] DES3
        [4] Blowfish
        [5] RC4

Enter your choices, separated by spaces: [1]

Checking for optional modules

Crypt::RSA is required if you wish to use the ssh-rsa public
key algorithm (ssh-dss is used by default).

Would you like to install it? (y/n) [y]

--------------------------------------------------------

These sorts of questions assume that the person installing the module understands the same things the author does. That is, they understand crypto well enough to answer questions.

I personally argue that people should be able to install CPAN modules without knowing Perl at all.

One common response to that is that CPAN should be for Perl developers, not Perl users that don't know Perl. But what is a Perl developer? What separates a one week old newbie programmer from someone that doesn't know Perl at all? Very little.

And in particular, in module installation the recursion problem dictates that the Perl you are asking questions is probably just trying to install Plagger or Jifty or something else that is several layers abstracted from your module, and has no idea whatsoever about that specific domain.

For example, there's another question missing from that set of questions regarding something called "bubble babble" which I only know of as a video game.

Ratcheted Decisions

Worst of all, many of these options commit you to one working one particular way in the future forever. The only way to change that decision is to reinstall the module.

In particular, if you are using binary packages from some distribution, you can't do that anyway. If your binary package of Template Toolkit comes without the optional "Splash!" stuff, then too bad

One nice way to sloganeer this problem is as follows.

"For every option that you put in your Makefile.PL, RedHat will answer it wrong"

(this refers to the long history RedHat has of doing weird things with Perl and related packages).

What can you do about this?

There's a couple of fairly straight forward things that can be done to achieve what you want in any case.

Firstly, if the dependencies are themselves relatively sane (they work on all platforms and you don't mind inheriting any baggage they may have) they just make them a full dependency.

This adds some module bloat, but if those modules WORK, then it's not a problem.

Secondly, don't make it an option at all, just don't depend on the module.

If something one layer above needs it, then THEY can add the dependency on the optional module.

Thirdly, can that optional functionality be separated into a different distribution? Then do that instead.

The highly structured way of splitting this out is to make use of driver or plugin APIs.

And above all, remember this.

There are probably hundreds of thousands or millions of Perl programmers in the world. There are only 5000 on the CPAN. So merely by virtue of having uploaded someone to CPAN you are quite probably already in the top percentile or better.

So try to ALWAYS apply a low estimate for the experience of your users, and try to "Just make it fucking easy to install".

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • It seems that being a Perler does not shield you from the hated that people have for Windows. I recently asked for a module to support Windows and the author put a rant in the change log about how effed up it is. He is doing it, I guess because he wants to support someone using Perl, but he does not like the fact that he has to.
    • As long as he only rants in the changelog but doesn’t refuse to consider the request, I can fully sympathise. Proc::Fork (which I’m herding) causes Perl on Windows to segfault, and I have absolutely no idea how to fix it. I tried a bunch of things and got nowhere, and no one else seems to have any useful suggestions.

      So I can well understand it if people rant in frustration about Windows. (Or for that matter, about any other part of the environment; Unix is generally saner, but it too is softwa

      • So I can well understand it if people rant in frustration about Windows.

        Also, it's not as if there's a legion of Windows Perlers providing patches for these issues. I've asked for several and received approximately none.

        I can't support a platform I don't use, and there's little point in trying to support a platform if its users don't care enough to help.

        • It isn’t even at that point for me. I do have a Windows box to test on now, at least. (That wasn’t the case half a year ago.) Unfortunately that didn’t help – I might have the tools necessary to support the platform, but I still don’t know how to do it. I tried everything I could think of, but none of it worked. At this point there is nothing left I can do but say “Windows is unsupported; patches welcome.”

          Oh, and rant in frustration in the Changelog.

  • Alias,

    I agree that Perl modules should be easy to install, even by non Perl people.

    Today I'm slogging through an install of the SocialText open source wiki. It's in Perl, and depends on about 100 perl modules (literally, I think).

    It's taking hours to install, and needs some advanced skill to workaround the oddities coming up.

    And of course, there's a great chance that the Perl modules I end up installing are not exactly the same as the developers have tested with.

    This is a different ease-of-installation than
    • There's two seperate issues here.

      One is the dependency explosion. That is actually not so big a deal as long as all the modules install cleanly on all platforms, and don't have options or ask questions.

      Yes it does become a problem when the dependency trees get large though, as a percentage of them will ask funny questions or potentially hit problems.

      Good managers of these large projects will try to choose dependencies that are relatively light, and will also put effort into caring for their upstream, patchi
  • I always thought that the security modules were developed far enough to get the ancillary task done, and no further.

    I don't think that's necessarily a bad thing. A lot of modules start like that. It's just that the security ones stopped moving forward. :)
    • ya

      It sort of seems like someone told them "Make it Work, then Make it Maintainable, then Make if Fast" and they stopped at the first one.
  • I don't know which author you're referring to, but that "winblows" attitude is awful. If you need that added layer of protection (and it seems reasonable), what are the odds of forking the module, ignoring the original author and switching to the fork? I hate suggesting that, but I can't think of an option other than "take the module away" and I would be horrified at the latter thought.

    • I think that there are many reasons why one could dislike Windows, and for those very reasons one would avoid spending effort/time in supporting that platform. This is something that happens all the time in many other fields, and not necessarily in the "Windows" direction - just think about the support for Linux of "mainstream software" like skype.

      Ranting and denying any type of support is probably a shortminded reaction, but I think that we should respect the decision of someone to *not* support something
      • Normally this might not be a problem.

        But the crypto modules impact everything.

        The result is "If you use security modules, you lose windows support".

        CPAN cares more about platform-compatibility than it does about security. Better that it work at all than work securely.

        Because of one single developer deciding he doesn't like windows, now nobody gets to have secure CPAN by default.

        All limitations imposed by an individual impact EVERYONE in your dependency chain.
        • I agree with you completely, and this is the very reason why a fork is not only justified, but necessary. Just to avoid being hostage of a single.

          Probably, the situation could be though as if one had to implement those modules from the scratch, but having a working reference implementation already available :)
    • It's not me that needs the module. This is always the problem.

      The Module is depended on by Module::Signature, via OpenGPG, via 3 or 4 more layers of recursion.

      It's a case of a few bad eggs ruining it for everyone.
  • Oh how I've struggled to install Net::SSH::Perl on Solaris because of dependencies failing to install. I'm looking at *you* Math::Pari [cpan.org] . Out of any module I've ever installed, Net::SSH::Perl is by far the most difficult to get up and running.
    • Net::SSH::Perl is one of the better modules that eventually hits all the dependencies that cause problems.