Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Alias (5735)

Alias
  (email not shown publicly)
http://ali.as/

Journal of Alias (5735)

Monday February 08, 2010
09:13 PM

Why Ruby is prettier and Padre changes the Perl community

[ #40170 ]

Why is PHP so much easier for newbies?

Why does Java have the best IDE tools?

Why is Ruby prettier than Perl?

Why does Perl have the best package repository?

As I've played through Mass Effect 2 over the last few weeks, I see some interesting parallels.

In the Mass Effect universe, human technology is bootstrapped by the discovery of an ancient abandoned alien observation outpost on Mars, and the further discovery that the dwarf planet Charon is really an abandoned but active interstellar jump gate covered in ice.

Other similar species have done the same, resulting in a galactic community of around a dozen civilisations all based around the same basic technological underpinnings.

Despite these civilisations believing a recently (50,000 years) extinct civilisation built the gates, it turns out the technology is perhaps millions of years old.

Every 50,000 years, the synthetic AI race that built them returns from hiding in intergalactic space to wipe out all of the existing advanced species based on "their" technology, and reset the galaxy for the next set of civilisations to rise.

In a conversation between the game's protagonist and one of these old AIs, we are lambasted by the AI for taking the shortcut on their technology. The jump gates and other technology is left in place intentionally, so that each new generation of civilisations take a controlled and predictable development path, making it easier to destroy them.

The AI posits that it is the overcoming of adversity on your own that drives true technological advancement, and that easy routes make you (technologically) weak.

I think you can see something similar in the development of the different programming languages.

Java is long and wordy, taking a long time to type. The need to work around this limitation resulted in the proliferation of powerful IDEs, resulting in the annual 20 million line of code Eclipse release train.

PHP as a web language would have been stillborn if it didn't deal competently and quickly with the need to easily deploy code, the result of which is that you can effortlessly just change .html to .php, add a hello world tag, and upload via FTP as normal (something Perl still can't do well).

Python's need to gain mindshare against an entrenched Perl led to a huge focus on being easy to learn, to a simplification of the language, and to hugely popular things such as the PyGame library and game competitions.

Faced with the lack of truly great package repository, and with a web-heavy community, Ruby became the "prettiest" language. Creating an elegant website is both expected and required if you are going to gain mindshare for an idea.

And Perl's messy syntax and difficulties in the area of maintaining large codebases, combined with a pragmatic sysadmin-heavy community, resulted in an unmatched packaging system that allowed code to be maintained in small pieces, with enormous volumes of support infrastructure around it.

The ease of publishing and trend to smaller package that the CPAN allowed conversely means that the Perl community has never really had the need for pretty and elaborate websites, and the smaller package size means that we lack the giant headline libraries that make the payoff from website work better.

Our bias towards a pragmatic tech-savvy sysadmin userbase means we haven't really provided anything like the focus on learnability that has driven Python's gradual dominance in the mindshare of the young. It takes a certain rigour in your prioritisation to intentionally remove and dumb down existing powerful features so that the language is easier to learn.

Even for Strawberry, which focuses on the userbase with the lowest traditional knowledge, we intentionally have the smallest and most maintainable website possible and we don't even have the kind of introductory screencasts that we really really need (which should be easy but which I never seem to find the time to do).

If you throw a bunch of Perl coders against some PHP coders in a website competition, it is not unexpected that when both sides play to their strengths you will see something like http://geo2gov.com.au/html?location=e.g.+1+Oxford+Street from the Perl coders and something like http://www.hackdays.com/knowwhereyoulive/postcodes/view/2000 from the PHP coders.

The former required a massive amount of data extraction, transformation, aggregation, a gigabyte-sized PostGIS database, and deployment via a Linux virtual appliance to Amazon EC2 to allow for strategic load-shedding.

The latter required the ability to turn data into presentable and understandable information for real humans, and to make it pretty enough that they WANT to look at it.

Driving true technological progress, then, may often be about identifying weaknesses that are hard to solve but aren't completely impossible (and don't have any crippling long-term conceptual flaws at an economic or project-management level).

The three best projects I have driven - PPI, Strawberry, and (in part) Padre - all share this property. All three of these represent hard but not impossible problems, and require an awareness about which issues are intractable and which issues merely exist because there's been no need to solve them any better.

Padre in particular has suffered greatly from issues with Wx quality and threading. But given the low takeup of both threading and Wx it was reasonable to move forward on the basis that these would be fixed once there was something depending on them, and driving a need to fix them.

All of our early problems are gone now, and there is continued pressure to find ways to improve our use of (and the efficiency of) Perl's native ithreads.

Similarly, the creation of Strawberry required a lengthy year-long process of fixing Win32 bugs in all kinds of toolchain and low level modules, because we'd never had a proper working developer feedback loop before.

Similarly, Perl's current push for marketing and blogging and websites is directly resulting from Python's success in mindshare capture.

So my question for you to ponder this week is the following:

What can you see that Perl as a whole struggles to do well, for which a good solution is not impossible, and is only being held back by smaller problems which would go away if there was a working candidate solution put in place that needed those small problem solved.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • The fact that you have to have adept-level Perl 5 experience to understand how to use Perl 5 (the language) in the fashion most useful for novices is a huge problem.

    • So what do you feel is the solution, and what are the small problems that would go away once we had a working candidate solution?

        • The only major problem with this is that most of the novices - at least those I encounter - need to use old versions of perl.

          So the usefulness of this would only be seen later down the road and only by some of the new users.

          But one needs to step on the road to get there.

          --
  • I don't try Padre because (1) it's a pain in the ass to install on Snow Leopard via CPAN or .dmg; and (2) I have no reason to use a Perl-specific text editor when I pay the bills with things that are at least partly not Perl. The Google juice from a bunch of fanboys posting "X, the Y IDE" will not encourage me to use X.

    • What does your rant have to do with your subject line? Is the default OSX Perl compiled without threads? How does your rant address Adams post?

      PS: I'm a vim user and will always be. I write C++ more or less all day. Yet when I have to refactor an icky bit of Perl, I switch editors because I'll take all the help I can get. Works fine for me. Doesn't have to work for you. Tough luck, get on.

        • You really don't have to explain why you are not trying Padre. The majority of the Perl community have not tried it (yet) and that's totally ok.

          IMHO "changing the Perl community" did not mean every Perl hacker now will have to use Padre.

          I never thought Padre would catch any measurable part of the vim/emacs users to change their editor. Maybe once in a while, as tsee mentioned, they will use it and maybe they will recommend it to novices.

          So the major change can come from:

          1. New people coming in to the P
          --
      • I didn't say "is changing", merely "changes".

        The concept/implementation of Padre taken over long time scales will change (and has started to, but only in tiny corners here and there) the nature of the Perl community.

        I don't claim that this change has happened yet in any major tangible way.

        But I see it coming...

  • I tend to think of this along the lines of "making hard things easy and impossible things merely hard". With that in mind, the frontier has shifted since Perl was developing rapidly.

    My gut feeling is that the "wow" factor has become harder for Perl to achieve because what wows people isn't insanely great string, dataset and administrative task manipulation. People want to see results quickly. That's not just about the learning curve -- it's the end-to-end process.

    Part of the advantage of Java in an educa

    • "Improve parallel processing and leveraging multi-core systems"

      I agree about this. It's important going forward.

      Also: Point me at a dynamic language that does this better. No. Not Python. They don't do concurrency. They do the same as Coro does and call it threads.

      As for extending the core: We both know that's against the opinion of the powers to be. And the powers have the good reasons for their agenda. The correct way to do this is an extended core project, either via EPO or not. Have releases of that wit

      • The correct way to do this is an extended core project....

        Exactly; that's the best way to do it well in any sense. If Strawberry has demonstrated anything, designing for the the utility of end users is inestimably important.

      • As for extending the core: We both know that's against the opinion of the powers to be.

        The "powers to be" aren't powers. We're just the people who do the work, rather than the talking.

        If people feel that there should be a bigger core distribution, please branch http://perl5.git.perl.org/perl.git [perl.org], add what you think needs adding, advertise it, get it established, demonstrate that you can support it on an ongoing basis, and if it's proven better than the current approach for the core distribution, then likely

          • I just phrased it wrong.

            Yes, just that. Sorry for annoying you. I know that you contribute massively to keeping the dual life modules in shape. I'm not as aware of the other things that you do.

            But I don't like the phrase. Particularly as it can be taken out of context, and likely will be some people who read it. It implies that the people referenced have the power to change that policy. Whereas the policy is totally defensive - we're already stretched beyond what a volunteer group can manage. If we tried to

            • This is only going to work for trivial cases.

              Strawberry currently has 50 odd extra modules, mostly crypt, build/cpan upgrades, and database clients.

              It's already bloody hard to maintain the list of dependencies by hand.

              You really need to be able to bootstrap from core -> ( list of classes ) to maintain sanity.

              Just installing Padre takes 100 packages and the dependency list changes (both up and down) on a weekly basis.

    • You're idea about deployment as easy as PHP is spot on. The PHP concept of a web-based installer is really intuitive to work with for a novice.

      I've been pondering this idea here:

      http://github.com/robinsmidsrod/unnamed-perl-cms-project/blob/master/README#L82 [github.com]

      If the novice could just download and install a web-based CPAN installer/updater to put on their shared hosting account and then just view that UI in their web-browser a lot would be done. To avoid the potential problem of a CPAN compiler fail (or lack of

    • On parallel processing:

      There's got to be a better way to do this.

      No dynamic language has found it yet. Even 2 full time Google engineers have failed to find it for Python [google.com]

      CPython has a Global Interpreter Lock, single threads the interpreter, and hence single threads compute intensive task. [C]Ruby has "green threads" (i.e. it does the thread scheduling). There's no prior art. There's no proof that the problem even has a solution.

        • Sorry, I wasn't clear enough. My fault. Prior art for a dynamic language implemented in C. Clojure is a dynamic programming language that targets the Java Virtual Machine. (To the best of my knowledge) Jython does threads just fine - it's CPython that can't, and even the folks at Google working on Unladen Swallow can't see how to take that Jython know-how and port it to the C implementation.

            • Good point.

              Sadly, once again the punchline from the "Irishman giving directions" joke bites:

              If I were you sir, I wouldn't start from here.

      • If you are going to move forwards with PerlQT, just make sure you solve (early) the problem of building and installing it on the three main desktop targets effortlessly (Win32, Mac, *NIX Gnome/KDE) because if you can't install a desktop program on the three main desktop environments, you lose a lot of traction immediately.

  • I recently thought of installing Padre on the most recent CentOS (which, in case you didn't know, is a community supported enterprise version of Red Hat Linux). The huge number of modules that need an upgrade is discouraging: it wants to upgrade 24 modules (and install 81 more), even after I already installed Wx.

    The main problem I have with that is that I don't want to change anything in the system Perl, as this is contrary to the spirit of CentOS: don't upgrade anything that is stable, as the upgrade might

    • Often the new modules are needed.

      There's a lot of work "on Padre" that actually goes into various dependencies, resulting in new releases and thus new dependencies.

      The ORLite family alone has gone through 10-20 releases to add embedded database features that Padre needs.

      0.56 was a bit more aggressive than most, since it bumps Module::Build and ExtUtils::MakeMaker to current, had some ORLite upgrades, and a few other minor things.

      Some releases are worse than others, but it's our preference to push developmen

    • We are aware of the fact that it is difficult and time consuming to install Padre from source hence we are encouraging the downstream distributors to include Padre and its major plugins.

      In order to allow you to use the latest (or almost latest) Padre on Linux we also have an experimental version of it that includes everything you need - even a threaded Perl.

      See the Linux section on the download Padre [perlide.org] page.

      --
      • Interesting approach — if only it worked... It turns out not to be so easy.

        I tried the "experimental" "Padre Stand Alone for Linux", but it immediately died, complaining about an incompatibility between Wx and libstdc++.so ("GLIBCXX_3.4.9 not found").

        Anyway... Do you see this approach as a solution for all big projects in Perl? Building all these binary distros seems like a lot of work to me. Plus, with several projects, you get a lot of separate file trees, with possibly a lot of overlap. That may

        • I am not surprised it did not work - I tested it only a very limited number of platforms. (Namely 1) but still I am sorry to hear that.

          As I understand in order this to work we need to build a statically linked perl and wxwidgets and apparently this was not the case.

          I think this can be one approach though I hope this could lead to a package similar to Strawberry Perl Professional but for Linux and similar OS-es which would mean other desktop application could use it as a platform.

          I still think inclusio

          --
    • The Strawberry team is going to bundle BioPerl in the default install of Strawberry Professional, which will also come with Padre pre-installed.

      So we can offer than a zero-effort bootstrap into a BioPerl environment.