Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I didn't reply before because I didn't have to contribute what you're looking for, but I was pointed at this again today and you've only got one reply... so... here goes.

    The Exegesis and so on are pretty good. It's hard to say how complete they are until people have been using an implementation of it for a while. That's the nature of the beast. You don't know how much further until you get there, and in this case, that's a function of observing people use what you've made for a while.

    The (I forgot what it's called... I blogged about it a while ago in a kind of state of things in Perl 5 land on my use.perl.org blog... the spec gimmie5 is built on top of) functional spec for Perl 6 is likewise good but not complete.

    Then there's Rakudo.

    This draws on an interesting topic... people looked at Perl 5 and said, "We can do better". That's the wrong kind of hubris. It's naive to think that you can make a 5.12 or 5.14 without a lot of prototyping ideas in modules, testing implementations in core, etc. Bad things happen when you try to pull a large, complex system out of nothing, whole cloth. Things like Second System Syndrome. If you haven't, stop and read The Mythical Man Month. Nothing I could possibly hope to offer as a reply here could begin to compete with what's already been written in that book.

    I like Pugs because it tried to protype these ideas and let people play with it *first* then build a serious implementation through a bootstrapping process (translating Haskell to the Perl 6 that Pugs runs) second. A lot of people liked that idea. No one helped Audrey. Or few people did. I wanted to. Like everyone, I had excuses... not lasting more than four months at any job, trying to get consulting going to compensate, dealing with all of the fighting of an unhappy relationship. You get the idea.

    Then there's Parrot. In general, I think it's a good idea not to complain about the people doing the work... lead, follow, or get out of the way... if you don't have anything nice to say... but again... there's a big gap of silence here. Maybe that silence speaks more than anything I could say. But I'm not known for my good judgement. See previous paragraph.

    Parrot has a combination explosion of methods for accessing data (through array index, hash value, scalar, etc), what's being accessed, and which operator is being performed. Each PMC has a huge list of tiny accessors to accommodate these permutations. The idea was to make this fast, but this is a textbook case of a guy fresh out of college, full of self confidence, but badly lacking in real world large project experience. It's a good idea on the surface that's really a bad -- it's premature optimization. It made early prototypes go very quickly thanks to very short code paths. But later versions are finding these benefits completely offset by code bloat. Having too far unrolled the loops, the cache is being busted. Inner stuff, when Parrot is finished, won't fit in cache. Perl 5 has a runops and pp_hot.c that fits nicely in a moderately sized cache.

    But that's not it... almost every aspect of Parrot follows this guiding principle of premature optimization. They decided on very tiny operations, sort of like the JVM (which should have been a clue that something was amiss -- how long and how much technology did it get the JVM running even passably?). Then, for *years*, the Parrot compiler guys went back and forth on call convensions -- which registers should be used for what when when a call is made from one function to another. They're in an early stage of development, completely lacking insight of a finished product, and they're spending *years* optimizing based on speculation and intuition -- something that most VMs never even think about. Even the JVM isn't that low level. By the time they're done, the LLVM is out and does a good job of this and the problem is obsolete.

    Let me delve into a story here. MUD -- multi user dungeon -- was extremely formative for me. I spent ten years there. I watched MUD after MUD go down because they had too many players, the machine couldn't handle it, they needed to rewrite the *entire* codebase from 2.4.5 compat to 3.x compat to make it faster, and it'll let them make the game *better* at the same time, since they're already rewritting it, right? One by one I watched dozens -- almost every single one of the best MUDs go down for rewrites -- and never come back up. A few came back up... but one or two out of dozens. These were huge active places with dozens of programmers each. The MUDs that are still up and still popular either went through several false starts in rewrites and wound up something completely different than what they were, or more likely, never went down for the rewrite in the first place, and are *still* 2.4.5 compat. But here's the kicker -- the games that *did* come backup, did so *far*, *far* after CPUs became cheaper and faster and the benefit of rewriting was completely gone. 12mhz DEC 3100s were replaced by 180mhz SuperSparcs in that time period. Long after the fact, most of the new ideas in the 3.x driver turned out to be pretty lame, as far as running a game goes -- shadows were like AOP, intercepting function calls. The security model might have made anal retentive wizards happy but did nothing for the experience of the players. Etc, etc. Everything that the 3.x authors thought was important, including more performance, turned out not to be. It was just sirens that lured programmers -- programmers that have the same programmer mindset as programmers do today -- the "I can fix this! I can anticipate what will be needed! I can take a huge system and do better! The neat stuff I'm thinking of will be neat when I code it!" false hubris.

    Programming is *slow*. Take on a large project, the world changes while you work. Wait a while, and most of the things you wanted just kind of happen.

    I worked a Qwest (formerly USWest) for a while. Many chores involved tn3720's into regional IBM 390 mainframes. Other tasks involved an old VB2 or 3 GUI app. Others were Web based on a Netscape Enterprise Server written in an app in C++ against an Oracle backend with JavaScript that required 4.5 -- nothing else would work. Still other tasks were done in a Java client-side app that talked to still another database. Every few years, the programmers working on the grand unified GUI layer over the mainframe would find themselves using horribly outdated technology and far from finished, and a decision would be made to start over with the latest technology. This happened for 20 years without any effort actually completing a GUI that did everything the mainframe screens did. And some of the more obscure screens were still modeled after the punch cards they replaced. I'm watching people do this with Perl 6 implementations right now, but in parallel rather than only in series.

    Now let's talk about Rakudo. It's impressive that it's as far along as it is. But that's the dumbest way to bootstrap I've ever heard of. If you have a language, such as BCPL, you change the parser to accept your new language, such as B. Then you slowly start changing the source code of the BCPL compiler to include B as its deemed safe. If you can build a compiler with the compiler, then build a compiler with that compiler again, and have the two binaries be identical, chances are very good that the code generation and parsing are working. Then before long, you have B, rather than BCPL. It was in exactly this way that C was created from B as well as B from BCPL. Working from scratch is what proud college graduates do, and it's guaranteed to turn into a ten year research program. Every now and then one succeeds at something, but no one talks about the numerous failures, as they do when you're actually working out in the industry.

    gimmie5 effectively fits this BCPL->B->C model, but the waters are muddled a bit by Perl 5 not being self hosting. Still, if Perl 5 is able to parse Perl 6, more and more of the Perl 5 that parses the Perl 6 can be moved into C. There are various other options -- options that are incremental and save optimization for last, if ever.

    Also on the topic of optimization... C is still relevant today, but it's becoming less so for desktop GUI apps and it's almost gone for Web stuff. C is concerned with very low level instruction level optimization. Perl 5 is concerned with handling things such as scope, temps, lists of complex types, etc quickly. As time passes, one kind of optimization becomes assumed while others become interesting battle grounds. Parrot is still fighting C's battles.

    The whole idea of simulating a register bank in memory in Parrot is... well, let's say Parrot were a pure stack machine. Then the call convention debates would be almost non-existent. And someone could later come along and write a translator from the stack-based stuff to register file stuff, probably at the same time they were doing a JIT that translated to x86 or whatever. In fact, that would be the normal thing to do in that case -- use the x86's own register file. But perhaps they decided not to go down to x86 but still do a register file. Then the whole process happens later, nothing is built on top of it or assuming it, it's implementation can change without breaking anything or upsetting other designs. It's win-win. But by adopting this optimization premature, everything is delayed, code is fragile, everything has to change, etc. This is an implementation detail that should never have been exposed outside of some optimizer's own playground. And it may not even prove to be much of an optimization. Terribly hubris supposed it to be to the point where it was willing to make this gamble to build everything on top of it.

    The second system syndrome exists partly in the Perl 6 spec -- everyone saw that coming. Everyone fought it while at the same time proposing stuff, myself included. Larry originally had a very small window of submissions and did a good job of keeping people posted as he sorted and scored the entries, and commented on them. But the real Second System Syndrome stuff is kicking in in the implementations.

    I think the first lesson you learn out of college is that it is simply not a good idea to try to build large systems. The value plummets in relation to size. Value lies in the "quick hack". And if you can do a quick hack that grows, awesome. That's the best ever. But setting out to build a cathedral is just dumb.

    In short, even if Parrot flies, it's going to be a bigger pile of crap than Perl 5, but the propensity for premature optimization will continue to sap time and effort from the things that matter -- as in the MUD analogy, from actual gameplay.

    As for myself, I have some projects going, consulting still isn't where I'd like it to be so I'm living on ramen, but I'm toying with various quick hacks that might tie together different efforts rather than create new redundant efforts that further fragment interest, resources, and the attention of others who would like to jump in to this fray but get lost in the myriad of projects.

    -scott

    • [People] looked at Perl 5 and said, "We can do better". That's the wrong kind of hubris.

      Anyone who's patched Perl 5 and hasn't thought "We can do better" has insufficient imagination.

      No one helped Audrey. Or few people did.

      I tried. Pugs was unusable. This approach did not work. Note that for any perceived or real flaws in the Rakudo approach, it makes visible progress toward its goals every week, and visible, dramatic progress in every stable monthly release.

      Perl 5 has a runops and pp_hot.c that fit

    • Now let's talk about Rakudo. [...] If you have a language, such as BCPL, you change the parser to accept your new language, such as B. Then you slowly start changing the source code of the BCPL compiler to include B as its deemed safe.

      If this is the approach you advocate, then your comment is more properly a criticism of Perl 6, not of Rakudo. I'll note that when Larry wrote his parser for Perl 6, he didn't start by modifying an existing one either. I suspect that's because there weren't any parsers in

      • I'm kind of sad to see you reply. I like to think that my kind of prattling on is completely off the radar of the people actually doing the work.

        It's true that Perl 1 kind of came out of no-where. I don't know the early-early history here. Did Larry peek at awk's grammar and copy bits from it? I don't know.

        I also know the Perl 5 parser is a bit horrifying and the muckiness of all of that is a lot of the catalyst for Perl 6, but, in theory, one approach to writing a Perl 6 grammar would be to hack up Per

        • I think you overestimate the difficulty of writing a parser for a language and greatly underestimate the difficulty of equipping a two-decades-old runtime with evaluation semantics like laziness that it was never designed for. The BCPLBC example is somewhat of a red herring in this because all of the languages are in the “portable assembly” class largely defined by ALGOL, without significant differences in evaluation semantics or memory model.

          I made a very small proposal for 5.12 and it took a [perl.org]

          • I'm confused. I said that the ideal case of morphing one language into another did not apply to Perl 5 and acknowledged that one of the primary reasons for a rewrite was the Perl 5 core.

            Can you double check that I said what you thought I did?

            -scott