Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

scrottie (4167)

scrottie
  scott@slowass.net
http://slowass.net/

My email address is scott@slowass.net. Spam me harder! *moan*

Journal of scrottie (4167)

Tuesday October 13, 2009
02:46 PM

Strategically ignoring things... on learning

The Perl camp, under Larry's guidance, embraces "baby talk", where someone learns just enough of the language to serve their purposes. Yet there's a large stigma against this sort of shallow knowledge among the ranks of #perl. The bulk of people who come to Perl Mongers meets or ask for help on the Internet wind up trapped between these ideals as they find themselves caught in the middle of this old debate. The "Little Bobby Tables" meme strikes squarely for one side of the debate; the success of PHP for the other.

I haven't blogged much this past month. Indeed I haven't done much. I've been busy learning git, DBIx::Class, being not fresh on jQuery and Apache, ...

My instincts were squarely against learning more Perl related technologies, instead wanting to continue to focus on ActionScript and Flash, where the grass looked greener, but once I got into it, I realized that I'd be unhappy with myself if I didn't stay current with Perl. I learned Java early on and then neglected it to the point where I could no longer claim it and trying to catch up with the past years would be painful. The same thing happened to me with JavaScript. It's the old story of being put into suspended animation and waking up to a strange but strangely familiar world you don't fit into and enticements to conceptualize or relate to it are dangerously misleading. But on the other hand, maybe Perl should just sit and spin... I've said a lot of mean things about it and meant most of them.

Deciding to learn ActionScript, I immediately ordered a pile of books. Then I checked out some from the library. Then I ordered a pile more. And I read most of them.

This is exactly what the kids are _not_ doing. And I think they know something I don't. Today my suspicion that I'm being slow was confirmed for me.

I know and have known some older programmers who worked their entire life on the same codebase. The number I've touched... eek. Many. Hurts to think about. My resume has an incredible breadth of skills. Almost all of them are rusty. I seldom have the luxury of using the same skill twice. Apple wanted to talk to me about my ARM accelerated 3D experience. All I could think was, crud, I do not want to have to work under the gun for Apple and learn my way around their massive codebases. Gadzooks!

The problem with learning how to do things is that you learn how to do them wrong. Like the famous quote about BASIC and the stigma Java shops have against hiring "reformed" Perl programmers (and Ruby shops have now developed about Perl programmers even though they used to proudly boast Perl as an influence), I would not want to have a software shop and hire COBOL programmers. There's a stigma against old programmers. I am now an old programmer. The novices on IRC are in a better position to tackle today's problems than I am. They only have to learn. I have to unlearn. "What's the command?"... "Do it for me?" is a better match for companies who want programmers to hit the ground running. In Programming is Dreaming terms, we're all in shallow sleeps. There's less learning technologies and more learning how to co-exist with them. "Sometimes a few hours of guessing can save tedious minutes of reading the documentation" is less true with codebases optimized for obviousness and accessibility and single point of modification but such codebases are less geared towards learning all of the interconnections and locations of the single points.

It was my intention to put my attitude aside and learn something for once but, as usual, what I'm learning isn't quite what I expected. In fact that attitude itself was off. A "where's X, dammit!?" attitude would have served better. Another advantage of youth -- being wrong hasn't gotten old yet. Being wrong does eventually get old. It wears on you after a while. More and more, I can understand the programmers who don't want to re-tool from COBOL.

-scott

Wednesday September 30, 2009
07:33 PM

Re: Where are all the Perl books?

http://www.kiffingish.com/2009/09/where-are-all-the-perl-books.html

First, I have to quote chromatic's reply: "Some people decided that Perl books wouldn't sell and stopped publishing Perl books. Then they congratulated themselves for not publishing Perl books when they noticed that people weren't buying the books they didn't publish."

When _Perl 6 Now_ came out a couple of years back, I watched the half.com price. It held steady for a long time. I stopped looking after a while and then when I next looked, they were selling for under a buck. I'm not sure what this means but the exercise got me to think about how easy or difficult it is to saturate a sector with a book.

Talking to APress about possibly writing a book, the editor I was in touch with was pretty clear that a book on Perl 6 would fare better than about any other Perl topic. The Perl 5 market was saturated. The people who wanted to learn stuff about Perl 5 had already done so. The Perl market isn't growing. It takes an influx of people to create demand for new books. Whether or not it's true, someone who has been programming Perl for a few years is going to decide they know everything they want to, at least in the vast majority of cases. Those who continue to buy books are rare.

Here's another thing I've been noodling... I've been digging through old Atari books and porting the examples. There's a vast amount of good in those old books and magazines that's still relevant. A good friend just commented earlier today that he didn't know how to do the math for the 3D rotation that's the basis of 3D style graphics.

We've all ready how many books and none of them that have covered that? Clearly the cultural amnesia affect is in force.

In trying to explain Perl to people, I've been claiming that it's impossible to understand Perl without understanding sed and awk. Perl is a bit like Prolog in that it incorporates ideas beyond the standard C-like-with-objects fare. The idioms are powerful but if you don't understand them, they're confusing and intimidating. Lack of knowledge of sed and awk is another bit of cultural amnesia. Hell, in my day, we wrote sendmail.cf "language" to get stuff done. Used to be that mail sent to @weehours.net went to that player's MUD-mail box in the village post office south of the pub.

When Perl goes out of fashion, it's going to be a tragedy just because of the amazing books written for it are going to lose an audience. Like the graphics programming for Atari books of the 80's, _Perl for System Administration_ and all of these are going to collect dust waiting to sell for 78 cents on half.com merely because one aspect of them went out of fashion.

This also reminds me of how the Atari and then later Amiga communities refused to die. You really couldn't tell either of those communities that they were dead. I remember having subscribed to a rag of an Atari 8 bit (talking about the 400/800/XL/XE home computers) and within a couple of years, the magazine folding. What's the deal with _The Perl Review_? I keep subscribing, getting a pile of back issues and one or two new ones, then no renewal notice. Are they even still around? The Atari guys did a better job of acting alive. Some very nice books for the Atari came out at the very end of the things mass popularity, all done independently after the major publishing houses moved on to writing boring texts about Lotus 1-2-3 and Harvard Graphics.

A lot of people care about Perl -- but how many people who don't already have the vested interest in it of having specialized in it for many years? How many people who have multiple, marketable skills care about Perl?

Towards the end of the reign of the Atari, more and more programmers were writing tools and whatnot for programmers, not for users. Epic adventure games, RPGs, and shooters had already been done and people weren't making more. When they did neato new things that were happening on other platforms such as flight simulators, the efforts were half-assed at best. But terminal programs, low level programming language compilers, advanced disk operating systems, C and other language compilers, and all of that were thriving. It was incestuous. Programmers were programming for programmers and no one else. It was a great time to be a programmer for the Atari and it's still fun to go play with all of this stuff -- what was accomplished in the end days truly was wonderful. Just don't be surprised that no books were written about it or will be until someone like myself gets the wild hair to do a retrospective documenting it far after the fact.

As for the lack of Perl books, tech publishers don't just follow the trend; they try to stay ahead of it. I don't know the industry nearly as well as chromatic does, but my impression is that it takes a year to get a book to market. Things can cool down a lot in a year. If something is already starting to cool, they could cool a lot more. Ditto for things that are warming up -- and that's what the publishers want. Perl 6 might have been warming up, from APress' point of view.

I always feel like I should follow these pieces with a bit of hope so I don't sound like such a downer. I think it's too late for an image makeover for Perl. It would take too fundamental of a shift of mindset for too many programmers most of whom don't take their profession seriously. IronMan generated a lot of prose, but most of it sucks. In fact, it could be said that it brought down the quality of writing about Perl. At least people are getting practice blogging. The time spent writing will serve them well for years to come.

Perl 6 failed the "worse is better" ideal. I know this concept is the subject of some debate but it's true at least in the extreme case: if you design the perfect OS, you'll never be able to implement it, or if you do, it'll be so fantastically buggy that it'll never work nearly as well as design. It won't be grounded in the practicalities. Often the better design has an element of cynicism in it in that it punts on certain things in the name of shooting for a level that's attainable. Perl 6's virtual machine and the language itself both shot too high in one important regard: people were intimidated by them and didn't feel as though they could help out. PHP succeeded fantastically in the "worse is better" department. Anyone with a little C could easily feel like they could write code no worse than what's already in the VM. The high flying goal makes sense in other important ways. I don't know if it's a net win or loss, but the lack of buzz from community involvement was and is a serious blow.

In the worse is better department, when Linux first came out, it crashed like mad; it was terribly incomplete; what was there was mismatched. The thing was not only unusable for serious work but also barely interesting as a toy. Within months, 386BSD came out -- a stable, complete, polished, production worthy system. BSD did not win out over Linux. People felt like they couldn't contribute to BSD -- the code standards were too high; the people who did that port of BSD were too aloof; patches languished; the promises about the direction it was going to be taken were intimidating. Linux, on theother hand, felt like "the people's operating system". You could write a better version of something in it and get that replaced with your version. Or you could port something to it that it needed and be harold as a heor. Within a year, Linux had reached a point where even though it lacked the coherency and polish, it was competitive, and at that point, BSD was done for.

Effort was made to include everyone in the Perl 6 process, I know. Perhaps if github had existed back then and the effort to write Perl 6 was done in Perl 5, just to make the whole effort accessible, things would be different.

Okay, now I'll say encouraging stuff. Both Atari and Amiga had very long "deaths" indeed. AmigaOS 4.0 came out not long ago and we're just waiting for the hardware to run it as we have been for years. They decided to create a PowerPC based architecture and port to that. That was a fantastic idea except that it's really hard to get someone to build boards for a whole new architecture and not go out of business and not want money for their design after they do. The custom chipset for the Amiga has since been reverse engineered and an Amiga 1000 clone was done using an FPGA and released as open source. Long after the 130XE ceased production, Atari brought the XE back with the XEGS. Software emulators for both were created and now people play many of their games in web applets. Time destroys but time also heals. Perl has a future... it just might not be the grandiose future it wants. That future might not include very many books.

Oh yeah. I wanted to talk about how the publishing industry did or didn't kill Atari and Amiga.

-scott

Thursday September 10, 2009
04:31 PM

Language choice motivated by greed

With this recent publicize Perl movement, I've been thinking about how to tell people that Perl is great. Specifically, how to sing Perl's praises in response to criticism. I've explained how meta Perl is, using B and B related hacks as examples. I've explained how it's not just another C like language, but incorporates radically different ideas taken from sed and awk that combine in powerful ways but scare people who are only familiar with procedural or OO languages. No one cares. It's futile.

I've also participated in several "what language should I learn?" discussions now. Generally, I praise Ruby, Perl, Python, and a dozen other language, depending on the style and temperament of the learnee, but diss PHP. No one cares that PHP mangles text in attempt to escape it and that PHP programs dominate the bugtraq list disproportionally.

Rationalism doesn't enter the picture. Neither does deliberation about personal style and a community that matches their own. Instead, if they see pretty sites done using PHP, they'll assume that somehow, magically, sites they do using PHP will be pretty. And they're probably right. Likewise, they'll learn Ruby if they want a turn-key MVC to start plugging actions into (I don't know wtf actions have to do with MVC, but whatever). They'll pick up Java is they want to do something "Enterprise" and ".NET" if they want to do Microsoft Windows UI stuff or Web stuff that integrates with MS Win. Maybe they'll learn Perl if they want to do some sysadmin-ish stuff or text processing. Even though there's little awareness of it, the whole thing seems to be as simple as "X language is good at Y".

And this is a realistic, practical approach if you consider that they merely want to tread the path most trod, not blaze a new trail. They want to do exactly what countless other people have done before so that people can tell them how to do it, and so that the snags are out. In fact, if mod_php returned a pretty web page without having one other PHP related file on the system, they'd be perfectly happy, as long as they could come in and tweak it easily. New programmers don't trust themselves and they certainly don't trust you. They trust those few things they can see with their own eyes -- pretty sites with URLs with ".php4" on the end. Literally, they're asking themselves, "how much can I get for how little effort/interesting/caring/work?". Telling them to care is counter-productive. So is telling them that something is interesting -- they don't want to be interested. Telling them about multiple steps ("first you must learn Perl, then...") is counter-productive.

Wal*mart never hassles you about updating your BIOS or updating your Microsoft Windows install from CD before connecting to the 'net. They just give you a poorly made computer with no support whatsoever and let you fall on your face. Superficially, this seems easier to everyone -- there are no *apparent* hassles. The pretense that everything will "just work" is disingenuous but essential. I think people don't buy computers from far more helpful mom and pop shops exactly for this reason -- accepting help is a chore.

Moose is good and Moose is great, but the real win would be to automatically install modules, automatically call the OS's package manager to install libraries and other deps, to have created PHP before php did (or after) and have a libperl linking executable that outputs web pages (period), and most of all, to have a sandbox mode enabled by default (tired idea, I know) where users can write code without getting yelled at by humans, only by the computer, even if just because the code is labeled (all is fair if you predeclare) as "babytalk". Perhaps this would be a mix of strict, diagnostics/warnings, and a Perl::Critic policy that tries to help them in only the most immediate sense likely to be useful to novices in the short term.

Dunno.

-scott

Friday September 04, 2009
03:12 PM

Office ambition

I write a lot about what's wrong with Perl (technically and socially), Parrot, commercial and free Unices, and the working world.

Reading this, it would be easy to get the wrong impression...

I want Perl to have a beautiful core, and I think the potential is there. I want the Perl community to be open-minded, educated, not threatened by other developments in the programming world, organized and efficient at helping novices, and actively painting a good picture of the language and community by addressing fundamental concerns rather than symptoms. I want free Unices to break out of bad implicits that they've adopted without thought because they hadn't formerly had time for critical evaluation and I want commercial Unices to maximize their good qualities and market themselves appropriately. Linux cannot yet touch the scalability, of 10,000 node single-image (one large virtual computer with a fuckton of RAM and CPUs) or stability of commercial Unices and due its priorities of rapidly adopting experimental features and supporting everything, it won't likely ever attain the stability of certain commercial Unices, which run forever, every time you start them. If nothing else, experimental hardware drivers for a billion devices on x86 prevents this.

All of these are ambitions. But I don't have time to contribute to contribute to most of them. In fact, I don't have time to contribute to any of them as long as I'm job hunting and don't know where my next paycheck is coming from.

My ambitions are not only on moulding the technical world to my whims. Despite ranting about how things go wrong in offices, I've had some really good office experiences. I think I've touched on them in bits and pieces. Honestly, I don't remember.

JaWS, Java and Web Services, the business incubator startup Web shop at the University of Minnesota... well, it rocked. It was founded on the idea of learning cutting edge technologies and applying them, mostly to UofMN Web projects. At the time they formed, Java was the thing to watch. They informally assigned themselves to learning interesting new things as they came out. There's an attitude in the working world at large that research and productivity and incompatible. Companies want to hire experts who already know every technology that is going to be used. That's a myth. Research institutions of old, such as Bell Labs, were hugely productive but also innovative. It is a balancing act to mix what you're learning with what you know. My rule of thumb is to be learning one thing at any given time and to at least once try to mix it in where appropriate. Of course you might have to mix it back out, but you're just as likely to have to mix out a technology that's no longer appropriate. Here, I was first exposed to insanely talented people and the loose style of management where the programmers explained what they were doing and management kept them abreast of all things political influencing the projects. I was also first exposed to the idea of telecommuting here, as the sysadmin was in Chicago. Everyone was on IRC, though.

Mayo Clinic first exposed me to the corporate IT management style done right. We were a tiny group of people who created something hugely important in the organization and found ourselves in the middle of a political tug of war between business units and important individuals who wanted things from us. I watched politics be played in a healthy, constructive, highly strategic fashion. Rather than be torn apart, our fame gave us leverage.

NG I'm going to miss dearly. My manager there was a case study in making friends in an unfriendly place -- Vegas. It's hard to say you're going to do something and do it in the software field but in Vegas, you don't have any choice. The bar was set exceedingly high. Systems had to resist physical attack -- it's common knowledge in the computer security field that this is not possible. Straight shooting was critical. The contacts for investors, regulatory agencies, standards bodies, and potential customers all had to be handled in way that I'm not sure any other startup could replicate. Then the two programmers would have brainstorms with lots of groans and sighs about how to accomplish astounding technical feats.

SZ was a chance to work with real coworkers again, in person, part of the time, and to share the humor and wit of smart programmers. One contact there I'm still in touch with and hope to collaborate with. The decision to learn Flash was partially inspired by one of his pet projects that I intended to collaborate on. It was good practice in working under the gun on daunting tasks -- optimizing an insanely large codebase for speed, debugging mod_perl coredumps in production systems, and in coming up to speed quickly on this codebase -- or trying to. After a month or two on the job, having an experimental branch that made all backend requests concurrently felt like a huge coup. Since then, I've been brainstorming strategies, visualization just one of them, for tackling extremely large, extremely crufty codebases. This could be a legimate way that I could contribute to Perl.

Ultimately though, I'd like to get something going like I had at Contact Designs -- two desks, and more if I wanted; one messy, one spartan. Key to the door, card for the server room, my own server in the server room, a bean bag, lots of strange old Unix systems set up and in operation. More authority than responsibility. I'm a far stronger programmer than I was then, but it's not the dot com boom any more.

-scott

Tuesday September 01, 2009
04:54 PM

Code badness doesn't bother me

If programming a microcomputer isn't a safe sandbox for experimentation, what is? The problem lies in humans trusting other humans. Banks trust data processes who they should not; shoppers trust sites they shouldn't (I'm sure I'm guilty of this too, and the credit card changed their ToS long ago so that only in extremely unlikely cases am I protected); everyone seems to want to trust an operating system that time and time again proves itself unworthy.

It's easy to accept something once you fully understand (or largely understand) why it exists. Bees serve an extremely important purpose in nature. Not to draw a close analogy, but I often attempt to explain Java's success and importance to people who only see the superficial verbosity and its pretense of seriousness. That'll be another rant.

Big balls of mud don't bother me and I don't mind working on them other than that doing so is a huge undertaking with slow results. What bothers me is management so eager to accumulate code debt but never pay it off. The blame for the lack of productivity of programmers working on their code needs to be placed squarely on them -- punishing the programmers for doing what they're told to do (keep adding features!) is a sick practice I've seen too much of.

Newbie code isn't inherently evil. Blame management who refuses to heed warnings again.

Half finished code isn't evil.

Insanely over-engineered code isn't evil -- that's the gist of my Java rant, actually. The Java style of handling projects amounts to programmers covering their ass. If they document the hell out of everything, formalize everything, and code every anticipated thing as something controlled by an XML config file, then the facts that humans are notoriously terrible at anticipating things and that smaller code is easier to change than massive projects become immaterial. By embracing management in their hubris, there's no question at all where the failure lies -- the management. This is like making a wish with one of those bottle genies -- management gets exactly what they ask for, which is of course not what they need, at least no more so than something much more easily sought after and gained.

I'm not anti-management; in a previous installment, I wrote at length in defense of management. The tone followed this post's, actually -- understanding why management does the things they do and thinks the way they do. The key, as usual, is understanding.

It's not all management here either. Novices who completely pull stuff out their asses and think that there's only one mistake in it and think that as soon as it's fixed, the thing will magically work frustrate #perl but they mostly frustrate themselves. They'll learn eventually that "the one last bug before it all works" is an exceedingly rare beast for non-trivial programs. You can tell these guys all day long that they're doing 1,000 things wrong and that they're thinking about it all wrong.

I hate the Matt's Script Archive guy whose refusal to accept fixes or take his code down wound up making perl, indirectly through formmail.pl, the second largest spam relay on the Internet for ages. His code was bad but I don't hate it; he on the other hand attained a level of apathy and hubris that warrants the title "evil".

I recently had a hard time answering the question about what my code pet peeves are. Perhaps it's telling that I couldn't come back with one. I started on AtariBASIC. It was all line numbers, sneaking lines in between lines and combining statements to squeeze things in so you didn't have to renumber your whole program (which involved retyping it). That and global variables and peeks and pokes and data statements with decimal representations of binary data. I guess I'm daemonspawn. Part of my Vegas talk, the theme of which was how Perl allowed us to do something that much larger companies couldn't manage, was how dubious features of Perl saved our asses. Source filters allowed public key encrypted modules (every part of the system is required to verify each other part against tampering) to be decrypted on the fly. Code was bytecode compiled indirectly with the B backend (care of indyperl), again as required by regulations. For some high performance stuff, I was essentially peeking and poking raw memory, care of mmap, and doing pointer arithmetic, all in Perl.

Subscribing heavily to the 80's and 90's school of OODA probably makes it worse. I can have esoteric conversations wherein byzantine logical structures of programs are devised.

How could I hate other people's code? That would be hypocritical.

In all seriousness, anyone can create complexity. Finding the right trade-off between complexity, adaptability, time (to write), speed, maintainability (it's various aspects), and so on is harder. I'm not overly proud of myself for comiting minor autocracies; but I am confident in the decisions I made under time, budget, and requirement constraints, including estimating how much tooling up and reworking and maintaining was going to be required to get it out the door. I've known other Perl programmers who were commendable in this way. A lot of them have moved to Ruby where they would be permitted to practice this craft. I give far fewer props to anyone whose philosophy is single minded.

Quoting Knuth: But there has been far too much emphasis on goto elimination instead of on the really important issues; people have a natural tendency to set up an easily understood quantative goal like the abolition of jumps, instead of working directly for a qualatative goal like good program structure ... What we really want is to conceive of our programs in such a way that we rarely even think about goto statements, because the need for them hardly ever arises. The language in which we express our ideas has a strong influence on our thought process.

I guess that's why I like the 80's and 90's variety of OODA, that the C++ old timers subscribed to. It not only acknowledges but explores with honesty the complexities and tradeoffs involved in design, rather than the thumb sucking rules-of-thumb approach in fashion now.

-scott

04:15 PM

Work Rant: I'm serious, but are they?

Bad jobs make for good blog posts. I guess good jobs do too. I should write more about the previous employer. But right now, in the middle of job hunting, I'm remembering a former employer/client.

They weren't serious. They compensated by making lots of airs of being serious. I was flown out last minute and put in a nice hotel. I was subjected to a battery of interviews and interviewers, more than one of whom subjected me to a battery of puzzles and riddles. Numbers -- of hits, megabytes, dollars, and employees -- were thrown around in attempt to impress me with their seriousness.

On the job was a little different. The VPN client software for Linux was supported on very few versions of few distributions. It took days with systrace to figure out what it was trying to do and where it was trying to find its various undocumented deps. The company waffled and then had internal battles about whether I would be loaned a company Mac to get around this. I'm not one to covet work hardware, so besides inconvenient, this was insulting.

That's anecdotal for much of the rest that would follow. Really, it was a warm up anecdote, as I really wanted to launch off of another one: internal security, specifically passwords.

Failed password attempts burned. Too many bad tries and the central Microsoft Windows Domain Controller would lock your account. But the account naming system was inconsistent and prone to collisions. Two users with similar names would burn each others passwords. Authentication was needed for everything -- Bugtraq, logging into the development server, access to the staging databases, email, yadda yadda yadda. Your username on one system though would be someone else's, and software that automatically did things on your behalf wasn't keen to this subtlety. IT support happily reset your password all day long as many times as you needed it, and you had to sign a paper documenting that you had it done. Perhaps to them, it made them look good and productive, but it left me wondering to myself, "are you guys serious?".

Thinking back, I'm sure lots of techs and managers had lots of meetings and this is what was settled on. Going about things grotesquely inefficiently was a happy, welcome alternative to starting another round of meetings -- if they could be started, which would surely require lots of coordinated, organized, carefully phrased making noise to the right people. This is not hypothetical -- I sat in on similar meetings.

Any effort to reduce Technical Debt was done on the sly. When the programmers started to organize themselves to improve the release process of versions, make plans for dealing technical debt in the long term, and to form an internal consensus on what the technical requirements (as opposed to marketing requirements) of the next generation platform might be, new managers were sent in to get programmers to stop wasting so much time. Literally, people's time was to be better tracked. They were to be assigned projects, quote them, and be held to task. The worst trouble makers were fired or monopolized by management. Many left.

Management wanted "yes" answers, or perhaps "not just yet" answers. They did not want to take time to understand the complex implications various potions had on technical capabilities and the resources of the technical staff. They did not want to know what tech needed from the business units in the way of policy changes or technology business decisions. Communication was strictly one way -- mandates from on high.

Not being serious about technical matters started in upper management, but the effects trickled down until even the programmers left were only working in a theatrical sense. To do otherwise and actually try to get things done would cause trouble for every other person complacent in management's decision to just not care.

Like the IT support staff who was happy to change your password over and over again, it was easier to pretend that nothing was broken and never getting anywhere was fine. Kenyan customs officials smile broadly with their hands out waiting for their bribe.

When management did want to know why the programmers weren't more productive, rather than launching an open minded investigation through every level of the company, they hired a tough guy and phrased the question closer to "isn't there something you can do to get the programmers to be more productive?". In psychological terms, I suppose this is a case of Fundamental Attribution Error. The higher-ups get paid megabucks not because they don't make mistakes but because they're able to convincingly paint a portrait of the talents because they legitimately believe that their dung smells like roses.

This is my long winded way of saying that among the hundreds of little red flags I've learned to look for over the years for working for countless failed companies is this little gem: beware management that's "busy". That's retarded. Management too busy to be concerned with the most dire circumstances surrounding their programmers is akin to a race car driver who really can't be bothered to be concerned with his camshaft, or an astronaut who just can't give a flying fuck about portal laminates. These managers will do a bad job. Besides sucking at what they do, they're reinforcing their own festering case of Fundamental Attribution Error.

-scott

Saturday August 22, 2009
03:28 PM

Porting AtariBASIC to Adobe Flash

This was a confluence of three ambitions:

1. How do those old Ultima underworld style first personal maze crawlers, the ones were you can only look north, south, west or east, work?

2. The old Atari, Commodore, TI99/4a, Apple II, Vic-20, etc etc rags had the neatest programs for teaching children and novices. Acceleration, Bresenham line drawing, recursion for things like flood fill and random mazes -- so many mathematical and logical concepts sneak in when you're not looking. I wanted to take stock of the old graphics examples I could find and start to catalog them for use in a tutorial to teach graphics programming in a similar style in Perl.

3. Trying to learn Flash.

So far, I've been picking over http://atariarchives.org/, specifically their archive of ANTIC magazines: http://www.atarimagazines.com/index/?mag=antic. http://www.electronicarchives.org/ is also on my list. And so is http://www.atarimagazines.com/.

It wasn't long before I stumbled over a maze game exactly like what I had in mind as I was working on the Tektronix MUD GUI stuff (see previous post) but was struggling with. My Tek version was having line of sight problems where things would be half way behind something and they'd be drawn entirely or not at all. I was afraid I was going to have to do the ray-casting algorithm (not raytracing but somewhat related... one ray is sent out for each column of pixels to figure out what the nearest object or wall is).

Part of my persona is the librarian. I cringe at the thought of communal memory being lost. Early on, I took up as an early historian, rightly guessing that the best way to understand the systems I was trying to understand was to understand a simplified, earlier version of them. First confronted with programming Unix, for example, I read through the bound and printed manual pages for an old AT&T Unix. Since then, I've seen the portion of humanity made up of programmers apparently forget over and over. My obsession with resurrecting the 90s style of OODA was similarly inspired -- it became clear to me that a lot had been forgotten and attempts to fill in the blanks were coming up short.

The downloads are in AtariBASIC save file format. I downloaded and built the "atari800" emulator and have been loading the save files and then printing to disc the listings and then studying the all upper case, commentless, unstructured spaghetti code. After all of these years, my heart warmed seeing the old BASIC prompt. It's remarkable how much thought people used to put into code that jumped around all over the place. Reading and writing old style BASIC certainly influenced me. Having old Atari BASIC code up in the Unix vi editor feels strange.

atari800, available from http://atari800.sourceforge.net/features.html, is fuckin' brilliant. You can launch it with an argument of a bytecode saved (as the Atari 8 bit computer would save to disc or tape) AtariBASIC program or a standard '.COM" style binary (6502 machine code with memory offset and size header) program and it'll load it and run it. Built into the rewritten and customized "BIOS" is a menu and program loader.

A lot of idioms from AtariBASIC translate directly to Flash. Changing graphics mode for blockier pixels can be accomplished with the ._xscale and ._yscale MovieClip attributes. However, Flash is inconsistent about whether it consults the scale for things like hitTest which made for hard to track down bugs. hitTest() uses the scale if you're using on the main window and not otherwise. A lot code written in AtariBASIC spends a lot of time doing PLOT X,Y and DRAWTO X,Y which is easily accomplished in Flash using .moveTo(x, y) and .lineTo(x, y). LOCATE is used in AtariBASIC to see what color had been previously plotted at an x, y location and Flash's .hitText is similar but requires an extra argument to be useful. Without the third true argument, it only tells you if there's a MovieClip object whose bounds includes that location, not if there was actually ever anything plotted there. This sort of premature optimization permeates Flash. Drawing lines in Flash, Flash insists on mitering the ends, dithering the line, capping the ends, etc, and this doesn't play nicely with scale. If you draw tiny lines hoping they get scaled up, they will, but then you'll wind up with no blocky pixels but instead high resolution lines with all sorts of fancy stuff done on them. Besides not faithfully recreating blocky 8 bit style graphics, this plays havok with .hitText(). Even though you clearly did or did not plot through a location, you cannot safely test this fact as the test might hit a dither or a miter or some other line decoration. And these decorations cannot be entirely turned off. All of Flash suffers from this sort of premature graphical flourish at the expensive of CPU. Yes -- both that and premature optimization at the same time. This is exactly the kind of schizophrenia commercial software always seems to suffer from. Drawing each line as a single segment with square line ends solved most of those problems except dithering. Even scaling up exactly a power of two, somehow in Flash's floating point rounding, it still managers to draw lines half way between pixels.

Another thing about commercial software -- and I've written this before -- the documentation is written by professionals more interested in toadying up to the company and painting a picture of roses than actually helping the users. This does great injustice to people trying to learn the technology. _Programming Perl_ is littered with warnings of pitfalls and ways people commonly become confused. Flash documentation seldom has any such thing. The pitfalls are things like "remember to save your work!" as if the product is perfect but the user is not. So, as a side-side-side project, I've been working on my own "Flash on Linux with Free Software tools" tutorial and, true to my butthead nature, documenting the hell out of every pitfall I can find.

This has been a good exercise. Like starting a job at a new company and having to find where everything is, I've had to grapple around inside Flash finding where things are. The next task up is to figure out how to read from the joystick... erm, I mean keyboard and mouse. (Do we *really* call this progress?)

As for seeking the problem to my clipping woes for my Tek MUD GUI, I didn't find the answer in the simple maze crawler but instead found simplifications to work around it. The AtariBASIC maze crawler, rather than having a "cone of view", with peripheral vision jutting out at an angle each way, has a "column of view". Looking down a corridor, you can only see three tiles wide for n (in this case, 8) tiles deep. Since each layer of depth outwards from the viewer is drawn smaller and zoomed into the center of the screen, there's an illusion of a cone of view complete with a vanishing point and further away things being smaller. And this relates to the other limitation -- it cannot render open spaces, only hallways one tile wide. To hide the fact that only three tiles for any depth outward are considered, everything else needs to be hidden behind walls.

This was a fun exercise. I'll better be able to relate to raycasting and other algorithms understanding the simplest form possible of the maze crawler. And I now understand why I'm having the problems I'm having -- my cone of view starts with one tile, then at the next depth goes to three, then five, and so on.

http://slowass.net/~scott/tmp/maze.as has the ActionScript 2 and the AtariBASIC preserved as a comment. It builds with mtasc. The output looks something like http://slowass.net/~scott/tmp/mazeswf.png -- that is, after it finishes drawing the maze in fat pixels and switches to first person view.

Some of the other ANTIC articles have been using custom words in FORT to do Logo style turtle graphics -- fun!

-scott

02:24 AM

Thoughts on _why

_why, or Why the Lucky Stiff, vanished from the 'net and took down all of his various project pages and web toys as he went. These include Shoes, Hackety Hack, a web based Ruby tutor that gave you hints on things to try in a REPL, and Why's Poignant Guide to Ruby. He wrote some of the most celebrated Ruby libraries. The catalyst for his departure seems to have been a person digging for, finding, and then loudly publishing _why's meatspace identity.

Some have accused _why of throwing a "tantrum". This is easy to swallow because it's a pretty solid generalization. But it's hard to swallow because _why doesn't fit many generalizations. He's never really thrown a tantrum. The once he made a pointed remark, he quickly apologized profusely and then struggled to reframe the discussion. In fact, that's most of what he did online -- took a conventional way of looking at things and turned it upside down. If things were at odds, why were they at odds? What was the underlying catalyst for putting them at odds? What way of thinking about the problem put something else at odds instead?

He was a prankster, but apparently someone ruined the joke. Perhaps he was only free to be Why as long as he wasn't real.

Through his work, his primary mission was to make programming accessible for laypeople, especially youngsters. This goal and his API design sensibilities allowed him to create power by dispensing with complexity. Along the way, he wrote childish poetry, performed silly songs, and drew ridiculous sketches.

It seems to me that _why had a mastery of asking himself, "what's the simplest thing I can do?" Most people don't draw because they're not happy with the simplicity of what they can draw. I wonder how many don't program for the same reason. Playing with ActionScript here, I'm daunted by the utter simplicity of the tasks I struggle to complete. Writing Perl strokes my ego. Writing ActionScript punishes it. But _why shows us what's possible -- the self-indulgent creative expression -- even in utter simplicity. In a rare live appearance where he wore sunglasses, his code demo was something he was working on on the subway that some kids on the trail took interest in and helped him with -- an ASCII animation of two sword fighters moving back and forth on the screen slashing at each other endlessly while hurling random insults. It wasn't the fight but the insults that made the program. Anyone can write silly insults that two ASCII sword fighters can exchange. This is dwelling on what you can do and paying far less mind to what you can't. Have fun. Don't let your ego, telling you that your work has to be important and impressive to impressive people, spoil your fun. This putting aside of ego reminds me of some of my favorite people in the Perl community.

_why has done a lot for all of us whether we realize it or not. He's ever so slightly changed the landscape. APIs not just in Ruby but in all languages near it will be ever so slightly better. Attitudes towards learning and teaching will continue to be better. Language features aren't the only things worthy of stealing; so are APIs. And attitudes.

-scott

Friday August 21, 2009
08:22 PM

Job Hunting... Like I remember it but worse

Yes, that _Perl in Vegas_ talk was motivated by fears that I had to begin to look for my next source of bread. One thing I've learned over my the course of my professional life is that finishing a project is always bad news. In this case, it was a known that the company would be put up for sale as soon as a specific list of features had made it through the Nevada Gaming Commission's certification labs. I wanted the audience to think, gee, I'd like to hire this guy. I only hinted at the company going up for sale -- I regret not plugging myself harder.

So, now what?

I've gotten in touch with old friends and asked for leads. I've applied at all of those companies who circulated their names around the Perl conference. I've posted a resume on Monster and dusted off my LinkedIn profile, and actually (going to hell for this one) fed it my address book straight off Google's POP3. Or maybe IMAP. Who knows.

I've consulted my carefully built list of companies that have in the past advertised for Perl programmers and sent them resumes and cover letters. I've searched Indeed and Twitter for leads and applied to all sorts of things that don't sound like good fits. I've spoon fed my resume to those obnoxious forms recruiter websites have for a few recruiters. God how I remember and hate those. And yet I do this for a shot at being a third shift server supervisor for some Web shop in Scottsdale.

A while ago, when I was organizing the part of the photo collection that's hosted on slowass.net, I noticed that a lot of my pics were screenshots of applications I've written over the years, so I created a little portfolio or visual gallery of some of my work and hung it off of my resume at http://illogics.org. I bet that could be integrated somehow to make something really spiffy. Anyway, I imagine that it's fun to look at even if it isn't compelling.

When ever I start the old job hunt, I start making frequent little edits to my resume, and then my resume starts placing obscenely well in Google for certain appropriate keywords. It's kind of embarrassing, and this has never done jack for me, but it's encouraging. Oh, there's a story attached to this too [1]. When recruiters find my resume online, I get emails from other states to the effect of "I found your confidential resume online and thought you'd be a good fit for $position. Please send me your confidential resume. Also, do you have $skill?". I've taunted recruiters in this forum in previous installments.

Going through all of this makes me realize that I have no idea how to actually find a job. And don't suggest jobs.perl.org -- I watch that anyway, and how many jobs do you see in Arizona on there?

The last time I got a job, a recruiter at a different company kicked me over to a friend of his that he'd worked with before. I worked for a brief stint at the company I originally applied to and then when half the company got laid off, I had this other lead waiting for me. It was fuckin' brilliant. But it was not the result of anything I did or any strategy -- from my point of view, it was a lucky accident -- something I couldn't replicate. Generally speaking, every job I've worked for any period of time has been a lucky accident.

I've also been working to retool myself as a Flash/ActionScript programmer, focusing on the free tools such as mtasc, HaXe, and, at least in theory, flex, but this also makes me realize that I'm not part of that community and don't know where to find them. Googling, the first hit is for a "silverlight users group", and then a whole lot of nothing.

And in reply to my software engineering mating calls, I hear nil.

Just like before. Same deafening silence. Same painful recruiter websites and recruiters. Same "urgent need" job posts that stay up indefinitely. Same resume boards hell bent on spamming you with education and training to improve your resume no matter what level you're at. Same extremely grim job market which only recovered partially for a couple short years after the dot com blowout before tanking again.

I still have some tricks up my sleeves. I discovered that I can search Google Maps for "* near: $my_address" to find essentially all businesses. Monday, I'm going to go apply at the various grocery stores and retail stores in the area. After the dot com boom, I worked phone support for Qwest Wireless for several months, but it seems like that's no longer an option -- all of the call centers really have been moved overseas.

Suggestions welcome.

[1] Wondering why my resume wasn't generating leads, I, at one point, installed a survey on it and asked people to vote on why my skills didn't appeal to them. 75% of the votes were for "don't like Perl". There were about 20 options.

-scott

Saturday August 08, 2009
09:50 PM

Myth of rarity of "good Perl programmers"

I learned Java before I learned Perl. This was in the Java 1.1 days. Since then, I've been wooed by Java shops and slid into positions of authority where I was treated as an expert and a resource. Despite my Perl being much stronger and despite having far more credibility in the Perl community and accomplishments there, no such thing has ever happened with a Perl job. Indeed, I've found myself working to prove myself far beyond any reasonable point and still being treated with contempt and suspicion. I hear Perl people comment all of the time to the effect of that they're still trying to convince the company to use testing or POD or other such things. Java programmers on the other hand are able to pitch management and easily sell them on expensive 3rd party solutions or wide sweeping initiatives.

Forgive me, this is a post of generalizations. I'll write up some case studies, perhaps, if people care.

Really f'n smart Perl programmers I know get mundane jobs working on obscure pieces of infrastructure where they stay anonymous and unacknowledged in their company. Java programmers are turned into stars and sent on lecture tours. Google loves to recruit Perl programmers and make them do system administration, allowing Perl only very tacitly for code that no one else will ever touch but demanding Python, Java, or C++ for everything else. Google did a brilliant thing by acknowledging the tenacity and intelligence of those few worthy Perl programmers.

I've written about this before but looking at this situation again I'm reminded of my hypothesis -- the only reason companies hire Perl programmers to write non-trivial programmers beyond the scope of system administration automation is because Perl programmers are inexpensive and submissive. They do what they're told, don't talk back, don't require acknowledgement, don't make a stink about things like testing and security that management doesn't care about nearly as much as deadlines, they don't aggressively negotiate on their salary, they agree to absurd timetables and requirements, and they routine suspend their better judgment to attempt whatever management has proposed.

Looking at Perl jobs, I see sysadmin support style stuff, maintenance jobs where new code is written in something else, but mostly I see sleazy fly by night companies. I keep hearing from companies that it's hard to find good Perl programmers. I'm calling bullshit on that one. I know well Perl programmers who are doing great things in the community and generating news who are working essentially dead end jobs and are not being wooed by anyone. I worked at a large Perl shop where a well known Perl hacker and former pumpkin applied, worked, and was treated like absolute shit -- as I said, with contempt and suspicion.

For companies to lament the alleged lack of existence of good Perl programmers is just another facet of the cutting off the nose to spite the face behavior typical of the haughty attitude they harbor towards Perl programmers. The motive is far more to blame problems on the programmers they have than to actually attract good programmers.

Perl programmers get no respect and aren't about ready to demand it.

Again, I'm speaking in generalities. A few companies are doubtlessly above this, but there are far too few of them to employ the Perl talent. Writing a thousand times more Perl than Java in the past few years, I'm still more marketable as a Java programmer to a degree that far exceeds the relative demands for Perl or Java programmers. Only fundamental attitudes can account for this.

-scott