jhorwitz's Friends' Journals

ah, dreams...

Matts — 2010-01-26T10:31:17+00:00

I just woke from a dream where it was rjbs' birthday and we got him an onion cake. Only the person who made the cake misunderstood, and instead of a cake shaped like an onion he cooked it WITH onions in it.

Bizarro.

Time::Piece test failures on Win32

Matts — 2010-01-25T23:23:53+00:00

Can someone with Time::Piece test failures on Win32 please contact me offlist? I have a proposed patch I'd like to test, but no Win32 perl to test it on.

Warning compiling perl stuff on Snow Leopard

Matts — 2010-01-04T22:13:57+00:00

I've searched but can't find any reference to this...

Compiling IO::KQueue on Snow Leopard I get the following warning:

KQueue.xs: In function 'XS_IO__KQueue_kevent': KQueue.xs:71: warning: format not a string literal and no format arguments

Yet that line is just:

Newxz(ke, max_events, struct kevent);

Which seems to me pretty standard perl, and no format strings involved (though Newxz expands quite a few macros).

still kicking after 7 years...

geoff — 2009-06-09T17:38:25+00:00

apparently, the mod_perl cookbook appears in the new Cracker video. thanks to perigrin for the discovery.

Early Komodo Edit review

Matts — 2009-06-02T00:05:35+00:00

Recently I started using Komodo Edit (the free version of Komodo IDE). This is an early review, of having used it for my work for about a week, which isn't really enough to know whether I'll be sticking with it, but it's a good start. I started to get a bit tired of the limitations of TextMate - an editor I like, and paid for, but the v2 which they keep hinting at seems to never be coming, and it has some short-falls which I find very annoying.

ActiveState actually asked me to do a review of Komodo IDE a while ago and I never got around to it, so I never got a free license key from them. Maybe they'll read this ;-)

This review is of using it on my MacBook (non-pro) 2.4Ghz, 4GB Ram.

The Good
========

Generally it has everything I'm looking for:

- all documents in tabs
- mostly the right keystrokes for everything (i.e. it's not VI or Emacs, though I believe it has vi emulation for the die-hards) - I only had to fix Cmd-[ and Cmd-] for indent/deindent.
- open over scp/sftp.
- syntax highlighting seems to just work (haven't found any flaws yet, though Perl can be tricky that way!).
- shows line numbers, folds, etc in a nice way
- autoindent mostly works well
- macros in Javascript/Python make it VERY flexible
- "show current file changes" (before you save) is an AWESOME feature, and loads them into the proper "diff" window
- macros even allow you to access that diff window, so writing some macros to access our version control system (AccuRev - don't ask!) was relatively straightforward for a non-Javascript coder like myself
- macro output looks like it'll be easy enough to write a Prove/Test.pm runner

The Missing
===========

There are always things missing in editors, usually things I've experienced elsewhere that I'd love to see replicated:

- I *love* textmate's ability to re-indent code when I paste it in, at the correct current level.
- I wish open over sftp would allow me to open a whole directory (i.e. treat it like a temporary project)
- I wish I could open a directory instead of have to create a project.
- I wish tabs would double over instead of flow off the side (as an option) as I tend to keep a lot of files open
- I wish the project pane would highlight the current file/tab instead of the last file I double clicked on
- Documentation of the macro stuff is a bit weak, though the forums are good for help on this (I haven't posted, but the search has helped a lot, and most posts seem to get a reply).

The Buggy/Broken Bits
=====================

- There's sometimes an oddness with macros - sometimes I edit them and the edits don't "take" - restarting fixes this.
- Can be a TINY bit slow. I'm on a fast mac, so I suspect this would be much worse on my G4 laptop.
- The find options seem a bit odd to me - there's multiple locations for "find" - in the toolbar and in a separate window, be nice if it were unified and accessible with Cmd-F
- Open over sftp seemed a bit fragile and has hung on me once

Overall though, I like it. I'm not really sure what Komodo IDE could add over and above this, I suspect it'll just come with a lot of cool plugins and macros that do clever things...

I'll maybe post more on this if I keep using it.

Perl on LLVM

Matts — 2009-04-09T17:57:03+00:00

There was recently some talk on p5p about getting perl up and running on the LLVM. This was following the recent excitement from the Python crowd about the Unladen Swallow project, and less so, the MacRuby Experimental Branch.

So following that post I decided to see how easy/hard it was to get to the first stage - getting perl compiled and running with clang, the llvm gcc-like compiler.

It wasn't too hard (a lot of compiling). After I got everything running I first ran perlbench, which looked reasonably promising:

gcc llvm --- ---- arith/mixed 100 86 arith/trig 100 86 array/copy 100 101 array/foreach 100 92 array/index 100 93 array/pop 100 96 array/shift 100 95 array/sort-num 100 89 array/sort 100 101 call/0arg 100 102 call/1arg 100 89 call/2arg 100 75 call/9arg 100 89 call/empty 100 87 call/fib 100 90 call/method 100 98 call/wantarray 100 89 hash/copy 100 95 hash/each 100 94 hash/foreach-sort 100 97 hash/foreach 100 91 hash/get 100 91 hash/set 100 89 loop/for-c 100 86 loop/for-range-const 100 111 loop/for-range 100 116 loop/getline 100 96 loop/while-my 100 94 loop/while 100 96 re/const 100 86 re/w 100 89 startup/fewmod 100 95 startup/lotsofsub 100 93 startup/noprog 100 101 string/base64 100 89 string/htmlparser 100 92 string/index-const 100 81 string/index-var 100 108 string/ipol 100 103 string/tr 100 86 AVERAGE 100 93

So next step was to try some more real-world code. I took 41k non-spam mails and ran SpamAssassin on them (using the mass-check tool), with no network tests enabled, and a HTML::Parser also compiled with LLVM (and gcc, in the gcc instance).

Results of the timings:

GCC: real 40m56.599s user 64m44.586s sys 0m59.644s LLVM: real 45m38.831s user 71m14.218s sys 1m20.882s

So rather less promising.

Still, an interesting start - see the original link for information on where it needs to go from here. I think this might have a lot of mileage if the actual internals were ported to LLVM style code. If someone is interested in picking up this project, and maybe being paid for it, please get in touch.

Dakar

Matts — 2009-01-19T15:10:00+00:00

Just want to publicly say congratulations to my brother-in-law Quin for coming fourth overall in this year's grueling Dakar rally. An amazing achievement for a non-factory team.

Perl Unsafe Signals

Matts — 2008-11-03T20:39:10+00:00

This is just to document (and place in google) how to do unsafe signals in recent perls without loading a non-core library to do it:

use POSIX qw(SIGALRM); my $timeout = 30; my $sigset = POSIX::SigSet->new(SIGALRM); my $action = POSIX::SigAction->new( sub { # re-install alarm in case we were in an internal eval{} block alarm($timeout); die "timeout working on: " . (caller(1))[1] . "\n"; }, $sigset, &POSIX::SA_NODEFER, # turns off safe signals ); POSIX::sigaction(SIGALRM, $action); my $prev_alarm = alarm($timeout); eval { # long running code here }; my $err = $@; alarm($prev_alarm); if ($err) { if ($err !~ /timeout working on:/) { die $err; # propogate this error } # process the timeout }

This is written for alarms, which TBH is probably where you really need it, since the regexp engine can get its knickers in a twist and not fire your alarm until the heat death of the universe, but the code will work for other types of signals too.

And yes, I know there are modules on CPAN for this, such as the excellent and very simple Perl::Unsafe::Signals, but sometimes another module isn't an option. I also know the code is a bit flawed in that the second installation of the alarm doesn't do the right thing (it should install at as $timeout - (time - $start_time)), so feel free to fix it yourself.

Dear lazyweb

Matts — 2008-08-25T16:23:58+00:00

How do you get your iPhone 3G wallpaper onto the "Home" or "Applications" screen? There's a screenshot of this on the iphonefaq web site, but my "Wallpaper" just seems to apply to the iPhone "lock" page.

Maybe found an email client...

Matts — 2008-06-09T17:49:19+00:00

I'm currently testing GyazMail. It's a little slow at pulling the messages from the IMAP server, but apart from that appears to do everything I want, and my desktop is no longer grinding along at snails pace.

Trying Thunderbird (on Mac) again...

Matts — 2008-06-08T14:24:45+00:00

I hate mail clients. Really there's no good one I've found yet.

Yesterday I got sick of Mail.app spinning disk for no good reason whatsoever. Yes OK, so I have an IMAP system with several gigs of mail in it, but pine copes just fine with it. When Mail.app is chugging I can run fs_usage and see that it's just a blur of "Mail" entries.

So first I tried Opera Mail. That's OK-ish, but it's so integrated with the browser that it doesn't obey your default browser setting for links, and had a number of other weaknesses that I couldn't live with (like it didn't like hierarchical folders in IMAP).

Then I thought I'd give Thunderbird another go. I actually use this as my PC mail client when I'm forced to use Windows, so I figured I'd give it a go.

My mail client has to be able to do a few things that I've become used to:

1) Sort threads by most-recent-arrival. This is absolutely critical - why would I want a new mail arriving to be attached to a thread WAY up in my scrolly view? Now ThreadBubble promises some of this, but it requires you sort by Date and not arrival time - which is utterly broken since so many systems produce incorrect dates on mails, that when I enabled it there were a bunch of completely out of order threads at the bottom of my subject pane. Plus it didn't seem to entirely work. For what it's worth, the feature I want is a more than 3 year old bug in Thunderbird, and not even due for 3.0a1.

Oh, and if it could do Mail.app's method of grouping by thread but just showing in arrival order that would be bliss (alpine now offers this).

2) Integration with iCal and calendar invites from my windows-using colleagues.

3) Cope with going offline without nagging. Due to my VPN/Firewall situation at work, things regularly disconnect, so it needs to cope with that without nagging. This is currently a showstopper with me for thunderbird.

4) Good fast search. For some bizarre reason Mail.app's Tiger version removed the "Subject or Sender" search. Now you have to choose which you want. That's simply not efficient. Thunderbird's is nice. But their full text search isn't fast. Integration with Spotlight would be good, but the support for doing that in Thunderbird is buggy, apparently.

5) Nice drag and drop UI - this makes it easier for me to drag missed spam into the right folders (they are categorised into "regular spam", "foreign" and "nigerian", so having to type these folder names every time (a-la pine) is a pain.

6) Support for a mixed proxy environment. I have two accounts, one requires a SOCKS5 proxy (that can go up and down all the time) the other doesn't require a proxy.

I'd do it...

Matts — 2008-05-17T02:59:46+00:00

If I were asked to speak at google I'd ask for this too:

I simply ask Google to change a few lines of code, you know, couple of minor alterations, nothing truly significant. That way, he said, anytime anybody in the world Googles, “photography,” they get sent to my website.

From: http://www.joemcnally.com/blog/2008/05/16/google/

Nothing much. Just a few lines of code. No speaker's fee. :-)

please don't re-spam me

geoff — 2008-05-05T12:54:43+00:00

have email administrators not figured out yet that sending the From address "the mail you sent was spam" mails just doubles the amount of spam traffic? everyone knows the From address probably didn't originate the mail, right? have we learned nothing over the past, what, 9 years?

Config file complication

Matts — 2008-05-02T18:59:51+00:00

Someone asked on IRC today what's a nice simple module to use for config files. Someone suggest YAML. Jeez, talk about not simple. And YAML is actually hard to write by hand. Someone else suggested XML with XML::Simple. Same thing - a large complex module and the config format isn't particularly user editable. No disrespect meant to those who suggested those things - I just think differently...

All you usually need is INI files. Here's a simple parser I often use:

sub parse_config { my ($self, $file) = @_; open(my $fh, $file) || die "open($file): $!"; my @lines = <$fh>; close $fh; chomp(@lines); my $config = $self->{config} = {}; my $section; for (@lines) { s/^\s*//; s/\s*$//; next unless /\S/; next if /^#/; if (/^ \[ (.*) \] $/x) { $section = $config->{uc$1} = {}; } elsif (/^ (\w+) \s* = \s* (.*) $/x) { die "key=value pair outside of a section" unless $section; $section->{lc$1} = $2; } else { die "invalid line in $file: $_"; } } }

It's nice and simple, requires no external modules or CPAN downloads, and works pretty well for most situations where simple config files are needed.

Seed

Matts — 2008-04-15T10:05:38+00:00

For a test. Please ignore. mark@advancedambient.com.

Fun with IMAP

Matts — 2008-03-05T03:44:15+00:00

So I've been playing with IMAP lately, processing a folder of emails to process. It has been kind of interesting (to say the least).

Those of you who follow my CPAN uploads know I created Net::IMAP::Simple::NB, which is a subclass of Net::IMAP::Simple but using non-blocking I/O. The idea for that originally was to create a webmail system for AxKit2, but it was also fun learning how IMAP works.

There are some really interesting corner issues in IMAP that surprise me. For example an IMAP server is expected to be able to parse MIME messages, and be able to provide different components of the body as requested. This seems odd to me as frankly this should be a client task - surely this slows IMAP servers down? Also when you ask for the body structure it maps it in a very odd manner - though perhaps its useful for clients - there seems to be a lot of structure data there that isn't useful. There are some other odd choices too, like dates are provided as "DD-MON-YYYY" rather than the more ISO compliant "YYYY-MM-DD" format, and email addresses are provided pre-parsed, which means if your IMAP server gets it wrong you're screwed (though I guess that isn't too common).

On the other hand there are some nice things too, like IMAP provides most of its data in an S-expressions-like format, which is quite easy to parse in perl. The RFC is also not as horrible as I had once thought, so decoding various things is reasonably easy. On the other hand, it seems Net::IMAP::Simple is a bit too simplistic, in that it doesn't parse the output formats properly, and honestly doesn't have enough methods in it (I've added 8 methods just for my one simple app).

I'll have to consider contacting the author about patches. Though I'm unsure right now if I want to have too much to do with it in the long term.

Time::Piece 0.13 released

Matts — 2008-03-02T20:24:10+00:00

Some of you may know that Time::Piece became part of perl 5.10. Anyone one of the things discovered was that I accidentally dropped a BSD copyright off one of the XS functions. This is now added back in. I also added an add_months() and add_years() set of functions, to make date arithmetic a bit easier.

This was interesting seeing what happens with the edge-case dates. It seems that when you do 31st March + 1 month you end up with 1st May. This mirrors what other date libraries appear to do. I documented the gotcha.

Enjoy.

Winter Creek

Matts — 2008-02-04T14:50:17+00:00

Image

A small creek frozen over that I found on the drive home yesterday. The
utter silence while taking this photograph was perfect, broken by the
distant screech of an owl.

Original here

Brrrrrr

Matts — 2008-01-21T14:05:56+00:00

Image

It's deathly cold out there, but I decided yesterday to wrap up warm
and get my ass out there to do some photography. It was worth it for
the results, but I sure couldn't hang around long.

Original here

On feeling out of touch

Matts — 2008-01-05T02:52:58+00:00

I haven't done much web development now for 7 years. I did a large-ish project at work using AxKit (probably one of the highest traffic mod_perl applications in existance, doing about 200 million transactions/day) about 4 or 5 years ago, but I don't really count that as it didn't expand my knowledge of web technology in any way.

I keep reading about new web "stuff" - Rails, Django, Seaside, etc. None of it really interests me all that much because frankly the core of the web doesn't appear to have changed a whole lot to me - you do a request and you serve a response. But maybe I'm missing something.

Email stuff of course still interests me because it's my day job. What's fascinating about my day job is that it's entirely built around people breaking the system and doing "weird" things, and me having to fix stuff. The most recent example was spammers sending QP encoded emails with high bit characters. That broke my QP decoding routines, but it also is a 100% spam indicator.

So is what is actually new and interesting that I should be looking into right now to keep my skills honed? When the web was dying (i.e. around 2000) it was XML, and learning and working with that served me well. What is interesting now?

Terrorists win?

Matts — 2008-01-05T00:28:54+00:00

So tomorrow was supposed to be the start of the Dakar Rally. My brother-in-law Quin Evans was supposed to be co-driving in the race with his friend Ivar Tollefsen. A years preparation goes into the race.

Now the rally is cancelled because Al-Qaeda were supposedly targetting the rally. I hate that. Giving in to terrorists is just the worst thing. But losing a brother in law would have been worse.

Rails has peaked

Matts — 2008-01-04T17:49:05+00:00

I found this blog post rather interesting. Though take it with a grain of salt obviously.

"it's just..."

geoff — 2007-12-21T14:22:39+00:00

I was in a meeting yesterday where a seasoned but non-technical person was surprised that the project at hand involved a substantial amount of re-coding. "I thought it was just..." the sentence began, which then continued (in my mind) as "completely re-thinking $concept and normalizing $concept data across 38 new tables instead of the current 5" but sounded like "switching around a few tables" to everyone else.

I've worked with lots of non-technical folks over the years, some good, many bad, and this person is one of the good ones, so I found it amusing rather than irksome. but, for the record...

there is no such thing as "it's just..." in software engineering anymore. at least not for mature applications with an established userbase where real money is involved.

Joke

Matts — 2007-12-02T14:41:09+00:00

What goes: Bump bump bump bump bump splash

Answer: Me falling down the basement stairs with 2 drinks in my hand.

Mother %^&#$%@! did that ever hurt.

Sand Grass

Matts — 2007-11-24T00:00:26+00:00

Image

Not much to say about this - shot at the same time as the previous pic.

Original here

The Beach

Matts — 2007-11-19T18:52:55+00:00

Image

Shot at the beach in Kincardine, Ontario. This is mostly as-shot thanks
to my new Cokin filters (polarizer, ND-Grad, and ND filters used here).

Original here

on garbled profiles

geoff — 2007-09-28T15:07:23+00:00

anyone who has used dprofpp has seen entries like this:

Garbled profile, missing an enter time stamp at /usr/bin/dprofpp line 785, <fh> line 144174.

Modification of non-creatable array value attempted, subscript -1 at /usr/bin/dprofpp line 679, <fh> line 142590

ungarbling a profile is a lesson in patience and rote: you go down to the line in question, figure out what sub it's talking about (the number at that line), go to the top of the file, grep for that number, find the subroutine it's talking about, add that sub to your ignore list, lather, rinse, repeat until the problems go away.

ugh

while overloaded subroutines are notorious for this behavior, they are easy to weed out via a simple code scan. but once you remove the low-hanging fruit, it's a chore to ungarble a profile if your codebase is anything more than something very simple. and, for some reason, it always seems to boil down to DBI-based calls once I get to the bottom of things.

well, this morning I decided enough was enough, I'm going to figure this out.I generally don't care about specific, low-level DBI-ish things - the amount of time (and number of calls) to fetch() is much more important to see than what DBI is doing under the hood in that mire of package madness. so, using a lot of verbose Devel::Profiler tracing and some moxy, I was able to add these packages to by bad_pkgs entry:

DBD::_::st DBD::_::db DBD::st DBD::db DBI::st DBI::db DBI::dr)

which hides low-level DBI calls from the profiler but not those from specific drivers (which contain enough of the database overhead to be interesting). so, with my profiling back in order, I can see all the stuff I'm interested in, including entries like

1.31 0.293 0.293 2365 0.0001 0.0001 DBD::mysql::st::execute 1.02 0.229 0.229 4942 0.0000 0.0000 DBD::mysql::st::fetch

anyway, I hope this helps some of you out there. happy profiling :)

Cwd failing?

Matts — 2007-09-20T15:42:48+00:00

Has anyone seen cwd() failing on an extremely high load server? I don't really understand why it fails (only happens under extreme load), and wondered if anyone had any insight.

DBD::SQLite 1.14

Matts — 2007-09-19T19:30:29+00:00

After a long hiatus I've finally done a bit of hacking on DBD::SQLite. It's updated to the latest core sqlite, and applied a bunch of important patches from RT. I also added in the FTS code (edit: FTS is Full Text Search) - though I haven't tested that so if someone else can do that'd be awesome (if you can supply me with a working .t file for adding to the distro that'd be doubly awesome).

Enjoy.

I made the front page of Slashdot...

Matts — 2007-09-07T18:07:48+00:00

http://it.slashdot.org/article.pl?sid=07/09/07/122200

Cool.