Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Wednesday November 20, 2002
01:08 PM

Simple math not so simple

[ #9041 ]

We have a client, who shall remain nameless, who provides data feeds for a few very large companies. However, those companies don't get the datafeeds directly. Instead, a series of small programs that I wrote read in the data (in the form of CSV files) and users who pay a small fee can view the resulting Web pages. The problem: this was a quick hack.

I was originally asked how fast I could get these pages up as our clients were in violation of their contract terms to provide this info, so I took about two days to get this put together. This was done with the caveat, carefully explained to our client, that the data format must not change. I built in a bit of flexibility, but I simply didn't have the time to do this right. The pages have continuously broken since then. As it turns out, there's only been one bug in my code (woohoo!). Every other time things have broken has been due to their changing the data format without telling us. Today was the latest. Originally, the HH:MM:SS in the data file would be in the form /(\d\d):(\d\d):(\d\d)/. Now they've changed that so the above regex fails. It's now /(\d\d?):(\d\d?):(\d\d?)/.

A couple of months ago, they stopped sending timestamps, just date stamps. Sometimes they've changed the data keys without telling us. What I really enjoyed, though, was the time they decided to increase the number of fields they were going to send. Naturally, we didn't find out until the people downstream from us called to yell at us. Our clients keep sending bad data and the people who see it yell at us and we can't sit around and trash our clients by pointing out that they're clueless. We can't win. In fact, this little job may just go away entirely because our client can't seem to send us properly formatted data and the downstream people are getting pissed!

So what's the "simple math" I referred to in the subject? Originally, we had a very, very small bid to put data validation in this project. If anything didn't meet our expectations, we would send email to our client and ourselves and nothing would get updated (and risk breaking the Web pages -- better old data than bad data). We were turned down and now the client has spent 10 to 20 times that cost in tracking down bugs, not to mention the damage to their reputation. Their reason for turning us down? "These are computers. Once we get it dialed in, nothing will change."

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.