Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Tuesday October 22, 2002
11:34 AM

The Strangest Bug

[ #8529 ]

Well, we're stumped. A site that has previously been working fine with MS SQL Server 2000, IIS 5.0 and Windows 2000 has started to crash and burn ... intermittently. I've been working on some updates and a section of code that I have never changed is causing the problem.

if ( $page eq $page_control{default}{page} )
{
    my $loginName = $db->get_user_info($user);
    my $message   = $db->get_admin_message;

It's that last line which is killing us. I even have it traced down to the actual statement handle execution, internally. I still don't know why it fails. That's not the weird part. Normal debugging methods have failed because when this crashes, everything locks up. We get no error messages from the database, from IIS, from anything.

I've tried $|++ and putting in carps before this code to narrow it down. No dice. There's nothing in the error logs.

I tried tossing in a BEGIN block to dump state information. No dice. Still nothing in the error logs.

It's almost as if the program senses that this method call, buried in a conditional a couple of hundred lines after the beginning of execution, is going to lock up so the program decides that it's a waste of even trying to run. If this program is executed in such a way that the condition is false, everything, including the debug warnings, works fine. The only way I managed to track down the bug was to dump some test output at the top of the program, put an __END__ token after the test output and do a binary search with this through the program to narrow down the exact cause of the failure.

Truly, this is the strangest bug I have ever seen. Frankly, I suspect that this is some weird ISAPI error, but I can't prove it yet.

Update: We've narrowed the problem down to one database table that has one record and one field. We can't select, update, or delete that record, though we were apparently successful in inserting a new record. I say "apparently" because we still can't select anything from the table to verify it. We tried dropping and recreating the table, but SQL Server will just timeout when we try and drop it. Of course, this doesn't answer the question of why a problem with SQL Server will cause ISAPI to stop writing error messages to the logs ...

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • What sort of database? Is each table a file?

    If so...

    Is it possible that an earlier instance of the script crashed or was interupted while it had that file open? Can you read the file in a text editor, or does NT give you an error?

    Do you see different behavior depending on whether you read/copy the file on the local machine vs via filesharing?

    If you can't read the file from a share (and possibly locally), it may be that Win thinks the file is in use, possibly by a crashed/interupted previous instance of
    • The database we're using Microsoft SQL Server 2000. We stopped and restarted that server and, before anything else could touch the database, we successfully dropped the table. I've added it back and everything seems to work perfectly. Of course, we haven't done any load testing on the new table, so it might be a bug waiting for its opportunity to bite. It's extremely frustrating that we cannot find any logs anywhere which refer to this table or what the problem might be.

      I've double-checked the code wh

  • The better DBA's I've worked with have been able to extract a healthy pile of clues from the admin tools provided with SQL Server (e.g., who has what locked).

    Since you're dealing with a single row in a single table, my number one cause of mystery grief on SQL Server--row locks getting escalated to page locks--probably isn't an issue.

    • dws wrote: The better DBA's I've worked with have been...

      Ovid replied: ha, ha, ha, grunt, snort, ha!

      We let our "DBA" go on the grounds that we couldn't afford him. That's true because while he might have been qualified to be an intern, he certainly wasn't qualified to be a senior DBA. He lied his @$$ off on his resume and when he was hired, no one was capable of evaluating his performance -- not surprising given that this company once hired a CTO who didn't know what FTP was. Then I returned to

      • "If you need to change it, you can just write a script to change all of the instances of it!"

        Hmmm...

        Actually, you could remove a layer of indirection by eliminating the username entirely. That way, if it changes, you don't have to update anything at all. You'll still be able to refer to users by other unique fields. The probability that any two users have the same hire date and birth date is probably sufficiently low ...and if not, you can just throw in more fields until you eliminate the duplication.

        The