Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

jjohn (22)

jjohn
  (email not shown publicly)
http://taskboy.com/
AOL IM: taskboy3000 (Add Buddy, Send Message)

Perl hack/Linux buff/OSS junkie.

Journal of jjohn (22)

Tuesday October 28, 2003
09:49 AM

When glibc attacks!

[ #15427 ]

Like a ethnic dish, some mistakes we are destined to repeat.

In the course of my Linux experience, I've hosed many a server. Sometimes I've assigned swap space on an external RAID box. Sometimes I've created overlapping disk partitions on hard disks. Sometimes I can't even tell what I've done. This is what makes Linux and Unix fun.

However, there seems to be one mistake I can't stop making. A Unix system layered like a onion. Each layer depends on the preceeding one to work. At the lowest layer is the kernel, which futzes with the hardware and attempts to make all Linux/Unix systems appear to be the same from an application programmer's point of view. The bridge between the kernel and applications (and even basic system services like init and bash) is the libc library. Without this library in good working order, it becomes difficult to exit one's shell, let alone reboot the system cleanly.

Now, I upgrade various bits of my system all the time. I'm not afraid to recompile the kernel. I'll update the C compile and attendent utilities with wild abandon. However when it come to updating that most scared of files libc, my track record is not so hot. You see, get lulled into thinking that a package manager like rpm can Do The Right Thing to install libc without hosing the system. In fact, a google search relieve a testimonial from a user with the same version of RedHat I have successfully upgrading his libc to the version I also desired.

Alas, not all messages should be taken at face value.

There are three libc (really glibc) packages that needed to be successfully installed. The first two worked fine. The last failed and precipitated a cascading failure that resulting in my reformating the root partition and laying out a fresh install. I got to bed around 2A. Good times.

One of the last things I did was to install Spam Assassin, since I was already getting swamped with spam. Of course, sendmail tripped me up. I forgot to add symlinks in /etc/smrsh to procmail and spamassassin. So imagine the hilarity as I awoke to find 4000 messages being downloaded by fetchmail, nearly all of the bounces from my local sendmail saying "I don't do procmail, biznatch." Boundless joy!

With that sorted out, DNS back in place, and offloading DHCP to my linksys router, I'm pretty much back up and running with the essential services. Of course, I'm missing a lot of formally installed Perl modules. Like SOAP::Lite, which I need to post this journal.

Sigh.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I run Debian unstable, and have upgraded libc and it's associated files countless times, without any problems. Of course Red Hat screws it up, but their package management system is horrid.
    • That's so Linux user. There's a little script that all Linux people go through when they're doing support:
      • Doesn't work? Upgrade your kernel!
      • Still doesn't work? Try a different distribution!
      • What was your problem again?

      I'm sorry. I've been reading jwz [jwz.org] too long.

      FWIW, I've been using RedHat a while and never had any problems with glibc upgrades screwing up. Lots of problems with locales (solution: nuke), but never glibc. :-)

      -Dom

      • Well, there are certainly other choices, like not using Linux. I doubt that any other Unix-y thing will be much better than Debian in terms of ease of upgrade, though.

        Then there's Windows, which we all know and love or hate. As for Mac OS X, it seems nice, except for all the things I need to do my development work that only sort of work/compile properly there. I don't want to do a compiler dance every time I need some new Perl module with XS installed.

        Basically I've really only used two Linux distros,
  • [insert obligatory anacdote about Debian and its ability to update libc three times before breakfast here]

    That out of the way, some advice. Back in the mists of time when Apple Powerbooks were black and named after streets in New York, some Linux nutters decided it would be cool if they could run something other than a crappy co-operative multitasking system (translation: we crash by mutual consent) on Apple hardware. Enter Debian Linux/PowerPC. I had interesting times in 1998 when they were just getting off the ground. There was no stable tree, just unstable and yes it was unstable. A libc upgrade causing catostrophic system failures was a regular occurance.

    So what I learned to do, and my advice to you, is to do a static compile of your basic utilities to have in an emergency. fileutils, bash, textutils, rpm, etc... Or keep a copy of sash (Stand Alone SHell) around and set root to use that. It'll at least let you do *something* when libc fails.

    Or you could just use Debian. *p0inT and l4f*

  • Moving from RedHat may be an option for home use, but my two jobs won't be migrating from RH any time soon. And here's a confession: I like Red Hat. Yup. I like RPMs and SRPMs. I've even figured out how to use chkconfig to register my sysV init scripts. Would it be cool is rpm was more like CPAN, in that it sought out missing dependencies over the net for you? Sure. I know that's what RHN is all about.

    That said, only a fool wouldn't try to avoid making the same mistake again. I will certainly look