My colocated box got hacked over the weekend and has been flaky all day. This means rhizomatic looses an IRC node and much more importantly neither I nor my friends can read their email while we bring it safely back up. It's really annoying. Why can't computers just work and never go down?
I haven't been without email for a while. I wonder how I'll cope? Will I go mad?
One of us is going down to the colocated place and reinstalling from scratch, which will be fun. It'll then be up to me to slowly install all the services, with an eye to security. Sysadmins rule. If only we had one
And after everything is up, we'll set up lots of monitoring as well as secondary MX etc. etc. so it never happens again and if it does we can do it all again quickly. At least the box has silly amounts of bandwith. Why does it always take an actual exploit to get me as worried about security as I should be?
Oh, and debugging SOAP::Lite stuff at work hasn't helped, either...