At work this weekend we noticed some outages on our website, and come Monday it was down cold. I initiated a service ticket with our web provider, Hosting Matters, and heard back from them pretty promptly. I had asked if there was any scheduled downtime or if our server had gone south. This was their reply.

Definitely not scheduled: when we replaced the backup drive, suddenly the server began throwing all sorts of errors, high loads, etc., without actually writing anything to the various system logs to tell us what it thinks is wrong. Based on what we’re seeing, either the primary drive is also dying, or the hardware controller is, but in either case, it’s bad news and we are trying to get everyone off the server as quickly as possible, pushing them to other (newer) servers in the network. This process is being hampered by a still-high load, and we’ve had to turn off apache to get things to move. Short version: the server itself is not down, but some services are, and it’s dying a slow, painful death as we try to archive sites and get them away. It’s a work in progress that unfortunately won’t respond to a giant hit with a hammer (but I think we will totally be doing that to the server once we manage to get everyone away from it).

Can you imaging someone taking the time to render that much detail in a reply when you know they’re overwhelmed with an urgent and prickly problem? They even have a sense of humor about it! I was impressed and replied back and told them so.

Come Tuesday our site is back up and page loads are faster than ever. They must have moved us to some fancy new hardware. We’ve been with this company for about 10 years and have never had a reason to consider switching.

I remember years ago when I was trying to go live with my little book database for the shop and needed certain PHP graphics functions to be enabled on our server. Everything worked on my test set-up on the laptop but no go on the live installation. I got in touch with them and they had TWO people fooling with it for a while, trying to figure out what was wrong. Finally someone guessed that the PHP modules might not like the order in which they were being loaded and so they switched that around and sure enough, problem solved. These people are cool.