Groundhog Day

January 28th, 2007

Today I decided to take a real Sunday. That is, to turn the computer off and do some massive chore-handling that I’ve been putting off for too long. I decided to clean and reorganize my office. Or at least get started. At around 1:30PM EST, I shut off my computer determined not to turn it on again until around 7:00PM.

Everything went great. That is, I got a lot done. I took my entire desk “apart” and put it back together, in a new location. Rerouted a bunch of clumsy wires. Mopped the floor. Went through files, etc. Hours later, I hooked everything up and turned the computer back on, to make sure it all still worked.

I so picked the wrong day to do this. Upon returning to my Mail inbox, I was greeted by news of a comment on my blog, regarding FlexTime 1.2: “So how come when I go to download it via MacUpdate I get a ‘file not found’ error.” Nightmare scenarios immediately passed before my eyes. I put the wrong URL into MacUpdate? I forgot to upload the file to the server? Surely if something that wrong had happened, I’d have heard by now?

I decided to take a browse to MacUpdate and then to my server. Sure enough, the file was not found. What the heck? I typed “www.red-sweater.com” into the URL field and decided to browse my way to the problem. Here’s where Groundhog Day Syndrome set in: I’m staring at a several-months old version of my site. Old design. Old software releases. What on earth? How could this possibly have happened?

A couple days ago I decided to perform some good housekeeping of a different nature. Although my site has been hosted on Pair for several months, and was hosted on DreamHost before that since April of last year, my domain name registration has been maintained on a pretty user-unfriendly Tucows-based registrar called OpenSRS.net. Since I’ve registered some new names since then on DreamHost, I decided it was time to bite the bullet and consolidate everything into one place. DreamHost has been good for this type of administrative stuff, so why not?

Apparently everything finally kicked in some time this afternoon, as I was knee-deep in cleaning supplies and for once every computer in my office was unplugged and stacked in corner. No problem, right? They just took over the registration of the domain. Yeah, no problem, if you overlook the minor problem that they changed the root nameservers to their own! Ugh! So the reason I’m looking at a months-old version of my site is because I just so happen to have the account with them that red-sweater used to be hosted on, and that old stale directory is still there, unchanged.

I immediately try to rectify the situation by changing the root nameserver values back to their correct, Pair.com based values. In response to this I’m greeted with a lovely internal error from the bowels of DreamHost:

Can't call method "request" on an undefined value at
/usr/local/lib/site_perl/Ndn/Registrar/Registry/NewEPP.pm
line 2137.

Oh, that’s just great. I also notice in looking at my DNS configuration on DreamHost that I actually have a custom A record entry pointing red-sweater.com to my Pair site. Unfortunately, they insist on keeping their own A record pointing at themselves. So if you ask DreamHost for my domain’s lookup, they’ll give you a schizophrenic response pointing both to Pair and to DreamHost.

So I decide the only thing to do is get in touch with DreamHost support. For the first time, I feel sufficiently screwed by the situation that I choose for the severity of the problem the typically cheeky:

“OMG! EXTREME CRITICAL EMERGENCY!! EVERYTHING’S BROKEN! People are DYING!”

The only part that’s not literally true is the part about people dying. But I decided to choose it anyway. An hour later, people are still not dying, but everything is still OMG broken.

I took matters into my own hands. I could either stand by and watch while my customers are treated to a version of my site fit for a museum, or go into damage-control mode. Since I keep my web site’s files in a Subversion repository, it wasn’t too hard to go back to the DreamHost version of reality and “svn update” everything to match the modern-day Pair site. Of course I’m a bit lazy about adding and committing every thing as I do it, so I had to go back into Pair and do some heavy committing. Then do some heavy updating on DreamHost.

But aside from the fact that the speed of the site is back to the lackluster DreamHost level, everything appears to be more-or-less back to normal. I’ll wait patiently now for DreamHost to fix my registration to point back to Pair. But in the mean time, I guess nobody is going to die.

One thing I almost overlooked is the PayPal “IPN” payment notification, which is configured to bounce back to my site and let me know when somebody has paid for one of my products. I have this implemented on my Pair account in a way that relies on some full paths and such to log files on Pair. I really didn’t want to hack this together on DreamHost just to survive the unfortunate retransition. Then I remembered that thanks to Pair’s “one IP per domain” approach to shared hosting, I can point PayPal (temporarily) at the raw IP address version of my site. I’ll breathe a sigh of relief and assume that PayPal purchases will be fulfilled as usual. If anybody wants to buy something just to check, that’ll be fine by me :)

5 Responses to “Groundhog Day”

  1. Jan Says:

    woha :-)

    This story is an excellent read. If you’ve been through similar (OMG!) scenarios, the words just fly by and you are close to regretting it didn’t go worse. But you had your share for today, I guess :-) Good luck with the permanent fixes!

  2. Conor Says:

    The excitement of running your own business. Luckily, after switching to site5 a few years ago, my hosting related problems have almost disappeared. Never switch off your computer and never take vacations, shareware apps have an affinity for Murphy’s law; as soon as you’re indisposed they need your attention.

  3. Justin Miller Says:

    Sorry to hear it… I’ve been on pair for over five years, so I’m glad they came through for you in a pinch :-)

  4. Daniel Jalkut Says:

    Pair has been great. In fact, DreamHost is great, too. In its own way. But not for uptime reliability. I had installed an “uptime check” through a web service, that basically pings the site for reachability every few minutes. I stopped receiving downtime notices when I switched to Pair. Of course, in the midst of this mix-up, when perhaps only half of the world is even getting rerouted to the old site, I wake up this morning to learn that my site was “down for 25 minutes” last night.

    I’m really trying to get past this. Breathe. Don’t get angry. It’ll all be over soon.

  5. Simon Says:

    Daniel, I highly recommend DNSMadeEasy for setting up your DNS, MX servers, etc.

    http://www.dnsmadeeasy.com/

    They are reliable and offer great support. It’s easy to set up a fail-over site, so you could have the web-site hosted at Pair, and if they ever go down, users are redirected to your back-up host e.g. Dreamhost.

Comments are Closed.

Follow the Conversation

Stay up-to-date by subscribing to the Comments RSS Feed for this entry.