What Happens When You Get Dugg

A few days ago, this site made the digg front page thanks to this post on Hidden Gems in OSX. This is the first time a site I’ve been involved in has hit the front page of digg and its an experience that many will never encounter, so I thought I’d run you through what happened. It certainly wasn’t all a bed of roses. I’ll also let you know why I’m now with a new hosting provider and the difficulties I had with the old one.

The story was submitted at about 4pm GMT on the 11th. When I checked at about 4:30 there were around 12 diggs and I thought nothing more of it. I’ve had stories submitted to digg before and they’ve achieved a similar number of diggs in a similar amount of time and then just tailed off. I went back to doing other stuff, oblivious to the fact that the site was about to be brought down for nigh on 3 days. Later that evening I was reading through my RSS feeds in Google Reader. I happen to subscribe to the digg front page RSS feed and as I was reading it something familiar popped up, it was the story on this site. I checked digg and it had reached about 700 diggs in around 3 hours (so far as I can tell). Oh, I thought, I’ll check my stats. The only problem was that I couldn’t get into my stats package, in fact I couldn’t get into the site at all. Luckily, the blog post seemed to be holding up intermittently. This is where the fun starts.

I checked the comments on digg and it turns out that people we’re reporting the site as being down after only 26 diggs! That isn’t entirely acceptable. It took another four hours before I could check my statistics, when I did FireStats was reporting just over 13,000 uniques. Over the space of approximately 6 hours, thats not a HUGE amount. Luckily dugg mirror picked up the site so the content was still available to those who wanted it, which is obviously a good thing. Unfortunately, it also means the stats aren’t entirely accurate as a large percentage of visitors would have ended up going to duggmirror instead of this site. The most accurate measurement of traffic I had came from Google Adsense. Despite the fact that most visitors from digg would have these advertisements blocked, it was still showing 19,ooo views at the same point FireStats was showing 13,000. I can only assume this is because duggmirror was showing the Google Adsense ads. For the next few hours the site was only up intermittently, dropping out more than establishing connections. Unfortunately I couldn’t get into the admin panel to view the stats nor could I get to awstats. By all accounts the server was struggling.

How to Kill a Server

At the time my site was hosted with Go Daddy. I’ve used them for many years as a domain registrar and had accumulated a few hosting vouchers, so I thought I’d give them a try. This site was typically receiving only a very small number of hits, so I had no trouble trying them out. I’d also heard good things about them and had never had any trouble with my domains. I went for one of their mid range Linux Shared hosting plans, this was a mistake. I’ve used shared hosting before for other sites that don’t require a huge amount of resources and never had any trouble, even when traffic spiked I’ve not had any problems. When the site was neither up nor down I posted over at Weborum to ask whether I could expect the site to stay up, and I got a shocking answer. one of the admins over there, Leo, pointed out that there appeared to be over 2100 sites hosted on that server! Thats alot, and far more than you would expect from any other host. No wonder the site was struggling.

Still, the site was still up intermittently when I went to bed. Unfortunately, this was no longer the case the next day. Every request made to the site was met with a 403 Forbidden error. This meant that the site was up and responsive but someone had changed the permissions on the site. This person was not me, so off to tech support I go to enqure what was wrong. Now I’ve always found Go Daddy tech support a bit fiddly to deal with. You get the impression that they don’t 100% understand the question you are asking. It’s always taken at least 2 emails before they give you a straight answer. And it took at least four for them to acknowledge a fault in their new domain management service. Anyway, back to the point in hand. So I fired of an email to Go Daddy support asking why the site was down and pointing out that I was nowhere near my bandwidth or storage limits (only around 1% of each). The reply I received, after only five hours (!), indicated that a script on my site was using a large amount of resources and as such they had changed permissions. Now there are only two scripts on that site, WordPress and a very small php script of my own doing that runs the main part of the site. WordPress isn’t known for being a resource hog and my tiny script certainly isn’t. Maybe it’s down to the fact that there are over 2100 other sites on that poor, dying server.

The following few hours really opened up my eyes. I replied to the customer support rep and asked what I could do. I pointed out that I could not access the admin panel of WordPress nor could I FTP in to the site. This meant that I could not disable any scripts or plugins in an attempt to lower the cpu usage. I was stuck between a rock and a hard place and asked the very obvious and simple question, what can I do to get my site back online? The response I received was really unacceptable. Because I mentioned WordPress I basically got the same response I received the first time but with a “We don’t support third party scripts” message tagged on the end. Interestingly, I also asked whether it was common practice for them to host so many sites on a single server. Now, bear in mind that this is publicly available if you know where to look, the support rep stated that it was policy not discuss those stats. Maybe I should have just sent him the link as proof. Obviously, by this point I had asked in very clear terms what I could do to get my site up, twice, and not received any hint of an answer. I was not a happy person. And to make matters worse, the hits were still flooding in as evidenced by the 28000 hits registered over three days in Google Adsense.

So to kill a server you put 2100 sites on it and to lose a customer, well, bad customer service will do it every time.

The Other Way

At this point Go Daddy had basically offered me no choice other than to move my site elsewhere. I could not access any part of the site, let alone the WordPress admin panel and I could not ftp in. What else could I do? Luckily, I had a local backup of the site and its theme. I also had a recent backup of the content although, unfortunately, not entirely up to date. After a brief search around the interwebnets I decided on a new host. i won’t say who it is for fear of cursing but, so far, its excellent. The site is a whole lot faster and at the time of writing there are less than 160 sites on the server. Compared to the 2100 other sites I was neighbouring with at Go Daddy, my site feels a lot less overlooked.

Moving the site to a new host was certainly an interesting experience, especially in such a short space of time. I’ll do a separate post on that at a later date.