Webhost in five day server FAIL
Update: This story has been updated with additional facts from Cirtex CEO John Xie. New York-based webhost HostV - a division of Cirtex - is five days into an server node outage that has left customer websites completely inaccessible. London-based Register reader Alan Ayoub says the outage has brought down 10 of his sites, and …
5 day outages?
Heck, I work for a hosting company, based in California , where a 5 day or more outage on a SHARED (non VPS/non Dedicated) server is practically a weekly occurrance.
This means that at any one time, there are generally 300+ customers who can not access their sites, email, or other features they pay for.
Company proclaims a 99.9% reliability. When called on it, they says its an overall average taking into account ALL servers (shared, VPS, Dedicated) the company owns.
However even when taking this into account, the average is much much lower.
These people complaining don't have backups because it's much easier for them to whinge on forums about sites being offline and their livelihood going down the pan, than to actually have any form of business plan or backup strategy and have to take responsibility for themselves.
As much user fail as server fail.
Poundhost also had a lot of downtime this week on their VPS's due to their SAN dying and it taking forever for HP to fix it for them. I'm not sure how much, as I wasn't paying too much attention due to it not affecting me directly. More than 24 hours though. Look into it.
,,,going around. We had a 12 hour outage at our data center. Accirding to our supplier, eSecure Data, it was caused by a fiber cut.
They have promised to get a backup loop to get rid of this single point of failure.
We run a few low volume sites, but only on our own server via our ADSL line. We haven't found any hosting company that can be trusted. Their reliability figures and claims to provide support are simply lies.
Our bulk data (images etc) is hosted elsewhere, where they claim 99.9% uptime (not down for more than 1.44 minutes a day). But our own monitoring system shows that they haven't achieved that on any day since we used them (monitoring connectivity, simple file fetches and script calls once per minute). We have a simple script there to reduce image sizes when needed and that often doesn't respond for as much as half an hour, with an average reliability about 50 times worse than their claim.
We couldn't sleep at night if we didn't have a fallback system to serve the bulk content from our own server automatically (with a banner apologising for the temporary fault and slow service).
Problems restoring backups? They obviously hadn't done test restores then.
And if you haven't tested your backups, you don't *have* backups, you have hopes.
Backups don't solve everything, you still need to setup and configure all your sites. @ Anonymous Coward, what company do you work for? I just want to make sure I stay well away from them.
Webfusion recently migrated to a new datacentre in Nov - entire nodes of VPS (including my client sites) were offline for a week. Then there were still issues about it being in the wrong container (eg running Win2003 SP1 but being placed in a Win2003 set) as there were functionality issues on the machine
This was raised for a support ticket (24/7 support my arse) and they "investigated" meaning they then knocked the server (completely inaccessible) offline for 27 days. Managed to get it online after a second migration and it's completely wiped. No configuration, no data, nothing.
As of yet, not apologies, no explanation and no compensation.
My host had a hard disk failure on my server twice in two years and had to restore from a backup - no RAID. These guys are lucky they had RAID at all, let alone RAID that has failed.
Maybe if they're doing business-class hosting, they should be using some clustering technologies underneath so that a single server doesn't cause these types of problems. N+1 or N+2 are pretty standard scenarios for business-class services.
If you're hosting in a virtualized environment, you should be using the technology correctly, including shared storage and high availability. A single server shouldn't take out your hosting environment.
Most people don't start making backups (including local copies of remotely hosted content) until they've suffered catastrophic data loss.
Then they don't start testing the integrity of their backups until they've suffered further catastrophic data loss.
Sign up, sign up for The Register's weekly mobile & wireless newsletter - click here
