ClaraNet email falls over
ClaraNet customers have been without email access all day, after the business ISP's POP3 system took a lie down this morning. Engineers are battling to fix the snafu, which struck just after 9 this morning. ClaraNet said it hoped to have a partial fix in place by 6pm this evening. "It will not however be fully functioning with …
"We couldn't get hold of ClaraNet to find out what went wrong."
Did you try emailing them?
We can do it in about 15 minutes. Allow an hour to integrate it with our user accounts system.
Christ - even if it's not a VM (which takes 20 minutes to restore back exactly as it was before without configuration) you can get an Exchange box up in well under an hour if using a deployment system - which one would hope as an ISP.
Even manually creating a server it shouldn't take more than a few hours....
At least for me. It was down until around 4:00pm.
I have been with ClaraNet for over 8, yes eight years. This is the first time I have had a problem accessing email. And during that 8+ years the number of times I have been without Internet access must be in single figures.
Expensive they maybe, but they are reliable and when they quote 8Mb bandwidth/download speeds they actually mean it, and it does not depend on the time of day either, in addition there does not appear to be any port blocking or throttling, at least I haven't noticed.
Yes taking several hours to get a pop3 server back up and running does seem a long time. Despite this they are way ahead of other ISP's in so many ways.
It would take more than a few hours of email sever down time for me to class ClaraNet's service as anything other than excellent. I wouldn't give an ISP 8 years of loyalty if they were anything less.
6pm comes and goes and nope, it's still not working! Good job they allow you set set your own MX records now!
try the 3rd floor in [their London DC] they usually can be found making a coffee ... oh, and stealing other people's sugar ;-)
Paris, as that's who should be running the noc
Someone should explain the concept of redundancy to ClaraNet.
[1] Apologies to Weller, Foxton, and Buckler :-)
I've had top top service from Claranet for years - these guys know what they're doing, so I imagine they've had more than a simple rebuild to deal with.
Not a criticism, as I say, I really rate them - but 12 hours without a fix doesn't sound good.
Give it time. If it's urgent, they'll be back to you in about a week.
(ClaraNet business customer)
just a thought but maybe as an ISP their systems are slightly more complicated than that 'server' you built in your mum's basement
I am a Claranet customer, have been for eleven years, and the only other interruption to service I have suffered on this scale in that time was when BT-operated facility in Ilford caught fire, and broke an awful lot of wholesale ADSL, not just Claranet's.
This is/was not rebuild a POP server. The error message displayed on the webmail login page for a time was: "We are currently experiencing service problems on the Claranet E-mail platform. Claranet Engineers are currently working with our Storage Vendor to rectify the situation.We regret any inconvenience this may causing you."
Claranet, between the business services and the retail services and the services shared with other ISPs they have bought or with which they have partnered, is pretty big. A lot of their stuff used to run on FreeBSD on SPARC hardware; I don't know how much of what still does. It always appeared to me, based on there being multiple public server names that one could use to configure sending or retrieving mail, that there were actual different internal servers providing redundancy and they were all capable of reading to or writing from the same disk files for serverside mail filters, mail filter logs, POP3 mailboxes, and so on -- presumably these files were in some way on a network share, which is not a configuration I'd have chosen, but then what do I know, never having had to spec up a mail handling system even a thousandth the size of this one.
"currently working with our Storage Vendor" I'm going to take a punt: that is to say, the controller card for the RAID went bang, the guy from Sun turned up within x hours as per the Gold/Enterprise/WeLikeJonathansPonytail -level support contract, and yet somehow, despite the customer having the biggest best support contract available, despite knowing what hardware the customer had, despite being told by the customer what component on what system had failed, the guy from Sun did not have the required part on the van.
The interruption to service was a nuisance. As best I can tell no incoming email was lost, and I (and every Claranet customer I know personally) had workable alternative provision for sending and receiving email. The biggest bitch I have right now is that the serverside mail filters aren't available and so 90% of the spam I get sent ought to be deleted before it hits my POP3 mailbox, but currently it's being delivered instead. Anybody want a stack of pointless emails from the likes of sue@casimme.co.uk punting ' ...Business seminar informmation'?
You mean like making their support staff redundant?
Some people need to learn that resiliency is measured in "number of 9s" for a reason... no-one who's worked in the industry will quote 100% uptime because there's always something that can happen, so lets do the maths:
The poster above says he's been with them for 8 years with no problems, so that's 2922 days
which is 70128 hours
They've now had 12 hours of downtime
12/70128=0.00017112
so 99.983% uptime - that's 3 nines.
Respectable, not exceptional but not too bad.
3 nines is all most people aim for, whilst hoping for more... some people/services aim for 5 nines, which in the same 8 years would equate to 4 minutes of downtime!
Paris, because I'd give her some uptime (maybe just one 9 though).
"Some people need to learn that resiliency is measured in "number of 9s" for a reason... no-one who's worked in the industry will quote 100% uptime because there's always something that can happen, so lets do the maths:"
Yeah, yeah, yeah. It's part of what I do for a living, FFS. My own, personal email system has been up and available, non stop, for over a quarter century. Granted, I'm not one of the big-boys ... this is a personal research system, initially built to look into RFC-821 in early '83ish, now used as the family's email, FTP and WWW services. The concepts scale. I have consulted for Fortune 150s, the ones who used my advice should experience similar uptimes.
Note that I said SYSTEM. This isn't a single box located in mommy's basement, and it never was. It started at Stanford, MAE West, and under Bryant Street in Palo Alto. Now I have nodes in three countries. This is a geographically diverse, multihomed, multiple box, multiple OS, multiple MTA system. Redundancy is designed in everywhere I can fit it. Over-kill for a simple family server system? Of course. But as a research platform, it's mostly tax deductible.
Anyone who can ID me from the above, please keep it to yourself ... but feel free to give me a call if you want to get back in touch. I'm in the book, in Sonoma CA.
(I got calls from two old friends last time I posted the above offer ... Thanks, El Reg! :-)
"Someone should explain the concept of redundancy to ClaraNet."
I've got personal experience that they know the concept of redundancy VERY well
Claranet's mail system is just a *leedle* bit more complex than an exchange server ;) My guess is that it was the storage platform that caused the issue, and rebuilding a filesystem on a storage platform that size might well take several hours.
What irritates me is the weak explanation of what is going on. They didn't have a PR available to the journalist on the Register to explain ["We couldn't get hold of ClaraNet to find out what went wrong"], their Service Announcement is limited and unclear and does not inspire that they are in control of this problem and earlier they made promises that they did not keep nor did they reply to me from support with anything more than an AutoReply that said nothing - not even a holding standard reply. At one point the web-link to support appeared to fail. These guys are communications clowns.
I appreciate accidents happen but this does not fill me with confidence and a failure to deliver a week of archived material on one email (which I look at weekly on PC and delete on blackberry on a daily basis, relying on 'hard' copies later) will mean I go elsewhere - losing a week of communications is a serious matter if it gets that far. Thanks to Facebook, I am getting a flood of recommendations on alternative suppliers and, I hope, that when I check in in the morning (it is UK time) I will find everything in order or the search starts on Monday.
We need an explanation from the company of who screwed up and why, with assurances that it won't happen again ... no assurances and we and others go.
I had several emails prior to 3/4/09 in my Inbox - they are now gone! They did manage to recover my Sent Folder - but that's all!
I expect losses on a free email service - not on one I pay for!
Eveb before their massive collapse last week, their email service had been abysmal at times - slow or non-existant! And I still got the occasional rogue empty email thast I could never delete!
Just why am I paying for this? And not even an acknowlegement from them to me as a customer!
Utter crap !! Thank god I still have several free email accounts to depend upon!
Sign up, sign up for The Register's weekly mobile & wireless newsletter - click here
