Engineers at 3 are busy rebuilding the firm's pay-as-you-go top-up database after the server carrying it fell over on Sunday. Customers were left unable to add new credit for several hours, either over the phone, online or in 3 stores. A 3 spokesman said today that it took action to ensure that pay-as-you-go users were able to …
if there was no service in three stores... you could like go to the fourth one
They should get this...
Try using Vodafone. Their top up system crashes all too frequently, often meaning you can't credit through your phone or the internet.
Cluster, failover, backup
They never heard of these?
It's still down
Still not fixed! a few hours time down? I couldn't activate my new broadband dongle for 2 - 3 days.
my3.three.co.uk is down too, three really do suck.
Paris cos she likes to suck too.
I realise that timing is of the essence [and I only top up my ChavAsYouGo phone once every couple of weeks] but I can't remember the last time I had the Voda top up thingy fail on me.
What really, really, pisses me off tho, is that while you can interrupt the prompts for most of the menus to get to your top up - that is dial 1345, 1, 2, 1, [details] [amount] - when you get to the confirmation you have to wait right until the very end of the spiel before pressing 1 to confirm that yes, after pressing about 20 buttons, I DO know how much I want to top up, from what card.
Anyway, rant over, and am I right for htinking that 3 suck noodle for not having some sort of hot/immediate redundancy in place for their top up systems??
"I'm sitting on £50 credit. It's inexcusable,"
What is inexcusable is lending Li Ka Shing £50 for no interest - no wonder he's rich. never leave more than £20 on a pre-pay phone - what happens if you lose it?
Paris, because even she is not that stupid
My mobile internet
went down on3 for a couple of hours last night too. Was a pain but I don't recall it happening before.
Anybody heard of this little thing called resilience nowadays!?
I was in awe of the first paragraph - ONE server for how many chavs I MEAN respectable customers!?
I mean in our lowly establishment, even we realise the importance of a simple failover or load balancing arrangement, so much so that we'd have our balls in a sling if we implemented a new system without it.
As an absolute minimum, if it were a system so badly designed that it couldn't be made resilient, we would recommend alternative products wherever possible, or put in place some kind of manual quick-restore procedure, such as standby hardware ready to accept one half of a Raid array for restoration.
I despair. Wasn't there mention of some kind of computing skills/systems design crisis back in the 80s? Its far worse now due to computing ubiquity.
Three is the fashionable one, fashion always changes.. someone pressed a button.. oops!
Use Orange or O2, atleast when they go down on you it feels good!
Tux, coz he never went down on me yet! :P
"went down on3 for a couple of hours last night too. Was a pain but I don't recall it happening before." This has so much comedy potential....
They do have resilient systems
I think some spokesman has over simplified the reason behind the problems. As an ex-3 employee, all their databases have resiliency, backups, etc... Even the lowliest database I was aware of was pretty well protected.
There are other problems though. For example:
With big 24*7 databases, there is a significant delay in recovering terabytes of data from tapes, particularly when everyone else still needs to be backed up at the same time. Maybe a tape has gone to offsite storage already. That imposes a nasty delay, even though the tape management company say they can get it back in half a shake, it always takes 3 days to get the wrong tape out.
There is probably a bigger delay in managing the "paperwork" behind such a massive restore, raising change reqs, filling in fault reports, keeping the problem management people off your back, responding to questions from popular websites and spokespeople, etc...
The biggest problem is : once you've got an HA database that's being used, how do you make sure that any related changes don't break the HA bit, without risking breaking the live service. Managers are always much more prepared to take the risk that if something went wrong 'one day' the system would fail than to say today "go ahead, see if it fails on purpose"
"Can't connect securely to my3.three.co.uk because the site uses an older, insecure version of the SSL protocol"
It's STILL down
So it's now 2 days later, and I still cant top up. How long can it take to sort out these problems? Was thinking about a move away from 3, this has made up my mind.
Paris, because she also goes down and stays down
And T-Mobile payments are down as I read this
Is T-Mobile out in sympathy? I can't use the payment system for either PaYG or contract.