back to article 3 pre-pay database server has a Sunday snooze

Engineers at 3 are busy rebuilding the firm's pay-as-you-go top-up database after the server carrying it fell over on Sunday. Customers were left unable to add new credit for several hours, either over the phone, online or in 3 stores. A 3 spokesman said today that it took action to ensure that pay-as-you-go users were able to …

COMMENTS

This topic is closed for new posts.
  1. Anonymous Coward
    Joke

    who cares...

    if there was no service in three stores... you could like go to the fourth one

  2. Kenny Millar
    Thumb Up

    They should get this...

    www.neverfail.com

  3. Alexis Vallance
    Flame

    And Vodafone

    Try using Vodafone. Their top up system crashes all too frequently, often meaning you can't credit through your phone or the internet.

  4. Andre
    IT Angle

    Cluster, failover, backup

    They never heard of these?

  5. Anonymous Coward
    Paris Hilton

    It's still down

    Still not fixed! a few hours time down? I couldn't activate my new broadband dongle for 2 - 3 days.

    my3.three.co.uk is down too, three really do suck.

    Paris cos she likes to suck too.

  6. Steven Raith
    Pirate

    @alexis

    I realise that timing is of the essence [and I only top up my ChavAsYouGo phone once every couple of weeks] but I can't remember the last time I had the Voda top up thingy fail on me.

    What really, really, pisses me off tho, is that while you can interrupt the prompts for most of the menus to get to your top up - that is dial 1345, 1, 2, 1, [details] [amount] - when you get to the confirmation you have to wait right until the very end of the spiel before pressing 1 to confirm that yes, after pressing about 20 buttons, I DO know how much I want to top up, from what card.

    Fucking annoying.

    Anyway, rant over, and am I right for htinking that 3 suck noodle for not having some sort of hot/immediate redundancy in place for their top up systems??

    Steven R

  7. Pie Eater
    Paris Hilton

    "I'm sitting on £50 credit. It's inexcusable,"

    What is inexcusable is lending Li Ka Shing £50 for no interest - no wonder he's rich. never leave more than £20 on a pre-pay phone - what happens if you lose it?

    Paris, because even she is not that stupid

  8. Andy Hards
    Unhappy

    My mobile internet

    went down on3 for a couple of hours last night too. Was a pain but I don't recall it happening before.

  9. Stu
    IT Angle

    resilience!?

    Anybody heard of this little thing called resilience nowadays!?

    I was in awe of the first paragraph - ONE server for how many chavs I MEAN respectable customers!?

    I mean in our lowly establishment, even we realise the importance of a simple failover or load balancing arrangement, so much so that we'd have our balls in a sling if we implemented a new system without it.

    As an absolute minimum, if it were a system so badly designed that it couldn't be made resilient, we would recommend alternative products wherever possible, or put in place some kind of manual quick-restore procedure, such as standby hardware ready to accept one half of a Raid array for restoration.

    I despair. Wasn't there mention of some kind of computing skills/systems design crisis back in the 80s? Its far worse now due to computing ubiquity.

  10. nutellajunkie

    oh boohoo

    Three is the fashionable one, fashion always changes.. someone pressed a button.. oops!

    Use Orange or O2, atleast when they go down on you it feels good!

    Tux, coz he never went down on me yet! :P

  11. Stephen Gray
    Joke

    @Andy Hards

    "went down on3 for a couple of hours last night too. Was a pain but I don't recall it happening before." This has so much comedy potential....

  12. Anonymous Coward
    Anonymous Coward

    They do have resilient systems

    I think some spokesman has over simplified the reason behind the problems. As an ex-3 employee, all their databases have resiliency, backups, etc... Even the lowliest database I was aware of was pretty well protected.

    There are other problems though. For example:

    With big 24*7 databases, there is a significant delay in recovering terabytes of data from tapes, particularly when everyone else still needs to be backed up at the same time. Maybe a tape has gone to offsite storage already. That imposes a nasty delay, even though the tape management company say they can get it back in half a shake, it always takes 3 days to get the wrong tape out.

    There is probably a bigger delay in managing the "paperwork" behind such a massive restore, raising change reqs, filling in fault reports, keeping the problem management people off your back, responding to questions from popular websites and spokespeople, etc...

    The biggest problem is : once you've got an HA database that's being used, how do you make sure that any related changes don't break the HA bit, without risking breaking the live service. Managers are always much more prepared to take the risk that if something went wrong 'one day' the system would fail than to say today "go ahead, see if it fails on purpose"

  13. Marcus
    Alert

    my3.three.co.uk

    Error:

    "Can't connect securely to my3.three.co.uk because the site uses an older, insecure version of the SSL protocol"

    !!!!

  14. David
    Paris Hilton

    It's STILL down

    So it's now 2 days later, and I still cant top up. How long can it take to sort out these problems? Was thinking about a move away from 3, this has made up my mind.

    Paris, because she also goes down and stays down

  15. David Beck
    Unhappy

    And T-Mobile payments are down as I read this

    Is T-Mobile out in sympathy? I can't use the payment system for either PaYG or contract.

    MrBeck

This topic is closed for new posts.

Other stories you might like