back to article O2 coughs to data failure

O2 has admitted a major failure of its data network yesterday, attributing the problem to dodgy DHCP service that wasn't giving punters a proper IP address during the connection process. The problem didn't hit every O2 customer, but many were affected. O2 assures us that almost everyone is back on the net now, and that …

COMMENTS

This topic is closed for new posts.
  1. lansalot

    err

    Are they saying they run the entire data network with a single-point-of-failure (in this case, a single DHCP server causing problems) or that they - perhaps - rolled out some change to multiple systems and that caused the DHCP failures ?

    Seems like either poor DR planning, or poor change-control / testing to me ?

  2. Anonymous Coward
    FAIL

    Pardon?

    "it's not as though mobile data is mission critical. ®"

    I'd suggest that if it forms part of a paid service with contractual commitment then it is very critical indeed (or is it just another example of screw the customer?)

  3. Mage Silver badge
    Alert

    Not mission Critical?

    Mobile is NOT broadband. Yet the Irish (ComReg) figures include all most 1/3rd "Fixed Broadband" connections are 3G.

    Someone should tell Eamonn Ryan.

  4. Sordid Details
    FAIL

    DHCP?

    My iPhone was muttering something about being unable to authenticate the PDP context. That sounds more like a RADIUS issue to me.

  5. McFlurry
    Paris Hilton

    so is...

    ...this related to the "permanant-service-denied" message I received from an unknown number? trawling fanboi forums seems to suggest so.

    Because she's pretty to look at but fairly useless like my iPhone was yesterday.

  6. alex_c
    WTF?

    @ AC (22nd July 2009 12:32 GMT)

    nnnnnnnyyyyyyyyyyyyyesssss.... Not mission Critical? I think there might have been a degree of sarcasm implied in the context!!

    I mean, what use is the "resurrection phone" without a data connection?

  7. nichomach
    FAIL

    Frankly, we're beyond caring

    O2's been so unutterably poor that we're ditching them for our last few guys with the misfortune to be on them for mobile broadband. Plus they seem to break or manage to death our VPN connections.

  8. Anonymous Coward
    Alert

    Can anyone help me out here?

    Ok so my non-tech neighbor is having the exact same problem on BT. All of a sudden last week BTs DHCP started issuing him a 169. IP address, where the normal from them is a 168., thus he cannot get a connection to the web.

    We thought it was his router so I reset it and then went in with my own laptop through his router and connected fine. But his laptop still could not connect. I checked all settings and tried multiple release/renew under different configs, then checked hardware profiles and no network Network Adapter faults of any kind. Then I disabled the firewall and ad/virus protection and did more release/renew (turned it back on after that did not help). Then I tried to connect his machine through MY router/connection and we got the same result.

    I am still thinking that BT is issuing his MAC with a dodgy IP address but they deny its their problem and want to charge some ridiculously high price to send someone out to fix the it (more than the laptop is worth in my opinion). Is there anything else I can check to (hopefully) prove me wrong?

  9. Phil Edwards
    WTF?

    What's that I can smell...?

    I'm intrigued by the phrase "wasn't giving punters a proper IP address"? Was it only giving the first 3 octets? Maybe it was assigning IP addresses in base13 or something.

    It's brown, it comes out of cows backwards, and it aint the Isle of Wight ferry!

  10. Annihilator
    Thumb Down

    Re: Pardon?

    "I'd suggest that if it forms part of a paid service with contractual commitment then it is very critical indeed (or is it just another example of screw the customer?)"

    If it was part of a business service with KPIs, SLAs etc then I'd agree. As it is, you've generally got a consumer package with bugger all. If your DSL connection disappears for a while, you're generally stuffed unless it's more than 5 days and even then all you'd get is a refund on the days.

    I you're relying on a third party (where you have no real partner agreement) for mission critical stuff, then I'd suggest applying that "FAIL" logo to yourself and disable any accounts you have on mission critical systems. You're clearly not up to looking after them.

    El Reg is spot on.

  11. Bassey
    Thumb Down

    Re: lansalot

    > Are they saying they run the entire data network with a single-point-of-failure?

    No. The article clearly states that not everyone was effected and that you could try one of their other APNs.

    What's wrong? Did you get tired after reading the first couple of lines so just made up the rest of the article in your head?

  12. Cameron Colley

    RE: Can anyone help me out here?

    Either you are missing out vital parts of the troubleshooting process or you're not so techie yourself.

    I'll give you a clue though: BT can't assign his laptop an IP address if they can't see it.

  13. BeefStirFry
    FAIL

    MMS...

    Is still not working...

  14. Anonymous Coward
    Anonymous Coward

    @Anonymous Coward

    'Ok so my non-tech neighbor is having the exact same problem on BT. All of a sudden last week BTs DHCP started issuing him a 169. IP address, where the normal from them is a 168., thus he cannot get a connection to the web.'

    169.x.y.z numbers are allocated to machines when they don't get an IP number from the DHCP server. It's a network issue. Have you tried another machine through the same connection in case there's a network adaptor fault in the machine? If you've tried more than one machine and more than one router it sounds like BT is at fault.

  15. David Neil
    Unhappy

    It's a dead machine

    'Ok so my non-tech neighbor is having the exact same problem on BT. All of a sudden last week BTs DHCP started issuing him a 169. IP address, where the normal from them is a 168., thus he cannot get a connection to the web.'

    Not cleear whether it's wireless or wired, or indeed a laptop or a desktop.

    If it's a laptop, then unless it's one with a seperate wireless cardthen it's dead as disco.

    If it's a desktop, try another NIC in it.

    No charge, and I didnt even look at your beach photo's :)

  16. James O'Shea
    Headmaster

    @Mike Richards

    It's not 169.x.x.y numbers in general, it's 169.254.x.y numbers. See further <http://compnetworking.about.com/cs/protocolsdhcp/g/bldef_apipa.htm>.

  17. Michael Beck
    Stop

    @Mike Richards

    "169.x.y.z numbers are allocated to machines when they don't get an IP number from the DHCP server. It's a network issue. Have you tried another machine through the same connection in case there's a network adaptor fault in the machine? If you've tried more than one machine and more than one router it sounds like BT is at fault."

    Nnnooo.....Yes to the first part, the system isn't getting an IP address from the DHCP server, but given that the poster mentioned that it didn't get one when they tried to connect it through a different router, I'd suggest that there's a hardware fault, even though the OS isn't reporting any hardware faults. We had one user who'd spent ages doing all sorts of things trying to get their network connection sorted out and we fixed it in 30 seconds by setting the wireless switch on the laptop to "ON".

    I don't see how you can get from the information supplied that the fault lies with BT. They'd on'y supply the network address to the local router surely?

  18. Dave Harris

    re 169. networks

    OK, so not on a UK provider, but my local frequently has wifi problems where everyone gets assigned a 169 address, hence can't do anything.

    Any ideas, since a reboot of anyy or all local systems (ie, the pub's) doesn't always solve the problem?

  19. Anonymous Coward
    FAIL

    Previous O2 network cock-up

    I remember working at the O2 Slough office once. Somebody plugged in a wireless router from home on thier network thinking it would give out a wireless connection for thier laptop. What it actually did, because it been configured incorrectly was act as a DHCP server for the entire office, creating dodgy IP addresses on all machines connected to the office network. Nobody got any work done all day until they finally discovered it sitting on a desk.

  20. Anonymous Coward
    Anonymous Coward

    MMS Problems

    I don't think the data connections were they're only problem...

    I had a call from someone I don't know asking why I sent them an mms... Did a quick search on the o2 customer forums for mms and turns out people are getting mms messages but somewhere along the way, the senders number is getting mixed up randomly...

    Still waiting for an answer from o2 on the issue, told it'll be within 5 working days.

  21. Anonymous Coward
    Pint

    Welcome to the El Reg Helpdesk...

    1st off - I have mission critical O2 broadband and it's only just come back after 2 days. At the moment it's like wading through treacle to try and surf.

    2nd off - does the machine have an IP Gateway address when you do an IPCONFIG. If yes, ping it -it should be your home router. If no then I'd suggest checking your cable/wireless/interface.

  22. steogede

    Re: Can anyone help me out hear

    I don't really want to turn this thread into a help forum, but I can't resist posting a reply.

    IP address which start 169.254 are link-local addresses. They aren't assigned by your neighbours home router or ISP, they are chosen by his PC. It decides on a number, broadcasts it on the local network to see if any one else is already using it, if not it sticks with that address. Typically it will do this if it can't see a DHCP server and it is configured to use an 'automatic address'.

    If your laptop connects through his router fine and there is no problem at BT's end. The problem either lies with his PC or his router configuration (though this is less likely). Perhaps he typed his wifi key wrong and it can't connect to the router to get a DHCP response (typing a wifi key wrong won't always report an error). Perhaps his router's security settings are setup to only allow access from certain MAC addresses and his isn't recognised (for whatever reason). Perhaps he is running Windows (particulary Vista) and his winsock catalog or TCP stack has been corrupted (http://forums.techguy.org/networking/805788-ip-keeps-defaulting-169-address.html). Or ofcourse, it could be a hardware fault, probably with the PC's network interface rather than the router. There are numerous points of failure on a home LAN - I wouldn't put my money on any single one of the above, as they are all fairly likely - you might even be lucky and have multiple problems -one thing that can be ruled out is BT's side of the network, if you truly managed to connect via his router.

    With regards to O2, it seems the problem passed my iPhone by.

  23. Andy 97
    FAIL

    Seems like the bubble 'popped'

    Fancy not building an a +1 layer into their infrastructure.

    Accountants..... pah!

  24. Dangerous Dave
    FAIL

    eh?

    i called up yesterday to find out why the hell my 3G wasnt working and they told me there had been a fire at one of their sites and it had taken the entire data network down. They said it could be down for days at worst

    something stinks and its not just O2's customer service

  25. wildmonkeyuk
    Paris Hilton

    Whiskey Tango Foxtrot

    Hmm....

    The missus just got one today and guess what - it will not load up and get on so she can purchase any of the mobile tarrifs.

    Paris - Why it's simple. Even she isn't able to suck for this long :(

This topic is closed for new posts.

Other stories you might like