back to article UK biz ISP Entanet goes titsup, 'broke' a bit of Blighty's internet

Brit internet and communications provider Entanet is slowing bringing its systems back to life today after they metaphorically keeled over last night. The Shropshire-based supplier of broadband, leased lines, telecoms and more has offered scant detail about its network outage, much to the chagrin of some of its biz and ISP …

COMMENTS

This topic is closed for new posts.
  1. Maverick
    FAIL

    still happening @ 11:30

    up & down like a whore's knickers this morning

    mainly authentication errors

    couldn't talk to my ISP as they are clearly using ip telephony :(

  2. Joe Montana
    FAIL

    Enta outage

    I have a DSL line from Enta, and i lost ipv4 connectivity but ipv6 was unaffected for most of the time... When ipv4 came back online later in the evening, ipv6 dropped out although both seem to be working ok this morning.

    1. Marco Fontani

      Re: Enta outage

      Could you tell me which brand adsl modem do you have that works _properly_ with Enta's ipv6 allocation? It's manual (gave me a /56 and a gateway) and not DHCPv6, and my self-described "IPv6 capable" Zoom ADSL router/modem unfortunately (and after several chats & phone calls with Zoom) only supports DHCPv6 properly (allegedly). Any pointer - any - would be appreciated! my surname at gmail if you could :)

      1. alain williams Silver badge

        Re: Enta outage

        I have a router from these people http://www.mikrotik.com/ - handles IPv6 well. A few small bugs but OK. Setting it up takes a bit of work to get right.

      2. Stephen 11

        Re: Enta outage

        The router will need to support IPv6 Prefix delegation to work with Enta. Unfortunately, there are a few companies, including Cisco, that advertise some of their products as being IPv6 compatible, when they are not fully compatible. IOS routers are fully compatible, however their non-IOS devices are a bit of a crapshoot. As far as the modem itself goes, the draytek vigor 120 should work with IPv6. I was unable to test IPv6 with it, as the router I had didn't support IPv6 PD.

        1. John Sager

          Re: Enta outage

          I've used a D-Link 300T modem in bridge mode successfully with Enta IPv6 to my own linux-based router. I've also successfully used a Draytek 120 in the same setup, though I'm back on the D-Link as the Draytek has become unreliable (either bad capacitors or a bad smps, probably).

  3. Anonymous Coward
    Facepalm

    Its just an ISP

    This happens, get over it. Its just the internet, not life and death. If its that important have a backup ISP as some clearly have or they wouldn't have got onto Twitter.

    1. Anonymous Coward
      Anonymous Coward

      Re: Its just an ISP

      From what we saw last night the network was giving over 90% packet loss but didn't actually flatline. So auto failover to another provider didn't kick in and had to be done manually. All of which takes time to sort out.

      Oh and they aren't "just an ISP" these guys sell data bandwidth in many forms as well as other "telco" services including colo. I can't see any of their corporates being too happy if they lost their private networks or access to their colo.

      As for twitter, I would guess that people were probably taking to the 3G networks to complain.

      1. Marco Fontani

        Re: Its just an ISP

        Here in fancy Edinburgh, their graph (http://noc.enta.net/21cn-interconnect-status/) was at 0 flat.

        In other places it had a bit of connectivity, just that tiny bit... here, nada. Zero. Zilch.

        Had to try phoning their support more than 30 times to even get to receive the pre-recorded message saying there was a known fault and they were investigating. All the other calls either mysteriously disconnected or gave me the weird "There's been a fault" message.

        Luckily it all came back ~8:20pm.

      2. Anonymous Coward
        Thumb Down

        Re: Its just an ISP

        "I can't see any of their corporates being too happy if they lost their private networks or access to their colo."

        If they want to bet their company on a small ISP thats up to them. If it was me I'd stick to one of the major backbone operators for my access. As for co-location, frankly, if you can't be bothered to maintain your own systems then you deserve what you get.

    2. Lee D Silver badge

      Re: Its just an ISP

      Leased lines and colos - not just a consumer ISP, thus the whinging.

      And, also, they didn't just fall over - by the sound of it they were advertising routes they had no reason to advertise and the whole internet tried to send them traffic. Similar to the India YouTube thing that happened earlier - basically your traffic may have been routed through Entanet even if you've got nothing to do with them - not only while they were down, but actually being the CAUSE of them being down too.

      It's like a guy with an Ethernet cable telling the world "Hey, you can all pass all your traffic through me!", except that they are big enough and trusted enough to not do stupid things like that.

    3. Captain Scarlet

      Re: Its just an ISP

      Entanet provide some routing services for BT's 21CN as well as having lots of resellers.

      The effects would have affected a lot of people.

  4. This post has been deleted by its author

  5. Anonymous Coward
    Anonymous Coward

    Hahaha

    Couldn't happen to a nicer company.

  6. Anonymous Coward
    Anonymous Coward

    incident report

    The incident report is available, seems internal stress testing of new kit leaked out of their office and broke their network.

    http://noc.enta.net/wp-content/uploads/2013/03/Incident-Report-21032013.pdf

    1. Marco Fontani

      Re: incident report

      Situation Normal, all.. Failing Under Constant Tests Under Pressure?

    2. Anonymous Coward
      Anonymous Coward

      Re: incident report

      Hmm were they doing some sort of stress/failure test when Chernobyl blew up?

  7. Dave Clarke 5

    Incident report is now available on the Enta noc site. Ooops!! is all I can say!

  8. This post has been deleted by its author

  9. Anonymous Coward
    Anonymous Coward

    "Between approximately 18.30"

    Usually the construction 'between' follows a format where there is a lower bound and an upper bound...

  10. James 100

    Double-fault?

    Nice to be quoted in a Reg article there*! From their later updates, it seems the original problem was excess route entries overflowing the TCAM (fast lookup memory in big routers), which makes them fall back to a much much slower routing approach - which can't keep up with the level of traffic you see on an ISP backbone.

    That was problem #1: the lab stress-test leaked out and flooded the live network, bringing it to its knees. (IPv6 itself was unaffected - it's routed separately anyway - but since the line is authenticated over IPv4, as soon as you disconnect, your authentication packets get lost, stopping the PPP link coming back up.) Like their autopsy says, filtering out the routes then re-booting the routers cleared the problem - then they discovered a Cisco line card was still misbehaving.

    The wonky line card seems to have been problem #2: it seems to be working again now, but they're keeping most traffic away from that card until Cisco can figure out why it misbehaved in the first place.

    * Apparently the router problem took out their VoIP lines and access to their own status system - downside to "eat your own dogfood" as an ISP, when your own services are down, you can't use them to communicate about the problem - normally, I've found Entanet to be very helpful and responsive. In this case, the live traffic graphs on noc.enta.net were enough to tell me it wasn't just my line that was dead - how many other ISPs give you that kind of detail?

This topic is closed for new posts.

Other stories you might like