The Virgin Media cable network collapsed overnight, leaving up to three million people without internet access for several hours in the crucial final shopping days before Christmas. Both broadband and digital TV services took a lie down just after 10pm on Monday. Virgin Media says "some customers nationally" were affected. VM's …
I sat there for a couple of hours last night watching my cable modem lock the up & down stream and then fail to get a reply from the Virgin/NTL DHCP servers.
hen I necked a decent bottle of red and went to bed early.
So the Mrs has the propper hump with NTL
Can you also ask them
why they haven't told the same millions of people about STM. nobody knows nothing about it and are calling the support phone number up at a cost of 25p a minute and 10p connection charge to find out why their speeds are suddenly cut!
The man in the data centre....
All down to a broken fibre (so i am told), and ntl:telewest engineers couldn't locate it all night.
Would this be the same Virgin...
...that is due to Give birth on the 25th? I thought I saw some wise men heading towards Bethlehem from the east?
Go easy on her, it can't be hard carrying the Son of God and trying to maintain the countries premiere ISP at the same time....
the slowly dying broadband
9pm: I was getting 40kbytes/sec out of my 20mbit service... typical Virgin I thought.
9:30pm: Down to 5Kbytes
10pm: Modem couldn't even lock on to the network !
I do have to wonder why I still pay them for a supposed premium service ?!?
Loss of Virgin?
Still had TV but no on demand and broadband so we went to bed early!
Worst customer service ever
I've been trying to get a problem sorted with my cable connection for over a month. I eventually gave up waiting for promised call backs that never happened. I've switch and thank GOD! I'm now with someone who actually gives a damn - I'm paying for the service I'm receiving (rather than paying for something I'm not getting). Wild horses could drag me back to virgin media - I think I'd rather stick a fork in my eye.
Its their own doing...
This happens to be caused by a routing fault which in turn is caused by the fact that NTL unqualified staff administer 90% of the network. you wont get an honest answer out of them.
Virgin does it again, eh?
Good thing that their service has been so unreliable lately that I got a backup ADSL connection. So when it died last night I still had 3MBit to play with. Shame that it had to get to this, huh?
I'm sure once their new super-duper-hyperfast-fantastic 50MBit service gets rolled out none of this will ever happen again. ;)
Nice one. You couldn't get any joy from the Virigin, so you went to bed with the wife instead - no wonder she took the hump!
Fine in Glasgow.
No probs with the Bridgeton exchange in Glasgow last night.
Service getting worse
When I took out NTL broadband almost 6 years ago it was very very fast. Even at the limited speeds of the time. Now, 6 years on, my 4MB connection is slower than my earlier 1MB connection speeds.
hope they can get it up before they take it down again
Received this yesterday from them
From time to time we need to undertake maintenance to our e-mail services
to ensure their smooth operation.
Our next scheduled maintenance work will commence from Thursday 20th
December at 22:00 and run for up to 16 hours.
During this work you will not be able to access your Virgin Media e-mail
via your e-mail software (e.g. Outlook Express) or Webmail."
so no confirmatory emails for my last-minute web shopping spree
re: The man in the data centre....
I doubt they only have one fibre that links everything together surly an ISP would have several connnections to everything
Not fine in Glasgow
I had problems in Croftfoot last night, and for once it wasn't anything to do with the neighbours in Castlemilk!
My modem has been playing up for several days now and technical support told me it was because the starting frequency of my modem was wrong. I changed it as per their instructions but it changed itself back. I did it again and it worked for a while but not through my router. I restarted the modem and everything was okay again for about two days but then last night the frequency changed itself back again. The modem log mentions something about losing DHCP.
Anyone else having similar trouble? Just wondering if "my" problem was actually their problem after all.
Edinburgh affected too
Was back on before midnight though.
Aswell as internet they also lost their customer support line, fault lines, service status and even the premium rate support line - all unreachable - giving no way to know if the problem was local or national, serious, or about to come back up any second.
I bet they are kicking themselves now - they missed out on a fortune in support line call charges last night.
Business as usual
Well at least that explains why my wife kept saying "unplug the modem and reboot it" and I kept growling back "I am, and besides whose the techie here grrrr"
I did tell her it wasn't the modem and was NT-Hell. Maybe it will fix my connection dropping in the evenings at random as its been doing for the last two weeks.
P.S. Need a Branson piccie
What they didn't say...
...is that the maintainance was probably to upgrade the network to the 50Mbit.
(by the way that was hearsay on my part, don't take it as gospel)
They also turned their phones off.
After fiddling around for while releasing & attempting to renew leases etc I thought I'd give them a call. 151 got "We are unable to connect your call" and 150 gave the number unobtainable continuous beep.
I tried 100 (from a VM landline) and got the BT operator, who said that I wasn't the first person to mention this...
no problems and a decent steady 20mbps all night long
It was exceptionally slow all evening, so I was suspicious, but thought it was just my wireless playing up. Naturally though, it went down, the router started getting a 192.168.100.10 IP and the modem logs told a tale of woe. It came back up half an hour later, though...then went down again, in a slightly more worrying fashion (the modem appeared not to operate at all - the router wasn't being assigned any IP whatsoever). Looking at the logs on the thing talks of software upgrade files and it just hanging onto the end of an unresponsive network. Not good.
If there's one thing I can say about cable, it's that when stuff goes wrong for me it does so catastrophically. The last time we had a problem was a couple of years back when some idiot with a JCB went through a rather expensive fibre somewhere in East Leeds (or so I'm told) - not exactly their fault, though it did take out TV, internet, phone...AKA everything. But as I said, not their fault.
Even this problem last night didn't really bother me, if I'm honest - as I said, it's a rare occurrence round here. What really grated with me was the fact that both the 'Service Status' and 'Faults' telephone lines returned 'Your call could not be connected' and 'Customer Services' was engaged. Yes, ENGAGED. How on earth can customer services for a multi-billion pound corporation be ENGAGED?!
Let me guess, the same people who screwed up last night are responsible for the incoming call routing as well.
Good and bad
Well, my Virgin broadband went down overnight (telly is always flakey), but it was OK this morning while I was surfing around looking for good ADSL and satellite TV packages to switch to :)
Simple fix for me...
The article mentions that an update just caused an issue with modems/stbs...
Sounds correct to me as I just rebooted the modem and all was fine, connection back up and running.
not the first time..
..Virgin/NTL do this with frequency - I was setting up a mate's a couple of weeks ago and there was nothing in the pipe, eventually rang Virgin only to be told "we are being havenning no service" which also included their tech support but no loss there I s'pose - considering that they're cr*p.
(but not as bad as Sky or as lackadaisical and uncaring as BT - but they're all in fookin' India so what do you expect?)
but where's the Paris Hilton angle?
Good excuse to read a book instead
4am in India!!
I called up to see if the IVR would tell me when I'd be likely to get back on-line. No message and 30 mins later I got through to their Indian call centre, apparently no problem.......nice!!
I then asked to speak to a supervisor, as there was obviously an issue and I was told "It's 4am in the morning therefore I'm going to struggle to find one" .............EXCELLENT customer service, not my fault it's in India!
Sorry......should I have gone back to bed and got up in the middle of the night to ring them at a decent hour!!!!!!!!!!
Yeah I'm in Edinburgh too. Around 21:30 I got kicked off World of Warcraft, and had a life for an hour. Because I'm all so Web2.0 (yeah), I connected to Virgins site via an iPhone to get a status update... which wasn't updated. The number on their site also reported "this number has not yet been activated".
Had a similar conversation with my wife...
hmm! I smell lies
"According to VM's rep, the maintenance process knocked out DHCP address assignments (leases) for some of its servers, which in turn caused IP addresses to be renewed en masse. No word yet on what the routine maintenance was."
But ive the same ip address as before..
....it's not "Virgin Media" it's "NTL"
"Most of the problems occurred in the North West, Yorkshire and the Midlands"
Hmm - well just north of Reigate it was out from around 9.30pm to midnight!
Bristol also affected
My friend and I live in different parts of Bristol, but both lost our broadband connection yesterday evening (not sure what time). The TV seemed to be fine though.
Well Virgin is anything but the Premium service provider. They now have an ace TV Channel with the junk that Sky didnt want, they have a fibre network which is run by monkeys and they have the customer service skills of a company going out of business.
Customer support open from 8am to midnight...NOT!!!
After loosing my broadband at 9.30 last night, I decided to wait a while to see if it came back on.
By 11.30, I still had no connection. So I decided to call the 151 number from my landline.
"Hello and thank you for calling Virgin Media. We are here to help you between 8am and midnight........All our line are now closed" (or words to that effect). Note that this was at 11.30!!
I was able to leave a message.
Oh, and the cable box at the end of our road is missing its lid. Has been for 2 weeks. This has been reported but no one has been out to fix it, and I can see that all the cables have already been pulled by some hooded oik! I reported it again yesterday...hmm..
I'm not complaining
So it went down for a few hours, shit happens. This is the first serious outage I've seen with them in 6 months. I didn't think it was this widespread but still. I just shrugged my shoulders and joined my girlfriend in bed. =P
So what's STM then?
Poor network design/management?
Wasn't the initial (military) idea behind network design/protocols/routing that the network (as a whole) would survive in the event of direct nuclear attacks on a routing node? I seem to remember DARPA being involved at some point. Someone remind me.
So, given there hasn't been a catastrophic nuclear attack on the VM network, is it safe to assume that it doesn't handle routing failure well?
Don't get me started on Virgin Media.
Worse thing I ever did was sign up with ADSL from Virgin Media.
Best thing I ever did was pay the £50 "exit fee" to leave and find a decent ISP.
A sense of proportion?
"Virgin Media network collapses nationwide ... leaving up to three million people without internet access for several hours in the crucial final shopping days before Christmas"
I'm still struggling to work out if El Reg thinks this was a big deal or not. No internet access! For several hours! In those CRUCIAL final shopping days before Christmas!
Nope, still can't tell, even though I'm a Virgin 'customer' myself. (And is that 'up to three million people' like Virgin's 'up to 3 Mbps' or what?) Meanwhile, in other news, thousands of people were dying. No Britons were thought to be involved. And finally, a hilarious talking dog...
Thats what happened!
Been resetting my modem and router for half an hour before I gave up.
Ah well.. shit happens.
Just the north eh?
Well in Bournemouht the "Broadband" went down aroung 9pm but came back up just after midnight and the telephone number for their status line was unobtainable. I finally got through to the status line by ringing the fault line.
Never mind I'm sure they'll reimburse us all.
I spent the time watching a recording of the Gadget Show which interestingly mentioned broadband speed again ...
Not all of Edinburgh affected
Had no problems at all, although I did notice the Online light on my V box flashing. Everything was still working OK though.
Apparently I'm one of the few who hasn't noticed any drop in service from blueyonder to Virgin. Lucky me! I still hate Branson, though.
Did I miss something?
Looking at my cable modem log it appears to have been mildly upset by something last night, including failing a software update. However, as I've had both an ADSL line and a cable line for some time, I didn't really notice anything wrong because if service did hiccup at any point it would have continued via the other link. No complaints from the TV watcher in the house either, so quite possibly we missed out.
The reason that they'r called Virgin
is that WE are the ones being screwed. :)
Virgin support fun
I tried the 150 number to be greeted with the dead line tone. Without the internet i couldn't look up the service status number so i started doing 15# random dialing.
152 - dead
154 - business support
155 -international operator "crap sorry"
Went back to 154 it asked for a pin tapped in a random number of ones, it said something else i pressed one, then it gave me the business support menu! I got through to a real person in minutes who told me all about the trouble, then i went to bed as she said no ETA!
We were down from 9:20pm till after 1am, still on the same IP making the DHCP thing seem unlikely.
Yeah, and we're still out!
Swansea went down from about 9pm to 9:30, when the whole area came back up. Then about 15 mins later our modem went down again and STILL hasn't come back up yet (12:50 the next day). Rest of the area's fine though.
Wouldn't be a problem except I'm not going to be in the house again until mid Jan and my housemates wouldn't have a clue what to do if they called CS!
re: re: The man in the data centre....
Of course they have more than one fibre link, but if a fibre to a primary POP is backhoe'd etc, then you have major issues. If said POP is a major peering point, then you have even more major issues. If your not even sure where the fibre was damaged, then your fsck'ed!
The service loss wasn't a 'no connection' problem, but a routing problem with core routers playing silly buggers because something somewhere had gone seriously wrong. We are a medium ntl:telewest customer with around 50+ links via them (ranging from 2Mb p2p circuits up to 100Mb LL connections and then up to 1Gb circuits), with even more circuits resold via us - and the explanation we got was a damaged fibre that the engineers couldn't locate.
Fine after 10 mins
My blueyonder cut out around 9:30, ten mins later it was back up and running fine :)
TV on Demand was flaky all day which is annoying when you want to watch Peep Show.
All ok now thou :)
That explains it...
I had wondered why I lost my precious connection to the world last night -- kept trying until 03:00 and finally gave up.
Spare a though for those of us who can't even threaten to leave Beardy Cable because our phone lines were last upgraded before the war, so ADSL providers can only give dial-up speeds. Seems BT want us to use some other phone provider and ISP.
More of the same...
I've been an NTL customer for about 6 years now... only a few minor teething problems early on.... and a good service for most of it...
Now, since VM are on the case... things are going steadily downhill... Not only did I lose my broadband last night for several hours (trust me... I was trying to get back online well after midnight!), but that was the second time in 3 days that I had lost service for an extended period.
As I work from home... this costs me an awful lot of money...
At least the other day, I managed to get through to one of the script readers in India... last night the broadband support number was out of service, and 150 just resulted in an engaged tone...
I finally got through to someone at about 1am who advised me to wait an hour, and if the connection hadn't re-established itself in that time... to unplug the modem for a couple of hours, and try again :(
Fortunately, it was back up and running this morning, so I didn't have to do the three hour drive to the office to get some work done.... but I am now on the lookout for a reliable provider, as this is now getting ridiculous.