Feeds

back to article BT does Italian Job on London traffic lights

BT is still working to restore access to thousands of people and businesses in east London left offline when a tunnel borer cut through fibre cables and copper wire. The problem is also preventing Transport for London from managing its traffic lights. Contractors working on the Olympic site in Stratford sent a large thrust borer …

COMMENTS

This topic is closed for new posts.

Page:

Black Helicopters

doh

ah oops...

you would think 32m below the ground that things would be safe...

0
0
Coat

Jobserve down?

Productivity up?

0
0
Unhappy

which contractor?

... er it wasn't me mate.

(that is the answer we always got when a hotel was being built next door to our offices and our plusnet link was cut 7 times in a 14 month project).

... or alternatively maybe its down to the fact that in Poland they don't have fibre 32 metres below the surface, so using a boring machine (intentional mis-spelling) is just that... pay monkeys etc.

0
0
Thumb Up

Fair do's

"So far we've managed to restore services to 50,000 homes and business. Work is continuing around the clock to restore service to the remaining affected customers."

That sounds like pretty bloody good progress to me. Cor blimey... Fixing a chopped fibre, doesn't sound like fun to me! I mean, which fibre connects to which? It's not like they're colour coded!

On you go BT.

0
0

How deep??

This morning the tunnel was 34 feet down. Now it's 34 metres. If you're confused about the depth, maybe it's not totally surprising that the diggers were too?

0
0
Stop

Brilliant

You do have to wonder if all these professional building contractors and companies can actually read maps and the such to plan these things! Or do they just go, "This spot looks good...CRUNCH!"

I do hope that the bill for all the work and all the loss of business and problems etc goes to the Olympic Commitee and the contractors. Teach them to do things properly!

0
0
Thumb Down

Nice job - made my morning hell!

Logging calls left right and centre for sites down on our WAN - both of our WAN providers affected!

0
0

Single point of failure?

No we don't have any of them - oh well just a few in places that could never get damaged.......

looks like we need a proper 20CN before a 21CN.

Shades of the manchester tunnel fire again from our wonderful net(un)work provider.

0
0

huh?

"... or alternatively maybe its down to the fact that in Poland they don't have fibre 32 metres below the surface, so using a boring machine (intentional mis-spelling) is just that... pay monkeys etc." Mis-spelling? What?

0
0
Stop

@jermey

believe me we do have fibre in poland...maybe it was another example briliance of British project management..exactly the same as Terminal5 and new Wembley stadium

0
0

Sarcasm Alert!!

"On you go BT."

At least I hope you're being sarcastic!

It took the useless fuckers almost a month to repair the damaged multicore to the pole serving our house after some dozy smegger cut it with a digger.

As someone posted on the earlier article, BT need to get their act together on the diversity and redundancy factors.

With some added redundancy for those responsible for not ensuring there was enough of both in the system.

Not so much a network as a few bits of damp string knotted together incoherantly

Twonks.

0
0
Pirate

Interesting

Interesting to know that the system has no redundancy. It makes it that much easier to cause mischief.

0
0
Thumb Down

Typical contractor and oh so typical BT!

Can these contractors even read a plan? or was the tunnel not marked on it?

Why did this happen? Why did this single incident cause so many people and businesses to lose their internet connection? where's the backup?

This whole incident just goes to show how BT have no viable backup infrastructure for when things go wrong. It's about time BT stopped looking at it's profits and invested in this countries quite frankly shocking infrastructure.

0
0
Bronze badge
Paris Hilton

Oh, you kill me....

......sent a large thrust borer through a deep level tunnel on Saturday afternoon...

Guess which Icon? Yep, the coffee-stained keyboard, 'natch.

0
0
Happy

@ Mark Wills

"That sounds like pretty bloody good progress to me. Cor blimey... Fixing a chopped fibre, doesn't sound like fun to me! I mean, which fibre connects to which? It's not like they're colour coded!"

Fixing a chopped fibre is fairly straightforward since all the fibres are colour coded thats why the fibre link is the least of the worries! in fact they have probably run some temporary fibre in around the unsafe area overground, fixed a joint with some terminating equipment at either end and there you go, service restored. I could have knocked that on out in half a day given BT's resources.

Now if they had (as some reports suggest) 50,000 subscribers on copper and they had managed to get those back on line I would be truly amazed!

0
0
Silver badge

Title is wrong surely?

The title seems to imply that it was BT's gaffe that caused the problem, but in fact it was an unnamed contractor who bored through their, hitherto working just tickety-boo, kit.

Now, I like to give BT a kicking as much as anyone, but wouldn't "Olympic contractors shaft BT up back passage" be more accurate? Or is my monday comprehension at an all time low.

All the best,

0
0
Stop

Well some of us have resilience...

We lost an entire BT Central DSL platform as a result of this outage - impact to our customers? Oh that would be none as we actually bothered to build a resilient network, if you've got access problems you need to take a look at your ISP as much as BT for this little disaster...

0
0
Linux

Networks.....

Isnt the whole idea of ISP networks and what not. To be able to quickly be re-routed when something like this goes wrong?

0
0
al
Paris Hilton

can BT enter ISP business ?

If they have fibre channel network all over lbighty.. and all they ever do is send bits to turn a light on & off every 30 secs.. probably they have some b/w to spare ?

PH, coz she does down more than the traffic light!

0
0
Coat

Snigger

You said large thrust borer

0
0

@Mike Shepherd

> why no redundancy

... I suspect that they probably had redundancy but that in the post incident analysis they'll discover that all the redundant paths were in the same tunnel.

N.b. I recall something similar happening in California when I was there 10 years ago (construction work severing major fibre artery) and then it took PacBell 2 or 3 weeks to fix it completely

0
0
Coat

internet structure

i thought that the whole reason the internet was built how it is, is so that if one connection is cut (or city is bombed) then the others can carry on as normal (or with money saving measures, with impaired capacity) not no connection at all.

0
0
Alert

Just to put the usual moaners right...

The contractors (Yhprum), working for a well known London water authority, had bored 48m sideways at the bottom of a 30' shaft. Far play they did own up when the BT vans turned up.

The boring machine went exactly 90 degrees through a deep level tunnel collapsing the tunnel and damaging 19 out of 80 cables that go through the tunnel. At present the boring machine cannot be moved for risk of damaging the other cables (which are still working).

True resilence is a problem from time to time in certain areas, that's why there are deep level tunnels, out the range of the usual JCB. Now that the cabling jointers can get at the cables they are being diverted now. trouble is, it's a big job.

0
0
Flame

@adam

"It's about time BT stopped looking at it's profits and invested in this countries quite frankly shocking infrastructure."

Err... how to make a company bankrupt... don't think about returns on your investments! Especially when the figures are in the billions...

Isn't it the job of the government to look after the country's infrastructure? And hasn't OFCOM made sure that BT cannot invest (up till now) and make a return on their fibre infrastructure investments?

If you want a decent infrastructure at cost, then the government has stump up the money. Otherwise it's market forces that do so, unless of course a regulator sticks their nose in and makes it impractical/unprofitable to do so. And anyone who mentions Virgin Media... they run a monopoly, no one has access to their network and every connection to their network makes maximal profits for them (not so for BT). Market forces work well in France cheap fibre for all, although for any given district there is only one supplier (mini-monopolies).

Not surprising that this country is in this situation it is in financially if people think in this kind of way...

0
0

At last

An article about BT which doesn't mention PHORM!

Oh,bugger

0
0
Anonymous Coward

@Chris Church

Cable are not always clearly marked. You are also dependent in the city to keep accurate records of underground wires and pipes .

0
0
Coat

Ooo-err missus.

"Contractors working on the Olympic site in Stratford sent a large thrust borer through a deep level BT tunnel on Saturday afternoon."

0
0
Thumb Down

Learning Grids down

I'm working in a West Country LA and all our connections to the schools (some 70 odd) are down thanks to this. The Metro-VPN BT bits are b0rked. Glad it's the start of the term break.

0
0

Entanet have an ongoing write up of progress

http://noc.enta.net/2009/04/outage-framestream-leased-lines/

0
0
Thumb Up

You were only supposed to blow the bloody doors off !

Someone has some explaining to do to their boss :)

0
0
Unhappy

heh..

this happens in wellington (new zealand) every few weeks lol.

0
0
Anonymous Coward

Owned up?

"The contractors (Yhprum), working for a well known London water authority, had bored 48m sideways at the bottom of a 30' shaft. Far play they did own up when the BT vans turned up."

Not exactly an easy thing to deny then is it. It wasn't us guv. It was some other bloke with a tunnel borer, but he was last seen heading for Wapping...

And spelling the contractor's name backward's isn't going to fool a lot of people. Actually, given the mugs that fell for the Registers April 1st story, maybe they would...

0
0
ian
Bronze badge
Black Helicopters

So...

With all of the chaos caused by the boring machine, are the E. London video surveillance cameras still in operation? If so, why?

Are the black helicopters grounded for the duration?

0
0
Flame

Only fibre

It would have been much more spectacular if they'd hit one of the tunnels carrying the newly-buried HT cables:

http://www.london2012.com/news/media-releases/2008-12/last-pylon-removed-from-olympic-park-as-250m-powerlines-project-delivered-on.php

Still, there's time yet, I'll bet the new tunnels aren't on many maps :)

0
0
Boffin

Missing cabls isn't easy

The problem is that most older tunnels, pipework and so on are not accurately mapped. In fact, a lot of them were last mapped by the Victorians, using survey reference points that no longer exist. And at 34 metres deep - or even34 feet deep - it is pretty much impossible to locate the tunnel or pipework by surveying. The only way of accurately locating such things is by doing what the contractor did - dig them up!

0
0
Flame

So where is this map of deep level tunnels?

There probably is no map! Half of these are secret, until recently they flatly refused to say there were any at all.

Oh and some are 100ft (30m) under the surface...

http://news.bbc.co.uk/1/hi/england/london/7672374.stm

0
0
RW
Boffin

Subterranean urban infrastructure

It's a nightmare in many (most? all?) cities.

Records are often not kept and when they are kept, they're inaccurate. Too many players, perhaps, plus human error both in the original record and in the aggregation that City Hall undertakes.

There's the famous story of a subway tunnel (or was it a utility trench?) being dug in New York. The contractors unearthed (but did not sever!) a pipe that appeared on no plan. No one had a clue what the pipe carried, nor whence, nor whither. Compressed air? Water? Oil? Natural gas?

As said pipe was In The Way, they finally went after it (very gingerly!) with a hacksaw.

The result was anticlimactic: there was nothing in the pipe except stale air.

Another example: a university in a small Canadian city wanted to run a wire to a remote location (RJE - anyone else remember that?) and leased underground copper from the phone company. The telco assured the network folks that said wire went directly from point A to point B.

Impossible throughput! A wire supposedly less than 2 miles long had the electrical characteristics of a wire tens of miles long and was simply unworkable with even the slow modems of the day. After much backing and forthing and finger-pointing, the telco investigated and discovered a fat inductor hanging off said wire, down a manhole. Left over from a previous use of the wire, perhaps for decades.

No record of it.

QED

"As ye dig, so shall ye unearth."

0
0
Stop

@ Chris Church

"I do hope that the bill for all the work and all the loss of business and problems etc goes to the Olympic Commitee and the contractors. Teach them to do things properly!"

So you'd be happy to pay if a contractor working for you damages a third party's property? Why should the Olympic Committee be penalized for the failings of a contractor? This is typical "government causes all problems" bullshit; just because the OC is paying someone to do a job doesn't mean they have operational control of their activities. No doubt you think that Sebastian Coe ordered them to do it?

0
0
Anonymous Coward

Detecting subteranean Serivices

Given the uncertain nature of records of underground services, then I wonder if the contractors generally use technologies which can detect underground services. There are several of these available, including Ground Penetrating Radar (GPR).

The range and accuracy of these can be affected by ground conditions, but a thumping great steel-clad service tunnel would surely stand out, although whether (or if) boring systems like this can be so equipped, I've no idea. I can't see a problem in principle, although I imagine it would increase costs and might slow things down.

0
0
Happy

@So where is this map of deep level tunnels?

Anyone for Google Mole?

0
0
Anonymous Coward

Resilience

BT do provide resilience if you pay for it.. These cables and fibres also carry broadcast, hospital and emergency service comms/data but they have backups so they all had other options.

BT is a business. Sadly what that means these days is that they are in it for the money like so many businesses unlike proper businesses which are in it for the love of the craft/product/whatever and making money is the mild irritation (instead of things being the other way around)

0
0
Silver badge

@Subterranean urban infrastructure

Doesn't have to be 'sub'

DSL failed at all the offices on our business park, no JCBs in sight, phones still worked etc. BT claimed nothing had changed at the exchange.

Turned out somebody installing a new phone line had decided to 'tidy-up' a junction box and manged to patch all the DSL lines to the exchange 1mile away, via a village 5miles in the other direction.

BT only checks the lines are intact and only do a DSL survey with a new installation - they have no system for dealing with anything that changes after a line is installed. It took a week of calls to different bits of BT to get us back online.

0
0
Flame

idiots.

For gods sake. BT this BT that. It's NOT BT's job to provide a fully redundant network. If you want that then DIY. You really expect BT to put 2 lines to every sodding building in the UK? Its not hard to create a fully diverse infrastructure for your self. You start off by using 2 different companies with there own physical networks, and no that does not include 2 companies who back haul with BT.

And oh look, the internet is working for any one who is not only linked through that point. Guess it shows that it works as advertised.

Bunch of winging fucking wankers. Only Londoners can be so fucking blind to the rest of the world. your worse than Americans.

0
0
Gold badge
Thumb Down

whatever happened to

The idea of a central database to track who and when one of the 200 companies who are allowed to dig holes were going to dig so they could all synchronise their work? Would this also have (indirectly) built up a profile of what and where pipes & tunnels are located over time?

Of course setting up all those black boxes for the IMP is probably hitting the BT budget quite hard.

V.Poor

0
0

What's happened?

Dammit! I thought living on the Isel of Dogs I'll be bale to skip this, thinking the hole is in Straford, but no, phone lines are dead :(.Weird thing is, I can still surf the Net?? Shureley shome mishstake?

Am I being daft, or is that not possible?

0
0
Anonymous Coward

Deep Phibre Interception?

Looks like someone misunderstood the remit. DPI = Deep Packet Inspection you fool!!!

0
0
Ben

@ anonymous coward @ Mark Wills

since the cut is half way through a link, you can't expect to see any 'colour coding' as you sugges. And to claim 1. you would route it overland and 2. 'you would have it done in half a day given BTs resources'? are you serious?

if a major cut occurred in our infrastructure, it would certainly be a disaster scenario only to be remedied by many hours of retubing, reblowing etc etc. of course we'd switch-reroute the less important stuff (say 50000 residential), but no monkeying around with critical infrastructure...

there are events which are financially viable to mitigate, and there are other rare-events which one should never ever pay to mitigate: ask 'what are the chances of someone cutting here... and would we have to pay the bill?' = <1e-9 = no = lose/lose for spending any money there.

Mr. Murphy's insurer will be receiving a very large bill, tant pis.

0
0
Flame

bloody bt

"It's about time BT stopped looking at it's profits and invested in this countries quite frankly shocking infrastructure."

Err... how to make a company bankrupt... don't think about returns on your investments! Especially when the figures are in the billions...

Isn't it the job of the government to look after the country's infrastructure? "

Ummm no, it's BTs job! They are a private company and as such they should provide the infrastructure that they advertise, NOT the tax payer. We have spent the whole bloody day arsing about because BT's idea of resiliency means shoving 2 cables through the same pipe!! BT should be paying HUGE fines for this screw up.

0
0
Thumb Down

actually....

The reason they were not on the plans is that BT dont bother adding new stuff and upgrades to their program called Netword Records.... its updated as far as the mid 90's cos they closed down local drawing offices....

The number of time ive lost due to bad prints is soo frustrating.. so glad i left that company.... grr.

0
0
Stop

Network design

Actually, the Internet was designed so you could lose a whole section and THE REST of the net keeps working, not every single node. From what I can tell, the worlds networks are fine...

Mind you, City Airport was affected. 3 hr delay for me, pilot blamed that outage

0
0

Page:

This topic is closed for new posts.