BBC engineers have solved a mysterious, long-running bug that has meant iPlayer and live TV streams have frequently prompted the BT Home Hub, UK's most common router, to reset itself. The source of the problem has been identified after a lengthy, "tricky" search and a fix is currently working its way through the BBC's back-end …
iPlayer on Be to work?
I need this. The Olympics would have been so much more enjoyable without iPlayer locking my BeBox (Speedie 585).
BBC engineers fix BT bug
So, now they've set a precedent, can the BBC send a team of engineers round to me - I've got several bugs which I can't be bothered to sort out.
(Douglas Adams reference:) If the beeb feel like fixing bugs that are somebody elses problem, does that mean that BT have produced a working SEP field and can thus be rendered invisible? Pleeeaase!!
Be works fine
IM on Be, iPlayer works fine for me..
Hang a mo...
The way I read this the fix has been applied in the adobe flash software, not in the Homehub. So this would suggst that the "bug" or "compatibility issue" actually sat on the Beeb's servers, not the home hub.
Furthermore the fact that some of the Beeb's servers worked OK with the Thomson hardware and others didn't would suggest that Auntie's techies had forgotten to make the change on one set of servers. Or perhaps the techies who set up the first set of servers had failed to document the setup properly.
So much though it pains me to say it, it appears that this wasn't BT's issue.
It worries me that it seems that the BBC hadn't done any compatibility testing on the most popular single home router in the country. This smacks of supreme arrogance. "We've written the code, it's up to the world to make sure their hardware supports it." If this is the case then all credit to BT, Thomson, et al for standinf firm.
Wasn't just iplayer
I also had crashes on Steam games (Team Fortress 2 etc.) and GTA4 on the PS3 just wouldn't even connect to online games.
Got it sorted eventually when they sent me one of their new home hubs. My advice to anyone still getting these crashes is to just get a replacement
Re: Hang a mo...
I've tweaked the intro to make it clear the BBC didn't "fix" the Home Hub. There at least four players you could blame here, or not blame anyone, so I'll leave that up to you to discuss.
The BeBox may not be the most stable piece of hardware in the world (quite an understatement really), but we've never had a problem with iPlayer on it. That's with our, albeit small, office of people all watching the Olympics at the same time - no resets at all during the last couple of weeks.
Seems to me from the article that the flaw was triggered by the method of flash video delivery so whilst the BBC cannot fix the flaw in the hardware they can produce a workaround.
So in my opinion they're not at fault but they did fix it.
@ Chris Williams
Four players you can blame?
No, there's just one: Thompson. Routers are not supposed to fold simply because of the data they're routing.
Re: Hang a mo...
Er, Hang on another mo... surely the issue here isn't that the BBC have fixed their streams, its that a certain combination of packets going through a router can cause it to reset itself?
This appears to be absolutely BT's and Thomson's fault - there really shouldn't be any way to remotely reset a router like this!
The BBC, presumably, took the path of least resistance...
Hang on a little more...
Even if you blame the BBC for this problem on the basis that it was their code crashing the BT router, BT are still to blame and should be paying for the fix.
Dodgy code should never cause a router to drop a line. The fact that the BT router dropped its line is a fault with the router and as BT supply the router its their problem.
Its nice that the BBC have done the honorable thing and provided a work around but the fact remains that those who own BT routers are likley to suffer simillar problems from other websites using new tech.
I am actually somewhat concerned that my licence fee has been used to diagnose faults for a private company. In this instance I have helped pay for BT's denial or responsibility.
"...the most popular single home router in the country..."
The most populous, possibly, but in many quarters, far from popular.
Re: Hang a mo...
"Er, Hang on another mo... surely the issue here isn't that the BBC have fixed their streams, its that a certain combination of packets going through a router can cause it to reset itself?
This appears to be absolutely BT's and Thomson's fault - there really shouldn't be any way to remotely reset a router like this!
The BBC, presumably, took the path of least resistance..."
Yep, pretty much. For example, Hulu still does the same thing to the affected routers.
The BBC have introduced a workaround, but the firmware on these chipsets is still boned.
A router that resets itself due to 'dodgy' data...
...is really the problem, not the servers that stream it or the clients that receive it. Assuming the client software isn't making unreasonable demands on the router, like opening up hundreds of connections (which would be a bug on the Adobe software), there shouldn't be a problem. If BT bought in shit hardware that doesn't work reliably under normal networking conditions, the fault lies with them, not the BBC.
It's just that in this instance, the BBC felt compelled to do a fix at their end because the crappy Home Hubs make up such a large percentage of the end-user router market, and that good ole' BT would probably take a year or two to sort this out.
The BBC have stopped their player crashing home hubs
But it still leaves it vulnerable for any other site to do it. Especially if someone sets out to deliberately break them
This took bl**dy ages for BT to admit there was a problem. Even when it was becoming well documented on BT's own support forums their customer service teams were still denying everything and telling people it must be their computer, filter etc.
With regards to the "tiny number" of people struggling, it was pretty much anyone who didn't have the very latest version of flash plus most Mac users even with updated flash.
It's great that the BBC have done a workaround, even though it's taking ages for them to update the videos on BBC News. This is still a problem because occasionally their videos auto-start and therefore break my router before I have a chance to stop the video.
BT should still apologise because it's their router and because they've been so crap about it.
Re: iPlayer on Be to work?
iPlayer worked ok here on Be*
However the watching directly on BBC Olympics site (http://news.bbc.co.uk/sport1/hi/olympics/default.stm) would kill the bebox (SpeedTouch 585) consistantly after watching one video...
Sorry, who paid for this exactly? I assume it was the taxpayers time and money being spent to fix BT or Thompsons sub-standard rubbish?
Are they going to do the same for my router if I find a similar problem, or is it just special cases? Surely it should be a cost for BT or Thompson who get the revenue?
Worked fine on our Motorola
router that what is now Virgin and was Telewest supplied. All during the olympics we were watching two things at once. One via the red button on the box and one via iPlayer on the iMac. The internet never fell over the whole time, though we had to reset the digibox on the telly numerous times... So obviously not the Beeb's fault.
Wouldn't touch BT's internet with a bargepole.
"The most populous, possibly, but in many quarters, far from popular."
Or you could buy yourself a dictionary.
and we've had all these problems.
As all the little Headaches are sports mad, I can guarantee, that during the Olympics. my bebox will freeze and go off-line about every 10 minutes.
Curiously, one of my clients phones to say that the Archers (R4) stopped every time at 3m34s and her box would reset. O2 this time and a different box.
Anyway, in this house it still wasn't working properly last night.
Not just BT
My Netgear router keeps falling over whenever I try to watch a BBC News Online embedded video. Wasn't sure if it was Netgear, BBC or Firefox to blame, but my iPod Touch plays back fine.
I've upgraded the router to the latest firmware and that's made no difference.
I'll wait and see if BBC's "adjustments" solve the problem.
[cue 9000 Reg. readers saying they have no problems with their Netgear DG834s]
My original O2 Wireless Box...
...had this exact same problem. I spent ages on the phone to an O2/Be tech support guy discussing this, and he found it as bizarre as I did.
When the iPlayer stream changed from the channel logo video to the programme proper, the router would go into a "BBC-only" mode - i.e. I could only access *.bbc.co.uk domains, and all other domains would time-out. (The programme would stream okay during this time, as it came from a bbc.co.uk domain.)
After a few minutes of this, the whole connection would drop, and the router would reset. (Probably 3m34s - the same issue as Ivan's client, above!)
They sent me a new O2 Wireless Box II, which works fine!
To Be or not to Be
If you're on Be surely you can afford to download and watch later rather than stream?
Screw the customer.
This isn't a BBC, BT nor Be*/O2 issue, but a Thompson and, to some degree, Adobe problem. So, what's the "obscure undocumented" setting and is it related to Adobe's RTMP? Thompson support denied any knowledge of the problem and the ISP's decided they couldn't afford to replace all the free routers for ones which work properly. Their support lines said it wasn't a problem with their service. What happened to proper engineering? And support? Oh, and the Beeb are pretty drawn into RTMP, so why are we only hearing about this now? 2008: screw the customer.
The heart? Cos we all need a little love...
It's worrying the number of products that ship with firmware which is plain not-fit-for-purpose, I'm talking about cable modems, mobile phones, data cards, routers even graphics cards.
I remember having to debug a TCP/IP stack in a certain product similar to the home hub, it's NAT layer recalculated the TCP checksum (which is required) without checking the original checksum was correct (The correct procedure is to drop packets who's TCP checksum is incorrect, or forward them with an incorrect chksum). The upshot being that it corrupted TCP streams when the packets got corrupted, leading to randomly corrupted downloads.
If that isn't enough firmware in standalone devices these days seems to be written so that if it crashes it reboots and starts from scratch, which is all well and good, but how the hell does it get through Q/C when it crashes periodically?
Another problem is that many users don't know how to update the firmware themselves, or in the case of some mobile phones you need to pay some junior engineer to flash it for you, because the firmware is not publicly available for copyright reasons.
Oh, and "router crashes when using iplayer" stinks of a network layer buffer overflow. That means someone can (probably, they may have to convince your computer to send certain packets which is a little harder) own your router remotely by sending the appropriate packets when they feel like, honestly some people should be banned from writing their own TCP/IP stacks, there's enough working stable free stacks out there already without writing another buggy one. It's like junior software engineers who feel they need to reimplement the ftp protocol, guess what, FTP is non-trivial and most people make a complete pigs ear of it.
Iplayer + BeBox...
funnily enough, I don't think Iplayer does crash mine. Everything else does though. I've still got my 2wire hg2700 (the only good thing from BT). Anyone got any info on configuring it for Be?
I'm a bit of a fan of Thomson Speedtouch routers, in my experience they've been reliable and robust. Particularly good Speedtouch router models I've had the pleasure of using are the 585i v6/v7, the 608WL, and the 780. None of these exhibited the symptoms described in this article.
Does anyone with one of the affected routers know which Thomson router it is based on? Or is it the case that BT have developed their own firmware, and the custom firmware is at fault?
This problem has been the BANE of my life when trying to catch-up on programmes via iPlayer.
BT support refused to help, instead pointing out I was being 'metered' for downloading and then saying that was the reason.
The 'small number' of customers affected is UNLIKELY considering there are millions of these crappy white boxes everywhere across the country. The only reason I haven't actively moved away is because a family member is using the email addys as their business addy (despite my advice) so there's been a degree of inertia.
The new Home hub 2.0 should be provided free to angry customers like me!
No Wonder I had to buy a new router in March. I only noticed I got randomly kicked off the net until I got that fed up with it that I went and bought a new one. Mine was a thompson speedtouch from Be, and it was a royal pain in the neck.
Just confirming that it did effect some users and cost them money. Thats the last time I buy anything from thompson or adobe.
I thought some rogue web app might be resetting my router, didn't realise it was the beeb. As stated above, it was security liability so I ditched it. I don't want people bagging my filth.
Hmm not so much a fix
but an exploit :)
There is something in the Home Hub that when sent a packet of information of a certain structure will cause it to crash.
Oh what amusement, this is a little denial of service trick, a lot of box routers have these problems, there was one for netgear as well.
Basically it is a flaw that allows people to drop your connection.
I don't know you lot, there are people who will sell you a nice little box of tricks as your router, all mini-itx'd up unix os of your choice, ultra low power ultra quiet, bit of tor and privoxy, perhaps a smidgeon of backup as well, firewall, deep packet inspection, anti virus oh I could go on, but you lot just feed the BT monster - don't worry about the little guy who can protect your arse :)
What's a good router to buy that doesn't cost too much ?
My (second) Home Hub packed in some time ago and I went back to using the Voyager. I can't be arsed going through BT's "unplug the master socket" routine that they put you through before admitting it just could have packed in, so easier just to buy a new and reliable router. Ta.
many short-lived connections?
I've only ever watched what iPlayer* does when connecting through a web proxy, and it's not pretty: Multiple HTTP POST requests per second. If it's doing anything similar when going direct then it's probably filling up the port translation tables of smaller/more simplistic ADSL routers.
(* I assume any use of Adobe's RTMP does much the same thing. Also does a nice line in causing Windows web browsers to hang.)
Search eBay for DG834G - plenty for under £20 and seem very reliable - they've become the de facto standard for people who buy their own routers (or at least those who know what they're doing but can't afford the Draytek)
No, just buy the Draytek!
Cheap and reliable form unnatural bedfellows at the best of times when your perimeter security and connection reliability are at stake.
On a small tangent; is it only me that objects to the BBC using my bandwidth bit torrent style on iplayer or did I misread that bit?
Thanks for the recommendations, I may in fact go for Linksys.
And yes, iPlayer is bittorrent, it comes up with "7 sources active" or whatever which are the torrent feeds.
Which begs the question, will my ISP detect this torrent activity and report me to the BPI ....
BBC fix BT problem?
Why should the BBC fix "BT's Problem" ?
Isn't this just the BBC making a WORK AROUND for BT's problem?
Good rant mate, I agree entirely (I'm a software test engineer and I agree with all of what you said).
Not just routers - I've had 4 occasions in the last year where shops couldn't serve me because their tills were "down". Abbey National (spit) have replaced software systems recently and they are f(*&ing up all over the place - wrongly spelt names on cards, lost direct debits, the works.
And if this router was based on Thompson Speedtouch then it's not surprising it fell on its' arse - I had a modem based on that chipset which was frankly shite. Now I have a DG series router. Solid.
I knew HomeHub was just a cheap glossy front with cack insides!
- Geek's Guide to Britain INSIDE GCHQ: Welcome to Cheltenham's cottage industry
- 'Catastrophic failure' of 3D-printed gun in Oz Police test
- Game Theory Is the next-gen console war already One?
- Apple cored: Samsung sells 10 million Galaxy S4 in a month
- BBC suspends CTO after it wastes £100m on doomed IT system