Testifying as an expert witness on bandwidth management at the FCC's field hearing in snowy Cambridge this Monday was a heady experience. The hearing took place in a cramped corner of the Harvard Law School, a building that was already decorated with pickets, banners, and reporters when I arrived. Gingerly stepping through the …
Besides the point
Traffic management for the greater good is pretty much besides the point.
Charge people for what they use - and use the income gained to pay for larger pipes. If people want to download the odd email/webpage then charge them for that, and if they want to download gigabytes then charge them for that.
Scarcity is best dealt with through pricing measures.
Not a bandwidth issue
I agree with you Andrew Ducker, pricing is the best mechanism for dealing with scarcity. But BT isn't causing a *bandwidth* crisis. Give it a T1 and it will saturate that in no time too.
BitTorrent screws other applications which need low latency and are upset by jitter, particularly on a shared cable segment. Comcast's solution is the least-bad, given what it has available.
All wrapped up again
Why is no one slapping Comcast hard in knuckles for stuffing the meeting with shills?
That behavior alone defines Comcast's views on net neutrality. They have no respect for authority when it is not in their best interests.
You can write/talk till the tech is coming out of your ears but it's not a tech problem it is a corporate governance issue.
It is becoming more economic for them to break their own customers and use the political influence gained with graft then to actually invest in facilities.
Put up, or ...
"Why is no one slapping Comcast hard in knuckles for stuffing the meeting with shills?"
Because it's irrelevant. One set of shills keeps another set of shills out of a room? Well, boo-hoo.
"it is a corporate governance issue"
The Neutrality campaigners are backing TECHNICAL legislation, which would decree how a network operator manages its own network - allowing some TECHNICAL measures and prohibit other TECHNICAL measures.
If you are in favor of this, please let us examine your version of such legislation.
Otherwise, shut up.
"No one has suggested that Comcast's management of BitTorrent caused any harm: as a Comcast subscriber and BitTorrent user, the practice kept the application running well, without degrading the rest of the neighborhood."
Okay, as a Comcast subscriber and BitTorrent user, I hereby suggest that Comcast's management of BitTorrent causes harm.
If you're unable to seed, it harms the operation of the program (it may keep "running well" for downloads, but if your goal is to distribute something that's little comfort). If every ISP followed Comcast's lead it would kill BitTorrent totally. That sure seems like harm to me.
I have a solution so make me the head of the FCC
Only sell bandwidth you have at rates you can afford to provide service at.
If my gym signed up so many people you couldn't get in or use the equipment or they had to throttle people at the door, that would be unacceptable. Why is it acceptable here?
What they claim they are selling and what they are selling are totally different, and somehow this is obfuscating a rational technical debate on network management.
There are several unrelated issues here colliding. I don't think we should be discussing the technical one (as Comcast wants) before dealing with the issue of Comcast being dishonest with customers.
"Because it's irrelevant. One set of shills keeps another set of shills out of a room? Well, boo-hoo."
Not quite. It is relevant to the extent that Comcast literally bribed people to support their cause as opposed to genuinely interested parties. It is an attempt to subvert the process and it reflects poorly on the over all integrity of the company.
It is deceitful and rather shameful and when held up with their past behavior of denial and cover-up (that even Bennett admits Comcast has engaged in), then it becomes hard to take them seriously at all.
Richard Bennett: Was incorrect, Still incorrect, Always incorrect
> BitTorrent [gets its performance boost] as a direct
> consequence of its scalability, by running dozens
> (or even hundreds) of TCP streams concurrently.
For each swarm, BitTorrent only uploads to 3-4 streams at any given time, out of somewhere around 40 streams that it establishes. The remaining streams are dormant and nearly silent. Using CATV-Internet, the low upload allocation given all customers normally permits no more than two or three active swarms at any given time. These fact breaks your next assumption ...
> The proliferation of streams gives BitTorrent
> immunity, at least partially, from the Internet's
> packet-drop-triggered congestion management system.
Given that you have set your readers up to believe that BitTorrent is actively exchanging data at a high rate of speed over dozens or hundreds of connections, your readers might accept your untested, unproven assumption as fact.
The truth is that anywhere from 3 to 12 of those streams will be actively uploading at any one time. The dozens (not hundreds) of other streams are quite silent and are not contributing to any congestion. The "packet-drop-triggered congestion management system," therefore, only needs to quell 3 to 12 streams and this works very effectively.
Evidence that this works can be found in the hundreds of daily posts on BitTorrent support boards from users who have mis-configured their clients and try to upload faster than their own modems can handle. Because of the "packet-drop-triggered congestion management system," they achieve poor upload AND DOWNLOAD speeds, owing to the fact that dropped outgoing ACK and REQUEST packets are causing delays and retransmissions.
> By contrast, most Internet traffic moving upstream on
> residential broadband networks comes from applications
> with no more than one stream active at a time.
This claim is not only unsubstantiated, it is incorrect in any manner that you might try to consider it.
If you are talking about "Most" as in surfing (http requests), then most internet traffic comes from web browsers -- often 2 simultaneous connections PER SERVER. Due to advertising, most pages are made up from resources on multiple servers. This comments page draws advertising, images, code, and content from five (5) servers. Loading this page opens 10, not one but 10, simultaneous streams making HTTP requests.
If "most" means the amount of upload traffic, then most of that comes from BitTorrent P2P. As I said above, this will consume 3-4 active streams per swarm, and someone would normally operate 1-3 swarms at a time.
These facts completely invalidate your argument, as any packet-drop-triggered congestion management would quell BitTorrent's 3-12 upload streams AT LEAST just as well as it would a web browser's approximately 10 upload streams.
You seriously need to stop spouting this defective argument . It is like an addiction. So any time you hear yourself saying "hundreds of connections," please snap a rubberband that you keep around your wrist until this unwanted habit goes away.
Comcast abusing network standards...
"By contrast, most Internet traffic moving upstream on residential broadband networks comes from applications with no more than one stream active at a time."
Actually, anyone who has more than one mail account setup in things like Outlook, Thunderbird, Mail.app will end up with multiple streams of traffic going, and most modern browsers enable a certain degree of concurrency. In Firefox, for example, network.http.max-connections is 24, although it will only open up to 8 per webserver.
While your list of suggestions is a reasonable starting point, it's hardly the case that Comcast's practices live up to them. Let's go over some details:
"Does the practice support a rational goal, such as the fair distribution of bandwidth?"
This is a fine goal. Fair distribution of network bandwidth could be readily implemented by having a queue per IP address and pulling packets from each active queue in a WFQ/WF2Q fashion.
"Is it applied, adapted, or modified by network conditions?"
This is a bit too vague to become a useful criteria.
"Does it conform to standard Internet practices, or to national or international standards, and if not, does it improve on them?"
Forging reset packets is obviously a violation of standard Internet practices and the denial-of-service aspect is quite arguably criminal.
"Has it been communicated to customers?"
As Cade Metz eloquently said, "Eight months after an independent researcher revealed that Comcast was secretly throttling BitTorrent and other P2P traffic, the beleaguered American ISP has at last admitted that's exactly what it's doing."
"Has technical information that would allow for independent analysis been made available to the research community and the public at large?"
Indeed. Comcast's customers ought to be notified of just how much overselling of the bandwidth they have paid for is going on, and just what the average available bandwidth is for the various service levels actually are.
"Does the practice interfere with customer control of traffic priorities or parameters consistent with terms of service?"
If you end up breaking functionality for people using Lotus Notes, or, for that matter, seeding things via BitTorrent, you've interfered with customer control over the network-using applications which they have chosen to run.
"Is the practice efficient with respect to both the upstream and downstream data paths?"
Indeed. BitTorrent is remarkably effective at utilizing the available bandwidth and copes much better than systems like HTTP or FTP where the server side often constitutes a central point of failure.
"Does the practice accomplish its purpose with minimal disruption to the network experience of customers as a whole?"
See above with regard to breaking users of Lotus Notes. Forging reset packets obviously constitutes a complete disruption to that particular network connection-- that hardly qualifies as "minimal disruption".
Richard, you take the cake!
> The composition of the crowd wasn't apparent
> until Comcast VP David Cohen got an overly
> enthusiastic round of applause at the end of
> his prepared remarks, but pretty much only
The composition of the crowd included five busloads of disinterested people who Comcast paid to occupy seats, keep the opposition out, and clap when instructed. They didn't even do that very well as the applause was several seconds late.
> They didn't hiss and boo - unlike the
> free-speech-loving neutralitarians who
> replaced them.
I heard the entire meeting. There were about 4-5 spontaneous moments of applause during a presentation. Otherwise, the crowd politely held applause for everyone until after their presentation.
PS: I do, however, GIVE YOU SINCERE CREDIT for not delaying the meeting when the slides you brought weren't ready. I might have panicked and held things up trying to get them, but instead you kept things going.
Maybe Comcast should do its own Torrent
If Comcast did its own torrent (seeding files) on the other side of the narrow pipe (the upload stream on cable), it wouldn't have problems. All it might take is a simple server with a few gig's of disk space or some such. Then the users can grab the downloads and Comcast's server can do the throttling all by itself.
Oh, and funny how DSL people don't have this problem.
What about providing the service they sell?
When will these companies stop to sell bandwith that they don't have? I think that it would render any other consideration superfluous.
But they keep selling theoretical super-high rates, and using "management" to prevent people from using it... is that considered as a "good practice"?
What a load of bol^H^H^Hrubbish !
>>> BitTorrent screws other applications which need low latency and are upset by jitter, particularly on a shared cable segment.
No, a number of high throughput transfers of ANY time will do that. BT is just unusual in opening a number of connections in parallel.
>>> Comcast's solution is the least-bad, given what it has available.
The fact that it has failed to invest in it's network is not an excuse for this. By the same argument, you could say that a highway authority would be reasonable in not putting tarmac on a new road and blaming the traffic for making the bare rubble full of potholes.
And I disagree with a statement made in the original article :
"... peer-to-peer uses the Internet's classical mechanisms in a novel way ..."
No it doesn't, it uses a TCP connection in the classical way - open a connection, stream some data through it, close the connection. I suppose you can argue that opening multiple connections is novel, except that web browsers have done that as a matter of routine for many years. There has been software that will open multiple FTP sessions and download different files in parallel. So the only thing novel about BT and P2P in general is the fact that it uploads significant traffic and opens more connections than normal.
But each of those connections will follow normal control methods - if you drop packets or delay packets, it will slow down. "Normal' methods of delay, queue, and ultimately, drop DO work on BT - so if there is a bandwidth shortage all you have to do is queue packets to constrain the client to what you can provide.
What Comcast, and it's supporters, are trying to do is justify failing to invest in a network capable of sensible management and use that as justification for what is downright nasty to the network.
But the biggest problem of all is that people are missing the point - that the ISPs are selling something they can't provide<period> It's time they started selling on committed rates, based on what their network can actually support. Once they do that, then it's much easier to define what's "fair" - anything up to your CIR is fair, anything over that is game for management as spare capacity allows.
"If you're unable to seed, it harms the operation of the program (it may keep "running well" for downloads, but if your goal is to distribute something that's little comfort)"
From Comcast's terms of service: "You agree not to use HSI for operation as an Internet service provider, a server site for ftp, telnet, rlogin, e-mail hosting, “Web hosting” or other similar applications"
I hereby suggest that seeding a torrent falls into the category of "other similar applications". It may cause harm to your seeding activities, but those activities are prohibited under the terms of service you agreed to when you signed up for Comcast's service.
You want to seed torrents? Go find an ISP that specifically allows it, like Speakeasy or buy a fractional T1. It's more expensive, but if you want to serve content, expect to pay for it.
I call Bullshit!
"BitTorrent screws other applications which need low latency and are upset by jitter, particularly on a shared cable segment"
I have some 16 computers within my LAN, two of which are running BitTorrent continuously. I have 15 Mbps downstream and 2 Mbps upstream on a FiOS line running through a single router; all 16 computers share the same 2Mbps upstream pipe, in other words.
With BOTH BT machines running at "full blast," I can play World of Warcraft, surf the Web, and upload and download with conventional ftp clients all simultaneously without *EVER* noting any undue latency; I routinely check my latency to the WoW servers and it is always in the neighborhood of 75 to 100ms, regardless of how much 9or how little) BT traffic is going through my router. ALWAYS.
Ergo, the concept that "BitTorrent screws other applications which need low latency" is at best a misconception, and in my opinion, it's a knowingly, willful, and deliberate lie.
ComCast has *already* limited their DOCSIS subscribers to 256Mbps upstream; throttling traffic by forging RST packets is unnecessary if ComCast has sufficient of the product they sell - bandwidth - to meet their contractual obligations.
And if ComCast does *NOT* have sufficient bandwidth available to meet their contractual obligations, then I humbly submit that they are in breach of laws regarding commercial fraud and contracts.
I find the author of this article to be disingenuous and untrustworthy on this subject. Robb Topolski has already shown this to be the case regarding the "ComCast shill" shenanigans.
Conclusion: Richard Bennett has taken the side of ComCast in a case where blatant fraud has evidently been committed by ComCast. The article is slanted to present ComCast in a favorable light, which light simply does not exists. One is moved to ask why?
Initiating multiple piece requests over bittorrent is not actually that different from initiating multiple HTTP RANGE requests to a single or multiple web servers. The only real difference is that in bittorrent, every user becomes a "mirror".
>> "it does so in an unconventional way that essentially exploits a loophole"
What loophole is that? It uses multiple TCP connections in the same way as many other protocols. If TCP was only ever designed to support a single open connection at one time, then the net would break. An open connection to your mail server would mean all incoming mail was dropped. An open connection to your webserver would drop all other visitors.
>> "It gets its performance boost from the ability of BitTorrent to access a deeper pool of bandwidth than a centralized program can; there's no way to transfer a (compressed) file faster than to take more bandwidth."
So your complaint is that it's not effecient to have only a single connection open? Meet the rest of the world who worked that out years ago.
HTTP 1.1 range requests can access a deeper pool of bandwidth by using mirrors and multiple connections than a dumb browser can without them, does that make HTTP 1.1 a bad spec?
>> "The loss of a single packet slows this application down, and hence the entire PC that runs it. The loss of a single packet by an application with dozens of active connections hardly registers on the host PC's bandwidth consumption scale"
So, a single packet lost from an application with one socket open will slow down all applications on that machine, also with single sockets open, but the loss of a packet in an application with many sockets open wont? Feel free to explain that one.
Either packet loss will slow down all connections, or it wont, regardless of which apps own the connections.
I think your complaint is more that a user will visibly notice when their single connection to an email server is suffering heavy packet loss, whereas they wont when a bittorrent connection is affected.If that's the case, it's not hard to see why you love Comcast so much if you dislike network resilience.
From the look of things, the shills weren't just in the audience.
The point is that *on average* other apps use much fewer connections than BT, largely because they all operate in bursts of single operations, while BT runs many connections in parallel persistently. Because BT's connections are persistent, they're not in slow start and have a higher bandwidth quota *on aggregate* before and after any single packet drop against any one of them. That's how BT beats the system.
So the only way to deal with this from a system fairness point of view is to police BT streams specifically.
Bad weather in Cambridge
I was surprised to see mention of snow in Cambridgeshire and Lincolnshire - I know that it is Winter in the UK , but I thought that global warming had put a stop to snow laying around on the ground.
After careful reflection, I realized that the author was probably talking about somewhere in one of the former Colonies. As the Reg's url is http://www.theregister.co.uk/ perhaps the author should have mentioned, say, Massachusetts.
Mine is the tweed one with the union flag emblem, pedantically the right way up, on the lapel.