Fujitsu reveals data transfer protocol 30 times faster than TCP Fujitsu has taken the wraps of a new data transfer protocol which it claims is 30 times faster than TCP and will help speed up mobile web browsing and file downloads and reduce virtual desktop latency. The new proprietary tech is based on the user datagram protocol (UDP) employed for streaming media. As such, it can …
I don't think I've ever seen TCP run at 1/30 efficiency except when selective ACK is off and hardware is failing. Making my ADSL2 or WiMAX connection 30x faster would break the known laws of physics for crappy telcos. I could turn down the ADSL S/N ratio until I'm burning away my all FEC bits and I'd only get 1.2x throughput. Maybe they mean that their protocol has 1/30 the latency of TCP on a network that's heavily congested with TCP traffic? If they've come up with a super-polite traffic-avoiding network protocol then they can expect people to tune it to be greedy like TCP.
I certainly read it as faster over high-latency links (what else cloud it mean?)..
TCP has a windows size, and you can't have more than that amount of data pending in a transfer, and if you have sufficient latency then it becomes the limiting factor. Using a thicker pipe doesn't help (although it can be difficult to get that point through to senior managers - who seems to reckon increasing the bandwidth solves things...).
With UDP you can just keep sending data. Provided the far end sends back relevant acknowledgments you might be able to get away with only a few re-sends.
A bit like how NFS was when it used UDP.
With UDP you can just keep sending data. Provided the far end sends back relevant acknowledgments you might be able to get away with only a few re-sends.
Alternatively, you can use forward error correction to eliminate the need for a lot of "back traffic" (or "packets traveling in the wrong direction which often hamper UDP communications" as the article states). Have a look at the udpcast project for an example of that. It's designed for multicast, where the problem with ack storms is much more severe, but it seems that with a little tuning it should be also be pretty efficient to use it for point-to-point transmissions too. There are also, IIRC, a couple of competing RFCs for implementing reliable delivery over UDP channels, and they include flow control algorithms as a means of congestion avoidance (similar to what's described in the article).
I think the most interesting thing about this paper seems to be how they convert everything to use their new UDP protocols. It seems like a good approach given that it's much simpler to implement congestion avoidance and flow control if everything is based on the same underlying transmission protocol. It does sound a bit drastic, though.
I was thinking much the same. Whatever these figures mean (if anything at all) they sure as hell don't mean what people will naturally assume. My guess would be that they've simply junked some of the TCP traffic management features (the soft start mechanism for example) and cooked up these outlandish figures by basically shitting on everyone else on the network.
Phil's "articles" are basically copy and pastes of press releases with all of the important details cut out and then no link to the original.
The original press release is here;
http://www.fujitsu.com/global/news/pr/archives/month/2013/20130129-02.html
It specifically states "New protocol improves throughput and latency in low-quality communications environments".
Ha! Beat me to it by moments.
This could be valuable work so far as mobile data comms is concerned. TCP is notoriously rubbish when the link quality is poor, yet loads of mobile software uses TCP.
I just hope that it does get picked up. Though they state they're looking to commercialise it...
That'll probably mean it won't make it into Linux (who'd pay?). And if it doesn't make it into Linux then web servers won't be using it. And if web servers aren't using it then there's not much point in Apple, MS or RIM licensing it. This could turn out to be a fab bit of technology that everyone needs but no one gets because no one's prepared to pay for it.
Thank you for the link. And I completely agree with you, rehashing press releases with no additional info or not showing the other side of the story (there are always two), and without giving so much as a link to the original is not what some of us come here for.
Indeed, but whats more funny is when they say a standard `TCP` which has error correction built in to be rubust is slower than UDP `fire a forget proptocol` and then type it upon a QWERTY keyboard. Have to laugh but when it comes to IT instead of ICT then you would already know all about this and be enjoying the laugh. It is when your ICT manager comes in and demands all computers upgraded to this `UDP` overnight that the real pain of such articles becomes reality.
I wonder how it reacts to multiple link load sharing that causes IP Datagrams to arrive in a shuffled order? On TCP that was a recipe for causing congestion avoidance mechanisms to slow things down to a crawl unnecessarily.
It will be interesting to examine the specification for any pathological conditions.
Wow! I just transfered a very large file at about 85 MByte/sec over GigE using TCP. You can imagine my elation to know that my Gigabit Ethernet NICs can now transfer 20 Gigabytes per second with only a protocol change.
So seriously, I imagine this is an alternate algorithm for lost packet retransmit over UDP with attempts to minimize TCP's backoff. There are dozens of schemes to do this with TCP already ( https://en.wikipedia.org/wiki/TCP_congestion_avoidance_algorithm ), but minus the hype codswallop anything new is interesting.
I think other posters have already covered that the paper's authors are only claiming improvements over high-latency links (and fairly lossy ones at that). The fundamental problem with TCP over such links is that each TCP packet needs to be acknowledged so (having a sliding window for acks notwithstanding) transmission speed is fundamentally limited by how fast and reliably the acks can be sent back.
Some guys at Microsoft tested out another scheme for improving transfer speeds over high-latency links (though they assumed low packet loss) a few years ago. It worked by the receiver sending ACKs to some number of packets it hadn't actually received yet, thus fooling TCP's flow control mechanism into avoiding its normal exponential backoff algorithm. That trick obviously only works over reliable data links with very little packet loss. I don't have a link to that paper, but ISTR that it was covered here on The Register. No doubt it also got its share of comments along the lines that you're making here (ie, >100% efficiency).
Maybe this would be something useful for MMOs, or other server based games. I can't count the number of times I've died online because of terrible latency thanks to the connection my ISP had to the server. I honestly see this being the main port of call here. Thousands of connections using both UDP and TCP concrrently, the 30x improvement would definately help.
CAR ANALOGY INCOMING
You're on a motorway and everyone is in big ass trucks. There's only 3 lanes either direction and traffic is grinding to a halt. One way to fix this problem is by widening the corridors and adding new lanes. But that's expensive takes a long time and expensive.
OR
OR you can put in lights at every intersection, this will wind up slowing down traffic on all fronts, but it'll average out the speed. Good for those with terrible access, bad for those with great access.
OR
You can tell everybody to get out of their trucks (TCP) and get on motorbikes (the new thing) congestion isn't as bad, no need for the traffic lights at every junction, and no expensive lanes being added.
If the road was clear though, hopping onto a motorbike would make no difference. And I see it being used to effectively lower running costs by these people. "lol, why should I buy the top of the line connection I need, when I can just buy this new transfer protocol, give people a crappy connection, and an easier way to use it"
"You can tell everybody to get out of their trucks (TCP) and get on motorbikes (the new thing) congestion isn't as bad, no need for the traffic lights at every junction, and no expensive lanes being added."
Yes, but now you need 100 motorbikes to deliver the goods that would previously only require one truck.
"You can tell everybody to get out of their trucks (TCP) and get on motorbikes (the new thing) congestion isn't as bad, no need for the traffic lights at every junction, and no expensive lanes being added."
Or you can have everyone get out of their car/truck/bike and move forward one vehicle. Hows about that? Guaranteed progress even in the face of deadlock....
Ok, so, another protocol that is "not TCP". I do think there have been several proposal during the last ten years.
From the CERN courier, Nov 2005:
Theoretical models now exist for alternatives to TCP that can be mathematically proved not to cause congestion collapse, and work is ongoing to test these models experimentally. Some of these alternatives promise to balance traffic across the Internet, and to deliver high data rates and much better quality of service for services like voice-over-Internet-Protocol (VoIP) and live video - all at the same time. It seems likely that these theories will find their way into the mainstream Linux and Windows kernels within the next five years or so, and we hope that they will serve for many years to come.
Indeed. Why is this "based on UDP"? Doesn't that make it UDP? There is a "protocol" (IPv4) and "next header" (IPv6) field in the IP packet, use it!
More important, where is the RFC?.
There is no RFC, as far as I can tell anyway. IETF hasn't published a damn thing about this from what it looks like, which makes it even more suspect to me. A new networking standard with no RFC isn't something I trust much.
This idea reminds me of old school proprietary networking standards, like AppleTalk for Apple, IPX,/SPX for Novell, etc. Some enthusiasts will use it, and be extremely vocal about it, but TCP/IP isn't going anywhere soon.
"Just the facts. Son"
Ok, so, another protocol that is "not TCP". I do think there have been several proposal during the last ten years.
Indeed. The world is full of sophomoric attempts to improve on TCP, apparently in complete ignorance of the handful of actually clever, but never widely implemented, attempts to do the same. Regular readers of comp.protocols.tcp-ip will have seen the refrain "You're trying to reinvent TCP over UDP" many more times than they can recall.
where is the RFC?
To be fair, you'd hope that anyone even vaguely interested in something like this, if it were at all viable, would be aware of it before there was an RFC. There should be an Internet-draft, though - if it's ready for public announcement (as opposed to discussion on the IETF mailing lists and the like), it's past time for an I-D. Then the inventors could submit a standards-track RFC once folks had a chance to beat on the I-D for a while.
SCTP was created about 20 years ago to solve the same problems and thats hardly used even though its an open protocol. The chances of this proprietary one knocking TCP off its pedestal anytime soon is zero.
Anyway , any idiot can get a 6 times improvement on performance over TCP on streaming media which is a medium thats conducive to losing a few packets. Try the same downloading a gigabyte zip file however and see how well it works.
Wow
Never seen such an ignorant bunch of numpty comentards (except one)
So many people who don't even understand the basics of how TCP works and behaves over high latence links yet feel comfortable shouting their ignorance to the world here.
I used to think that reg readers were reasonably tech informed, how wrong!
AC for obvious reasons, though I would have chosen the D'Oh! icon, or Paris for more intelligent responses
Yup. Chances of the World at large letting Fujitsu become 'transporter of choice' to the t'Internet World? Hmm, let's not take that bet.
Sony for example, a company that consistently through the nineties and naughties delivered new consumer tech which used proprietary interfaces/transfer mechanisms/cabling/etc. yup, that really worked for them. Oh, hang on....
Interesting the bit about it coming from the Middleware Division. here's an excellent example of how in tune with the World they are:
(EA) 'We are using Agile methods to develop software solution Zzz'
(Middleware VP) 'this is your problem, using rubbish methods to develop software. All software should be developed slowly on at least an 18 month delivery cycle"
(EA) 'what, including bug fixes?'
(VP) "all software. Anyone who delivers bug fixes is delivering bad software. We never release anything in less than an 18month cycle. It's why you Western Companies fail to produce good software"
Real world anyone?
Aspera has been selling file transfer software that is based on UDP for many years now. As have other vendors (eg Data Expedition). For an open source starting point, see Tsunami. The trick is using UDP to overcome high latency connections that kill TCP throughput. The clever bit is detecting when UDP packets get lost along the way and then reducing the rate at which the packets are sent to keep packet loss to a minimum.
Agree, the performance problems with TCP (and IP) have been known for years and as you note various non-IETF approved or published solutions do exist.
It would seem that Fujitsu have implemented a connection-oriented protocol on top of UDP - I assume it also uses the published UDP API's. Along with a gateway for TCP.
Whilst people have picked up on low quality communications environments, I find the diagrams in Fujitsu's press release more interesting - these show that the new service may well find a very profitable niche in the Enterprise, doing those jobs that TCP doesn't do very well...
Devices and protocols for improving TCP throughput over high bandwidth x latency links have been around for DECADES, for about as long as TCP has been carried over satellites. This is not a new problem, the question is what would make Fujitsu's solution any better than what's already out there.
>This is not a new problem
Agreed, however I suggest some companies are starting to see the limitations of straight TCP/IP solutions.
>the question is what would make Fujitsu's solution any better than what's already out there
1. It will most likely work straight out of the box - unlike NETBLT
2. Its integrated into something useful (also unlike NETBLT and other solutions) and is likely to work on data centre platforms that don't natively run Windows or Linux.
3. It's supported - by a single enterprise vendor.
The downside is whether you want yet another middleware vendor - but if it lets you get more out of that international leased line between your data centres, or your international support desk to give a better service ...
UDP has always been better at passing data then TCP - a lot cluster solutions use the UDP packet as part their communication background as well as data sharing. RAC does not use TCP/IP over the interconnect.
And then there is infininininininininininininiband, that supports many different protocols er.. verbs for transmitting data at very high bandwidth. Not sure this is even close to news....
"A Fairer, Faster Internet Protocol"
--- Set up a website with the docs, philosophy and rationale
--- Write some RFCs
--- Write an article for IEEE Spectrum (in 2008)
Been done before - XTP from the late 1980's comes to mind as an exemplar.
And I note that even given the names promoting this protocol and their sponsoring employers they don't have any real implementations...
Basically the problem is TCP & IPv4 are the networking equivalent of the QWERTY keyboard, which also could be dramatically improved upon but it is good enough for most purposes and due to its massive installed base will be difficult to change and displace. Which is a shame because there is a real danger of protocol research becoming another branch of computing that is largely ignored by the outside world.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
