2 posts • joined 3 May 2010
IB and RoCE are not intended to compete
Alien Anthropologist is quite correct. IB has traditionally led Enet by several generations in bandwidth and latency and has always been the lowest price/performance.
That said, there are environments where it is simply not possible to transition to a new IB infrastructure. For example, many commercial enterprise data centers could benefit from IB's value propositions other than its raw bandwidths and industry leading latencies. For example, RoCE can bring major decreases in CPU utilization and reductions in memory bandwidith demand to an existing Ethernet infrastructure thus significantly reducing the number of servers required and reducing power consumption and footprint. These are the types of markets at which RoCE is aimed. RoCE does not compete with IB in terms of pure horsepower or price/performance. The fact that RoCE delivers 'IB-like' latencies is a very nice advantage.
Chief Scientist, System Fabric Works, Inc.
Chair, IBTA RoCE Technical Working Group
iWARP and TCP
What Tom says is true. iWARP is usually implemented in an RNIC which effectively offloads TCP into hardware. This is how iWARP is able to deliver its latency numbers. The drawback is that you cannot differentiate iWARP traffic from conventional Ethernet traffic until the packet has been cracked at the TCP layer. At that point, if the packet is conventional Enet, and not iWARP, it is effectively forked off to another process running a standard network stack. I would guess that this would make it challenging to run both conventional Ethernet (for networking and FCoE) and iWARP for IPC over the same Enet fabric although I believe the iWARP vendors are able to do so.
RoCE, on the other hand, is identified by a unique identifier in the Ethertype field. This means that Ethernet traffic is sorted and handled at the very bottom of the network stack, before any processing has been executed on it. Conventional TCP/IP packets are sent to a TCP/IP stack and RoCE traffic is sent through the RDMA stack .
- NASA boffin: RIDDLE of odd BULGE FOUND on MOON is SOLVED
- Microsoft's Euro cloud darkens: US FEDS can dig into foreign servers
- Worstall on Wednesday YES, iPhones ARE getting slower with each new release of iOS
- Pic 7 Amazing Experiments on NASA's 2020 Mars Rover – Including Oxygen Generation
- Plug and PREY: Hackers reprogram USB drives to silently infect PCs