What the foo?
'"There is definitely demand for 10GE, but it has been limited by high cost," Gilad Shainer, '
This guy gets paid to figure that out?
Mellanox Technologies is not happy about the 10 Gigabit Ethernet switch and adapter ramp, and it is doing something about it: slashing its own prices. In the wake of its acquisition of InfiniBand switch-maker Voltaire, Mellanox last year launched its SwitchX converged Ethernet and InfiniBand ASICs for 10GbE and 40GbE switches …
4x (ie 10Gb/s) dual port IB cards can be had on ebay for less than $50. So a point to point network between 3 or 4 machines with IPoIB and some routing magic is not expensive.
I did this with three such cards and three PCs. I measured 5.8 Gb/s testing with netperf, and upwards of 300 MB/s NFS file server operations (limited by my RAID array).
I do not run MPI code at home, so I think getting DDR or QDR cards would be overkill, but 10Gb/s IB proved to be very enabling, very much faster than gigabit Ethernet.