Per port costs on 10Gbe are very, very, high. We're configuring some storage aggregation scenarios and it's one of the few cases where hardware (vs. software and maintenance costs) really varies.

Keep in mind that CX4 is limited to a very short range for 10Gbe (but not 4 times as fast infiniband) and then price some per port costs for 10Gbe that include the optics and NICs.

And from a pure (predictable) latency perspective native Infiniband (RDMA) will beat 10GigE everytime. Leaving aside the cheaper cost per port/higher throughput.

I'm sure ethernet will catch up - but we're a few years away from anything production ready.

With Infinband already at 40G and FC at 16G, the Ethernet meat grinder needs to quickly increase its grinding wheel to adopt 40G Ethernet and not just sit back with the promise of 100G sometime in the future. We've seen this over the last decade where Fiber Channel has embraced higher speeds in a more incremental fashion (1, 2, 4, 8and now 16Gbps) at a much lower cost point than the "less flexible" Ethernet, which just offered 1Gbps and 10Gbps (take it our leave it).

The Ethernet community need to be more "flexible" to the faster changing needs of Data Center SAN's, they've made a good start agreeing to standardize 40G Ethernet (though with a few cries of heresy) and adopting the same physical layer as Infiniband (i.e ability to use the QSFP copper/optic modules). But we have to wait and see if Ethernet vendors offer this flexibility to customers or if they insist on sticking to the 10x dogma.

With Infinband already at 40G and FC at 16G, the Ethernet meat grinder needs to quickly increase its grinding wheel to adopt 40G Ethernet and not just sit back with the promise of 100G sometime in the future. We've seen this over the last decade where Fiber Channel has embraced higher speeds in a more incremental fashion (1, 2, 4, 8and now 16Gbps) at a much lower cost point than the "less flexible" Ethernet, which just offered 1Gbps and 10Gbps (take it our leave it).

The Ethernet community need to be more "flexible" to the faster changing needs of Data Center SAN's, they've made a good start agreeing to standardize 40G Ethernet (though with a few cries of heresy) but we have to wait and see if Ethernet vendors offer this flexibility to customers or if they insist on sticking to the 10x dogma.