Lies, Damn Lies, and Benchmarks
-Disclosure NetApp Employee Opinions are mine, not my employers-
Mr. Hollis has never been much of a fan of SPC-1, or at least the uses to which it has been put by various vendors including that of my employer. I'd be more impressed if Oracle put up an SPC result, because most of the Oracle ZFS array performance comes from Oracle throwing LOTS of cheap CPU and RAM into their arrays, not the most efficient use of resources, but if they're cheap enough, most people don't care. Pot meet kettle ... you both appear to be covered in soot.
In any case, benchmarks like SPC-1 can be remarkably useful tools, even when wielded by a Vendor .. though it's usually worth noting that the benchmark usually proves a couple of interesting engineering points, which marketing usually spins into as much gold as they can (that's their job).
That data core can stitch together 72 CPU cores to get 5M sub 1ms IOPS without much if anything in the way of data services, is an impressive piece of engineering, I know of other more efficient IOPS/CPU core numbers out there, but that still stands as a great result.
It doesn't invalidate the benchmark, which is still remarkable in its disclosure rules and verifiability, but it does mean that people need to look a little more closely at that full disclosure report to figure out how useful / applicable that result is to their needs. Being #1 and having bragging rights for a few months is cool, but having the best overall fit for a customers needs is still probably more important.
Biting the hand that feeds IT
