You want to play these games? Fine, lets play.
Kaminario's claims described in this article are so ludicrous and in some cases blatantly false it is hard to know where to begin, lets start with the simplest.
The word "system", Kaminario claims their product is a "scale out" not "scale up", in which case when you offer a small product, that, is a single system, and when you offer a full rack product, that, is many systems, plural, if it is one system, then you are scale up not scale out. So Kaminario's results are not for a single system all the numbers they boast about are for a multi-system configuration. We will get into this and see that all their claims are backward, they trail rather than lead in all the configurations they have chosen to discuss.
Then there is the issue of latency..
> The gloating Kaminario also said the system's latency was less than 1 millisecond,
> claiming this was "four times better than the closest competitor".
They don't name the supposed "closest competitor", but I will volunteer myself in that role given that they make plenty of comparisons to our product later on. Now they don't say how much lower than 1ms their latency is, but if we look at the SPC-1 results we see the K-2 at 95% load with 3.7ms read latency, the only results less than 1ms are for the 10% Load Level Test Run where their latency is 420us. A Violin array under similar load percentages would have latencies of around ~800us and ~100us respectively and wouldn't have any of the >30ms latency spikes the K-2 displays. So It appears Kaminario meant to say they were "four times worse than their leading competitor".
It is worth noting that the K-2 SPC-1 benchmark configuration was an entire rack at $490,000 with a whopping 1.159TB of storage, that's a "." not a "," so that comes out to ~$300/GB by my calculation. I'm not going to discuss exactly how many times worse than their leading competitor this is, lets just say its a lot closer to 40X than 4X.
>"This benchmark reinforces the importance of a … scale-out architecture compared to the
> proprietary scale-up systems of TMS and Violin. [It] allows our customers to buy one K2
> and grow it as their needs grow. The equivalent benchmark by Violin required two SLC
> (faster single level cell flash) systems; we scaled to 2M IOPS and 20 GB/sec throughput
> with one MLC system," the firm responded.
Again, if you are a scale-out architecture then your rack is a collection of systems, its hard to say how many systems one should view the K-2 rack as being, 3?, 15?, 33?, 45? but a single system it is not. But that aside lets consider a rack's worth of 12 V6616's, and remembering that their "benchmark" is pure random read, so the rack of SLC 6616's gives 15M IOPs, 60GB/s throughput from 120TB of capacity. Or 7.5X the IOPs, 3X the throughput and 2X the capacity. If we consider a rack of 12 MLC 6632's we would see 12M IOPs, 48GB/s of throughput and 240TB of capacity or 6X the IOPs, 2.5X the throughput and 4X the capacity.
So all this benchmark shows is that in all possible ways the performance and capacity of the K-2 lags far behind the products of their leading competitor.
> If you are going to Oracle OpenWorld over the next few days (it starts today and runs
> until 4 October), you can see the K2 in action, presumably spitting through complex
> Oracle database applications as if they were trivial spreadsheets.
If you are going to OOW over the next few days certainly stop in and see a collection of K-2 systems in action giving 2M IOPs from a 40U rack, which is only 7X worse than their leading competitor, then walk 10 feet across the isle to said competitor and see 2 V6616s giving 2M IOPs from 6U. And read about how that configuration produced the most recent World Record TPC-C benchmark on Oracle, which will will assume is like spitting through complex Oracle database applications as if they were trivial middle school algebra homework.
Jon Bennett
Founder & CTO
Violin Memory Inc.
Biting the hand that feeds IT
