Re: Are SPC Benchmarks useful?
The SPC-1 benchmark provides a real source for comparing performance of storage systems. It is a single workload that demonstrates storage system performance under typical critical applications such as OLTP and database operations. As Chris mentioned, the process of submitting guarantees that the results are in fact valid and match to an existing product that customer can buy - not a special configuration that is built only for the purpose of achieving a result. It’s a benchmark running on a valid product and includes all the other elements of the product beyond performance, including high availability and Enterprise storage features that customers expect to get beyond the performance. The process of reviewing SPC-1 results guarantee that the results are accurate and real. An SPC benchmark measurement becomes a new SPC benchmark result upon successful completion of the required SPC Audit and SPC submission process. At that point the new result is in “Submitted for Review” status for a minimum of 60 days during which the SPC Peer Review occurs and allows SPC members an opportunity to review the details of the SPC benchmark result and raise any compliance issues resulting from that review. A result that is approved means that it passed this process. Which is what happened, when we, (full disclosure here, I run engineering at Kaminario) Kaminario submitted and won the world record title that Chris referred to in his article. There is no other storage performance benchmark that guarantee this level of validation. Oh, and p.s. we’re still the record holder.
Biting the hand that feeds IT
