Infinidat claims it can beat any all-flash array, uses innocent pooches to appear convincing

Re: It's called the SPC1

So, assuming you can't bring a dozen different array vendors in house to test them with your workloads, what other options are there other than widely used benchmarks. Shit some storage companies don't even do evaluations (NetApp being one, from at least 2006 until 2011 they(multiple people at NetApp explicitly told me they do not do evals -- maybe that has changed for solidfire etc). NetApp refusing my requests for an evaluation back in 2006 is why I ended up on 3PAR.

I totally agree SPC-1 is far from perfect, as are most(all?) benchmarks, but it's a lot better than nothing. The disclosures in the benchmarks like SPC-1 are very informative (e.g. showing that the workload in question is running entirely out of cache, as Infinidat likes to show, or if high availability is disabled on the array(as I have seen on one or more 2nd/3rd tier vendor arrays in the past). At least the benchmark gives a common workload that people can use for comparison at some level. I would like to see SPC-1 revised though. I haven't paid attention to it in a while(haven't paid attention to storage in general for a while my arrays just happily run along).

Bringing everything in to evaluate with real workloads is just not realistic for 99% of customers. I know there is a company out there that makes a high end storage load simulator, forgot their name, but even that seems likely to be only used by very very large customers.

I remember getting a presentation on Hitachi's AMS2000 series back in 2010 I think it was, just before the arrays were coming out(the 2300 came out 2 or 3 months after the presentation), and they had slides showing their systems could do 1 million iops or something like that.. Certainly sounded impressive, but it wasn't until I explicitly asked them that they admitted that was from cache(and obviously the AMS2k series had a tiny cache relative to something like Infinidat). I could absolutely, positively see the VP I had at the time making a purchasing decision with data like that slide in mind(granted he was an idiot, but there are lot of those out there, if I had to bet I would say the majority are). So I'd happily take SPC-1 over a marketing slide like that any day of the week.

I remember another presentation(maybe 2012), it was from X-IO (at the time they were still called Xiotech), they did tout SPC-1 numbers, they showed themselves as #1 for that particular metric, and then they showed 2 or 3 other vendors (they just named them "Vendor A" "Vendor B", did not name the real names). I think the metric was IOPS/disk or something, or IOPS/GB or TB. I recognized the 3PAR result at the time, and asked them, and they confirmed yes the #2 system was 3PAR, I remember my boss(Director, not the VP) saying "oh, looks like we made a pretty good purchasing choice then", thought it was funny. We brought Xiotech when they came up with a trade in program where they give you free storage and you give them old storage. We had about 150TB of old storage from BlueArc (they used LSI storage at the time), so we thought hey maybe we can give this shit to Xiotech and they give us Xiotech boxes for free, but in the end they had no interest in that old storage we had so they went away. Ended up having to pay someone to take the old stuff and toss it out, I had a friend who worked at a recycling company for IT stuff, he initially expressed great interest in those racks of gear, but after I gave him more in depth information I guess they decided there was no value in it after all so they didn't even want to take it to recycle (their thing was give them your old IT shit and you get a tax write off).