But did they actually test them?
Otherwise this is just marketing puffery.
Who's the mid-range deduplication boss? ExaGrid is, beating Data Domain and Quantum appliances in a survey ranking 37 dedupe appliances. ExaGrid's EX1300E has scooped the mid-range dedupe pool, being named Best-in-Class in a DCIG survey comparing appliances priced between $20,000 – $100,000, and beating Data Domain and Quantum …
Otherwise this is just marketing puffery.
As you probably already realize after now having time to read the Buyer's Guide, DCIG does not test the products covered in its Buyer's Guide (which is disclosed in the Buyer's Guide) due to the cost, time and effort involved with physically testing this many products and then still producing results that are relevant. Further, it is arguably impossible to come up with an objective means to test this many products and then still produce results that are any more meaningful or relevant than what you will find in this Buyer's Guide.
In terms of "marketing puffery" as you refer to it, this Buyer's Guide is a 62-page report with in-depth technical analysis and details about each deduplication appliance. Of course, after reading the Buyer's Guide, you probably now realize that and will come to the same conclusion that most who read DCIG Buyer's Guides reach: DCIG produces the most exhaustive Buyer's Guide on different topics that you will find publicly available.
You can find all of them here:
"The report has been effectively bought by ExaGrid and made publicly available. DCIG says that, although it has paid relationships with some of the suppliers mentioned, none of them paid for entry in the report or for any influence whatsoever in its preparation and production by DCIG."
So basically DCIG did the work up front knowing that a smaller company looking to get market share and a few reports under its belt would pay for the publishing rights. No doubt it was all agreed beforew they even started writing it.
'Oh we didn't pay for the report, honest! We just paid this huge publishing fee to be able to share it with all of our potential customers!!'
Do these companies honestly think it makes a difference in our eyes?
A little testing goes a long way, inline vs post process ?
Did you happen to read the testing environment on page 3 of that Network World article which you reference? If not, here it is:
"The test bed consisted of Windows 2008R2 server connected to two Fibre Channel volumes (actually snapshots of the same 600GB volume about four months apart), running Symantec Netbackup 7.0. Each appliance was used to create a backup of the first volume, a second full backup of the first volume (which should have used very little additional space, since all the files were the same), and then a full backup of the second snapshot of the volume, which had 4552 files either changed or added, totaling about 32GB. (See how we conducted our test.) "
One Windows server with 600 GBs? That is not an enterprise test bed. That is barely a PC test bed. This is why DCIG does not attempt to test the 30+ products contained in its Buyer's Guide. As this article and test illustrates, it adds very little value to the buying decision process.
DCIG Buyer's Guides provide a much larger, more comprehensive examination of a much larger number of features on many more products than anyone writing an article can ever hope to accomplish. These are all factors that enterprises should consider when making a buying decision and which are intended to help guide organizations to making an informed decision as to the options that are available to them.
Never said it was an enterprise test bed, but your precious Exagrid took nearly 3 hours to backup and dedupe the data set (slowest of all tested), whereas the competition completed the process in less than 30 minutes.
BTW having previously read your mid range array buyers guide I don't value your feature comparison methodology. Knowing in detail some of the products you compared, I can state many of your conclusions were inaccurate. I don't blame DCIG for this, it's largely due to the marketing spin being fed to you by vendors.
As you should know the Devil unfortunately is always in the detail, especially around features and the more isn't always the merrier.
I could not agree more - the devil is in the details. However, you need to consider the details you are quoting to refute my argument.
ExaGrid may have taken 3 hours to backup and deduplicate, but it still may have outperformed its competitors, depending on what benchmark you are looking at. As you are probably aware, inline deduplication solutions deduplicate little or not at all on the first backup so even though they completed in 30 minutes, how much deduplication really occurred? Second, on the second pass, if all of the data is the same or very close to the same, it may only be indexed and then discarded. So in this sense, yes, ExaGrid lost.
But before we throw ExaGrid under the dedupe bus, why did it take 3 hours? It is possible, because it does post-process deduplication, that it was actually deduplicating the data that was backed up?! What a concept! A deduplication product that deduplicates data!
So it raises the question, who "performed" the best? The one that completed the backup the fastest? (And both products may have actually completed the actual backup in the same amount of time - I would need to check on that) Or the one that deduplicated data after the first pass and used less storage capacity?
So maybe ExaGrid did outperform its competitors based upon this benchmark but maybe the tester was predisposed to favor in-line deduplication or he simply did not have time or think to look at it from this perspective. Who knows for sure?
This is why I wrote a blog entry over on my site this morning discussing the issues associated with product testing and why DCIG only evaluates features and does not test products in its Buyer's Guides.