* Posts by DCIGJerome

20 publicly visible posts • joined 12 Sep 2011

DCIG mid-market array guide: Why we left those companies out

DCIGJerome

Re: Maximum raw capacity of no more than 500 TB

Chris,

Good, constructive feedback and thanks for taking the time to share it. However, you suggest we should include all models that scale past 500TB. Well, how far past 500TB? 750TB? 1PB? a billion PBs? What would you do?

It appears people are reading more into this Buyer's Guide Edition that it was intended to convey. This Edition is intended for those organizations that want to acquire a storage array that only needs to scale to 500TB to meet the needs of their environment. If this is your use case, this Buyer's Guide is for you and these are the best models to do so based upon our research and opinion. If not, we suggest you leverage DCIG's online research to find the best storage array for your environment.

DCIG has the capability to produce other Buyer's Guide Editions that reflect other use cases using the same body of research into enterprise storage arrays that was used to produce this one. The results contained in each one will depend upon the use case and inclusion/exclusion criteria applied to it. As DCIG publishes each Guide Edition, we disclose how we arrived at our conclusions. In this way, individuals can make an informed decision if this particular Buyer's Guide Edition is the right one for them to use in their environment.

DCIGJerome

First, I am not sure DCIG can “fix” this (by which I assume you mean the unwillingness of vendors to publicly share prices) or that DCIG should even necessarily try to fix it. Buying a storage array based on price is very difficult to accomplish since there are so many hardware and software variables involved in the configuration of an array that organizations have control over.

Second, comparing a storage array purchase to a car purchase is an apples-to-oranges comparison. Unless people order a new car, their options are largely already fixed for them when they buy the car.. Further, a car is only designed to have one driver and a few passengers. Storage arrays may be administered by one or more people and host a single app or dozens, hundreds or even thousands of apps. Due to these differences, in the use case of buying cars, price becomes an obvious point of comparison and differentiation. To arrive at a price for a storage array, many more variables generally enter the equation that must be determined before one arrives at a price.

Third, feature evaluation is the only metric that we have found to objectively select products for inclusion or exclusion in a Buyer's Guide Edition. While it may be “shocking” DCIG takes such an approach, taking any other approach often becomes too subjective to arrive at an objective conclusion.

DCIGJerome

Re: Maximum raw capacity of no more than 500 TB

I agree that this report does not cover arrays that scale past 500TB nor did it intend to cover them. DCIG may produce other Buyer's Guide Editions that will cover those models. DCIG can create multiple views into its enterprise storage array body of research based on different use cases that may result in Buyer's Guides. This is one such example of a use case.

DCIGJerome

DCIG has been all-inclusive in its Buyer's Guides in the past such as you suggest. Unfortunately, due to the number of arrays that DCIG would cover, people could not easily recognize the arrays that were best suited for specific use cases, such as those best suited for mid-market organizations. As a result, DCIG adopted a more exclusive approach in developing and releasing its Buyer's Guides so the number of models included in them are more limited than in the past.

Any reader who would like to make a fully informed decision about buying a storage array may subscribe to the DCIG Analysis Portal and access the DCIG body of research on enterprise storage arrays. The DCIG Analysis Portal has information on literally hundreds of different storage arrays from dozens of vendors. Further, this Portal includes much more product detail than what DCIG publishes in its Buyer's Guides.

DCIGJerome

The generalization that all arrays that scale to more than 500+ TB cost more is not true.

However, the use case as it pertains to this Buyer's Guide concerns hybrid storage arrays. In this use case, I would suggest those hybrid arrays that scale to higher capacities cost more than those that do not. Again, this would likely depend on the number of SSDs and HDDs that are part of its scale-out and/or scale-up configuration..

DCIGJerome

DCIG has actually tried to use list prices to create Buyer's Guide editions in the past. But list prices are usually far above their street price and very few vendors willing share either their list or street prices, especially if they know these prices may be, in turn, shared publicly in a Guide such as this.

The price of an array is often largely determined by the configuration a specific client orders or needs. As such, the price for an array can vary greatly, even on the same model.. Agree or disagree with DCIG's criteria in how it created this Guide, they are what are.

I am unclear what you mean by "pass a sniff test." What is the criteria "to pass a sniff test?"

DCIGJerome

Thoughts on Including All Arrays that Scale from 100 - 500TB

The concept of including all arrays that scale from 100-500TB regardless of how high they scale as the author of this article suggests sounds noble on the surface. Unfortunately that logic quickly breaks down as literally dozens of arrays fall into that category. DCIG has other Buyer's Guide Editions that will address the markets for which those array are intended.

DCIGJerome

Re: DCIG Validated Itself

DCIG simply disclosed how it arrived at its results in this Buyer's Guide Edition. After all, these are storage arrays, part of their value proposition is storage capacity, and this is certainly a criteria that organizations evaluate when buying storage arrays. We think organizations do not act wisely if they do not evaluate storage capacity as part of the storage array buying process.

DCIGJerome

Re: Maximum raw capacity of no more than 500 TB

DCIG needed objective criteria by which it could justify the inclusion and exclusion of certain arrays in this Buyer's Guide edition. This Buyer's Guide Edition clearly sets forth those criteria. At no point in this Buyer's Guide Edition is any guidance given to storage providers as to how to size or sell their arrays.

DCIGJerome

Re: Suprising Results a Result of Inclusion/Exclusion Criteria

Rest assured, there was no secret list of vendors for inclusion or exclusion. These are simply the products that met the criteria as defined and shared in this article and in the Buyer's Guide itself.

However, anyone (vendor, reseller, or end-user) who subscribes to and uses the DCIG Analysis Portal can dynamically apply their own criteria to DCIG's body of research into enterprise storage arrays. They may then immediately see what the outcome is based upon that criteria.

When the particular set of criteria as disclosed here were applied to DCIG's body of research on enterprise storage arrays, DCIG got these results.

The DCIG Analysis Portal is a very powerful tool and it may provide outcomes that even the writer of this article was surprised to learn. The difference is, he had to go and spend time to validate DCIG's research. DCIG gets these results with a few mouse clicks which it then publishes as Buyer's Guides.

This approach is better than using a spreadsheet, faster than trying to do it yourself and much more reliable. In short, this is the application of analytics to Big Data and represents the future of analysis. Pretty slick, huh?

DCIGJerome

Re: well that was as useful

The inclusion/exclusion criteria was included in the original report and it was fully disclosed. Please check your facts before making daft comments,.

DCIGJerome

This Buyer's Guide Edition did not attempt to answer that question as you pose it. DCIG would generally agree with the sentiment that organizations will rule out a solution solely because it scales out to a greater capacity. But if a mid-market organization needs 500TB or less of capacity, why pay for a storage system that scales out (or up) to a greater amount? Conversely, if a mid-market organization gains access to a storage system that scales out or up over this amount for the same or a lower price, it may make sense for it to acquire that system in lieu of the one that only scales to a lower capacity.

Mid-range storage array buyers' report leaves out .... guess who?

DCIGJerome

Re: Midrange Array Capacity is a Key Qualification Criteria

Not assessing capacity when reviewing midrange arrays makes no sense. Capacity is absolutely an evaluation criteria. If an organization's only concerns are performance and cost, it should buy an SSD and put it inside the server. The SSD will likely perform faster and cost far less than a midrange array.

DCIGJerome

Re: Not Flawed but Objective Assessment

DCIG uses raw capacity as a measurement as it is one of the few capacity metrics that DCIG can reliably and objectively assess. Metrics such as effective or usable capacity are influenced by a number of variables that DCIG finds subjective. Usable capacities can vary greatly between arrays depending on the data protection methodology applied to the flash and/or hard disk drives they use.

NTAP was simply not covered because none of its array models satisfied the inclusion/exclusion criteria for this particular Buyer's Guide Edition. NTAP models will appear in other forthcoming DCIG Buyer's Guide Editions that are based upon DCIG's research into enterprise storage arrays.

DCIGJerome

Re: Gun for hire

Have you ever read one of the DCIG Buyer's Guide Editions and examined at the tremendous amount of detail and research that goes into each one? These are very credible and influential analyst reports.

DCIGJerome

Re: Flaw is in the qualifications

This Buyer's Guide Edition does not cover the HPE 3PAR for the reasons you cite. However the HPE StoreVirtual 4335 Hybrid model is covered as it is included in this Buyer's Guide Edition.

DCIGJerome

Re: Response from a co-author of the DCIG Buyer's Guide

DCIG clearly discloses the different ways it generates revenue on its webstie. These options may be viewed on the DCIG website (dcig.com) under the Analysts Product subheading that appears at the top of the home page. DCIG provides both Sponsored and Unsponsored Analysis.

ExaGrid scoops the dedupe pool

DCIGJerome

Anonymous,

I could not agree more - the devil is in the details. However, you need to consider the details you are quoting to refute my argument.

ExaGrid may have taken 3 hours to backup and deduplicate, but it still may have outperformed its competitors, depending on what benchmark you are looking at. As you are probably aware, inline deduplication solutions deduplicate little or not at all on the first backup so even though they completed in 30 minutes, how much deduplication really occurred? Second, on the second pass, if all of the data is the same or very close to the same, it may only be indexed and then discarded. So in this sense, yes, ExaGrid lost.

But before we throw ExaGrid under the dedupe bus, why did it take 3 hours? It is possible, because it does post-process deduplication, that it was actually deduplicating the data that was backed up?! What a concept! A deduplication product that deduplicates data!

So it raises the question, who "performed" the best? The one that completed the backup the fastest? (And both products may have actually completed the actual backup in the same amount of time - I would need to check on that) Or the one that deduplicated data after the first pass and used less storage capacity?

So maybe ExaGrid did outperform its competitors based upon this benchmark but maybe the tester was predisposed to favor in-line deduplication or he simply did not have time or think to look at it from this perspective. Who knows for sure?

This is why I wrote a blog entry over on my site this morning discussing the issues associated with product testing and why DCIG only evaluates features and does not test products in its Buyer's Guides.

http://www.dcig.com/2011/09/product-testing-has-its-own-set-of-issues.html

Jerome

DCIGJerome

Read the Article

Anonymous,

Did you happen to read the testing environment on page 3 of that Network World article which you reference? If not, here it is:

"The test bed consisted of Windows 2008R2 server connected to two Fibre Channel volumes (actually snapshots of the same 600GB volume about four months apart), running Symantec Netbackup 7.0. Each appliance was used to create a backup of the first volume, a second full backup of the first volume (which should have used very little additional space, since all the files were the same), and then a full backup of the second snapshot of the volume, which had 4552 files either changed or added, totaling about 32GB. (See how we conducted our test.) "

One Windows server with 600 GBs? That is not an enterprise test bed. That is barely a PC test bed. This is why DCIG does not attempt to test the 30+ products contained in its Buyer's Guide. As this article and test illustrates, it adds very little value to the buying decision process.

DCIG Buyer's Guides provide a much larger, more comprehensive examination of a much larger number of features on many more products than anyone writing an article can ever hope to accomplish. These are all factors that enterprises should consider when making a buying decision and which are intended to help guide organizations to making an informed decision as to the options that are available to them.

Jerome

DCIGJerome

Gordon,

As you probably already realize after now having time to read the Buyer's Guide, DCIG does not test the products covered in its Buyer's Guide (which is disclosed in the Buyer's Guide) due to the cost, time and effort involved with physically testing this many products and then still producing results that are relevant. Further, it is arguably impossible to come up with an objective means to test this many products and then still produce results that are any more meaningful or relevant than what you will find in this Buyer's Guide.

In terms of "marketing puffery" as you refer to it, this Buyer's Guide is a 62-page report with in-depth technical analysis and details about each deduplication appliance. Of course, after reading the Buyer's Guide, you probably now realize that and will come to the same conclusion that most who read DCIG Buyer's Guides reach: DCIG produces the most exhaustive Buyer's Guide on different topics that you will find publicly available.

You can find all of them here:

http://dcigbuyersguides.com

Jerome