It's Not About the Media
Chris,
I couldn't agree more with you sentiments around the 4th stage. While most every solution that is developed goes thought some evolution of innovation, when it comes down to it, the clients needs are the same. They need highly reliable, as fast as practical, easy as possible to manage, scalable storage solutions. And in that "as easy as possible to manage" comes the ability to integrate with applications that we use every day to make our lives easier. (To the comment above where it is mentioned that "it is difficult for a storage provider to integrate with VMware" - it really isn't, we have done it at INFINIDAT and we are helping make customers backup lives, for example, 10,000x easier with integration into vCenter and at no additional charge by the way.)
In order for storage to get 'better' its not about the media. It is about 4 fundamental aspects of storage to help clients not have to compromise on performance, reliability, and scale - which means you get all of these features for one low price (along with all the storage features you need). In order to do this you need:
1) A server to disk ratio that is beyond what systems do today - it needs to scale way beyond the 24 to 1 ratio of drives to CPU in today's hyperconverged space, the 12 to 1 XIV uses, and the 4 to 1 in what google is building for their scale, - having less cpu, power supplies, etc to drive the disks means less space, less power and cooling and fewer components which all help to have a much better TCO.
2) System efficiency needs to be much better than it is today - having traditional RAID which using as much as 50% overhead or more and RS coding which drives down performance is not going to help clients get what they need. RAID means more drives for failure scenarios and as data scales so do the number of drive and hence the number of failures.
3) The ability to take advantage of high capacity drives. I should state in such a way that you can achieve good performance - both for reads and writes but also for when there is some sort of a drive failure. It is a little known fact that the vendors who make spinning media are still investing a great deal in these devices to help client store more, for longer periods of time on line.
4) The ability to best take advantage of the ratio of flash to spinning media. When done correctly systems can achieve the necessary performance for the applications without breaking the bank.
These fundamental concepts are hard for traditional storage vendors to think about without completely re-architecting their systems which is too costly for them to do. However once a system like this is available, you can then start thinking about the integration with the applications and all of the things a system like this is able to do.
In addition as data continues to scale, which it will, without these new fundamental concepts all of the traditional storage systems, REGARDLESS of media type, break down. So managing data at scale requires so fundamental rethinking of traditional storage... Slapping some 'older' software (XIV) behind some flash / SSD disks is NOT going to help customers.
New scalable systems that work more tightly with the applications like VMware or especially now OpenStack are more and more important. It is time to help clients be competitive - what does that mean, it means stop charging them millions of dollars for disk and help them figure out how to use that money for smart people that can collect more data, analyze more data and build new businesses - that is what will help.
Biting the hand that feeds IT
