Thanks for this great analysis of my last blog post. I can honestly say that I agree with almost all of your analysis, but would like to add two additional ‘layers’ that are at the core of the paradigm shift I see in the market.
The first layer of the new hardware / software paradigm shift, as I see it, is not about the location of the CPU, but is about the location of the services and management. This can be demonstrated by the Cisco Nexus virtual switch. It doesn’t try to save CPU cycles, to replace the physical switches (obviously Cisco wouldn't want that), or even to commoditize them, but instead the Nexus switch provides a better network framework that is completely aligned with the virtualized environment, supporting the flexibility and mobility of the ‘new’ virtual infrastructure, together with all the right interfaces to the physical switches. That’s the key. When you are hypervisor resident, you can support things like VM vMotion and storage vMotion, without requiring any reconfiguration or complex management tasks.
Physical storage systems have additional severe limitations when working with virtualized environments, several of them cannot be solved by just better integration with VMware. For example, check out VMware's own SIOC functionality. It guarantees responsiveness for different VMs, even if they are running on the same disk. This functionality cannot reside in the storage array (or storage virtualization appliance, even if it is a VM) since the array cannot differentiate different IOs from different VMs. This is the reason VMware provided SIOC to solve this problem for their customers. Talking to customers using virtualization, we recognize several additional problems that they struggle with on a daily basis and cannot be solved within the arrays.
The second layer is complexity, driven mostly by manageability. When talking to our design partners and prospects, I see a real pain developing in virtualized infrastructure around complexity, flexibility, and support for truly dynamic IT. Storage virtualization appliances, physical or virtual, do not solve this pain. These appliances are a step away from "hardware", or proprietary architectures, but they do not complete the paradigm shift brought by virtualization and clouds. They still operate within the same storage constructs as the physical arrays.
We are not trying to reinvent the wheel, and definitely not invent a new form of locomotion :) I leave that to scientists and people smarter than me. All I am saying is that the paradigm shift caused by virtualization is creating new storage pains that will require new solutions. At the end, the solution makes the difference, not the architecture. The IT world IS changing.