Though I'd rank em all much lower than what Gartner rated them.
Wake me up when
(admittedly I haven't used amazon in well over a year at this point but...)
* Get rid of (or provide the means to) the "built to fail model"
* Your IaaS is billed based on what you *use* not what you provision (Amazon and others do have usage based billing on some/most of their PaaS stuff)
* You have common storage/memory/compute pools which you can provision from/share resources between
* Support thin provisioning and thin reclamation (sort of ties into first two points)
* Provide vMotion support (goes to the built to fail model)
* You get consistent & persistent IP addressing + NAT both internally and externally - having to tell partners to open their firewalls to 50 different IP addresses (even if they are "elastic") is pretty piss poor.
(IPv6 die hards must really hate amazon as they are a huge waster of IPv4 addresses)
* Maintain internal and external IP reservations whether the VM is turned on or off
* Load balance internal resources (goes hand in hand with static internal IPs) - their shitty ELB product could only LB external IP addresses (even if it could do internal it's feature set sucked. I used Zeus instead, which worked quite well, though the costs multiplied quickly, I think there was something like a 4-6 month ROI for buying a real hardware LB vs using Zeus in Amazon at the time -- on the order of something like $4-5k/mo for a redundant pair which was limited to a single internal and single external IP address).
* Eliminate the "your ELB IP address may change at any time" rule, IPs should not change, if there is a failure the backup should do IP takeover and keep going. If you want to see an example as to WHY this is look no further than Java's default DNS caching policy of *FOREVER* - so yeah - go fix every Java VM on the planet..or deploy proper failure infrastructure. BTW using Zeus for this worked fine - it integrates with elastic IPs and performs IP takeover, though for load balancing internal resources (e.g. a pair of mysql databases) the internal IP would not be preserved and the application config (or DNS) would have to be updated in the event of a Zeus LB failure because there was no means to perform internal IP takeover.
* Provide remote access to the *console* of the VMs - last time I used EC2 at least you could sometimes MAYBE get *output* from the console, but seems more often than not the data in the console output was obsolete. But not control of the console, as in be able to login and fix something rather than try to reboot and hope for the best.
* Be able to install an OS using an ISO image file (or remote virtual CDROM)
* Be able to provide the customer with a private PXEBOOT environment (for me and I suspect several others kickstart is preferable to image-based installs)
* Get full insight into underlying host performance, seeing 20-30% "CPU STEAL" from the linux kernel means the host is oversubscribed
* Get full insight into which underlying host(s) are being used (useful at least for software licensing purposes for things that are socket or hardware server based - you can save a lot of $$) - rack or row information would be useful too for availability purposes.
(I could go on I think I'm just scratching the surface here)
* Do it for less $$ than I can do it myself (even Amazon's 3 year reserved pricing is absurd - a big part goes to the lack of resource pooling) - the big players have big purchasing power that small companies don't have.
Remember this article/report is calling out ENTERPRISE USAGE. Not your fly by night web operation that will implode in 60 days and lay everyone off because your idea didn't work out.
Though I suppose I could settle for just the last one, paying much less for a shitty service is one thing, but paying WAY MORE for a shitty service is just doubling down on complete shit.
I've talked to a few other enterprise cloud players -- some of them have some of the features I outlined above though their cost models are even more out of whack with reality. One such company wanted to charge the company I was at a few years ago $3 million to install a cloud infrastructure(+$120k/mo) -- something that was going to cost me ~$700k + $20k/mo. That 700k was buying all tier 1 hardware with 4 hour on site support and premium vsphere enterprise plus at a tier 4 data center with fully redundant everything. Their other option which had a $10 installation fee(!!!!!) was $272,000/mo.
Another enterprise player wanted to charge (again, a few years ago, though I have talked to other folks recently and the situation really hasn't changed much) ~$200,000 for 20TB of tier 2 enterprise storage for *1 year* (so roughly 17k/mo). I could of bought 20TB of *tier 1* storage from the *same storage company* (the reason why we quoted from this cloud player is they used this particular storage company) the cloud provider was using for slightly less money - obviously the savings would skyrocket after the first year. Not to mention my quotes were for tier 1 and this cloud company was quoting tier 2. In talking with said storage company the response was they hear that a lot, IT directors would quote out some storage to buy, CxOs would say "oohhh too expensive!! go cloud!!", said directors would go to cloud and find out the cost is 3-5-10x the purchase price and then they would feel uncomfortable goin back to said CxO and say "hey look the cloud is 10x more expensive then doing it ourselves" -- the answer didn't feel right (having been in that position on several occasions I know that feeling precisely).