Good questions. I will do my best to answer.
"Why should containers do this, surely that is the job of the hypervisor?"
Containers shouldn't do this. They will likely try anyways, as a great many of those who are invested in containers and adjacent technologies want to see them replace hypervisors.
The argument goes that containers are so much more efficient than hypervisors that we should do away with hypervisors altogether and use containers for everything. Based on that, we'll either have to throw all our workloads away and recode everything (not bloody likely, especially since there are any number of workloads that can't be "coded for redundancy" in the container/cloud sense) or containers will have to evolve to add these technologies.
As it stands now, I know a number of startups working to bring hypervisor-like technologies (HA, vmotion, etc) which are in stealth. We are at the beginning of mass market contain adoption, not the end. Thus the technologies enjoyed by the mass market for their current workloads will have to be recreated for previous ones, just as we have for every technology iteration before this.
Does this mean it's rational or reasonable to do so? I argue no. If I run 4 VMs on a node and each node contains 100 Docker workloads, I am giving up a small amount or overhead in order to virtualise those 4 Docker OSes, but gaining redundancy for them in exchange. Meanwhile, I can load up that server to the gills and keep all it's resources pinned without "wasting" much because I'm using containers.
To me, it makes perfect sense to have a few "fat" VMs full of containers. Resilient infrastructure, high utilization. But this is considered outright heresy by many of the faithful, as - to them at least - containers are about grinding out every last fraction of a percentage point of efficiency from their hardware.
"Is it the cost of requiring both that keeps you from using containers 'till the above condition is met?"
No, it is the pointlessness of using both that keeps from using containers. Right now, today, most of my workloads are legacy workloads. They don't convert into these new age "apps" that "scale out", as per docker/public cloud style setups. If I want redundancy, I need a hypervisor underneath.
So, I could do what I talked about above and put my workloads in containers and then put the containers in a hypervisor. That would increase the efficiency of about 50% of my workloads, ultimately dropping the need for an average of two physical servers per rack where a rack contains about 20 servers.
That's a not insignificant savings, so why not jump all over this?
1) Even with the workloads that can be mooshed into containers it will take retooling to get them there. It is rational and logical to move them into containers, but that is a migration process akin to going XP --> Windows 10. It takes time, and is best done along the knife's edge of required updates or major changes, rather than a make-work project on it's own.
2) If I start using containers I need to teach my virtualisation team how to use these containers. That's more than just class time, it takes some hands on time in the lab and the chance to screw it up some. That is scheduled, but I'm not going to adopt anything in production until I know that I can be hit by a bus and the rest of the team can carry on without me.
3) Politics. Part 4 of this series will talk about the politics of Docker. Not to give anything away, but...the politics of containerization is far from settled. I don't want to be the guy who builds an entire infrastructure on the equivalent of "Microsoft Virtual Server 2005" only to have all that effort made worthless a year or two later. Been there, done that.
4) 2 servers out of 20 isn't world-changing savings for me. Oh, that's Big Money if you're Rackspace, but at the scale of an SMB that only has a few racks worth of gear, there's an argument to be made for just eating the extra hardware cost in order to defer additional complexity for a year or two.
Really, in my eyes, it's Docker versus the cloud...sometimes in the cloud.
If I was building a brand new website today, I would have a really long think about Docker. Do I use Docker, Azure, AWS, Google or someone like Apprenda?
The choices are likely to be informed not by the technical differentiators between these platforms, but by business realities ranging from Freedom of Information and Privacy regulations, marketing success around Data Sovereignty, cost and availability of manged workloads.
Do I run my workload in one of the NSA's playground clouds, pick something regional, or light it up on my own infrastructure? Is the particular set of applications I am looking at deploying into my Docker containers available from a source I trust, and likely to be updated regularly, so that I can just task developers to that application and not have to attach an ops guy?
New applications make good sense to deploy into Docker containers. And Docker containers in the hands of a good cloud provider will have a nice little "app store" of applications to choose from.
But if I am lighting them up in the public cloud, do I really care if it's in a Docker container? Those cloud providers have stores of stuff for me to pick from in VM form as well. And I don't care if what I am running is "more efficient" when running on someone else's hardware; that's their problem, not mine.
I'm not against using Docker in the public cloud, but I see no incentive to choose it over more "traditional" Platform as a Service offerings either. If for whatever reason we decide the public cloud is the way to go, I'll probably just leave the decision "Docker/no Docker" up to the developers. The ops guys won't really have to be involved, so it's kinda their preference. I really don't care overmuch.
So from a pragmatic standpoint I really only care about Docker if it's going to run on my own hardware, either as part of my own private cloud, or as part of a hybrid cloud strategy. As we've seen, there are layers of decision-making to go through before we even arrive at the conclusion that a given new workload is going to live on my in-house infrastructure. But let's assume for a moment we've made those choices, and the new workload is running at home.
This is where we loop back to the top and start talking about inertia.
All my workloads are on my own private cloud already. They're doing their thing. If I don't poke them, they'll do their thing for the next five years without giving me much grief. My existing infrastructure is hypervisor-based. My ops guys are hypervisor-based.
If I simply accept that - if I give in to the laziness and inertia of "using what I know and what I have to hand" - then my new applications require no special sauce whatsoever. I can let the hypervisor do all the work and just write an app, plop it in it's own operating system, and let the ops guys handle it. What's one more app wrapped in one more OS?
Change for change's sake offers poor return on investment. So for me to move to Docker there has to be inventive. Right now, today, at the scale I operate, the ability to power down 8 servers isn't a huge motivation. I could write pay for the electricity required to light those servers up by writing 4 additional articles a year.
Two years from now, I may have a dozen applications in the cloud, all coded for this "scale out" thing. I may have gotten rid of one or two legacy applications in my own datacenter and replaced them with cloudy apps. Five years from now...who knows?
It would be really convenient for those new applications to be written to be Docker compatible, scale-out affairs that provided their HA via the design of the application rather than the infrastructure. But I don't know for sure that Docker will be the container company that wins.
For that matter, the hypervisor/cloud companies could see Docker as a threat in the next two years, declare amnesty and agree to a common virtual disk format.
Docker offers a means to make my apps more-or-less portable. Ish. As long as there isn't too big a difference between the underlying systems, they'll move from this server to that one, from private cloud to public. If I kept the OSes at the same patching levels on both sides, I could move things back and forth...though not in an HA or vmotion fashion. That has some appeal.
But a common virtual disk format would allow me to move VMs between hypervisors and from any hypervisor to any cloud. Were this to happen, I'd really lose most of my incentive to use Docker. At least at the scale I operate.
All of the above is a really roundabout way of saying this:
Docker is cool beans for big companies looking to make lots of workloads that require identical (or at least similar) operating environments. (See; scale out web farms like Netflix.)
Hypervisors are just more useful to smaller businesses.
I'm way more likely to care about a technology that lets me easily move my workloads from server to server and from private cloud to public cloud (and back) than I am one that will let me get a few extra % efficiency out of a server. Docker could do this one day. So could hypervisors. Neither really do it today.
Hope that helps some.