And then there is security
While providing an abstracted view of the hardware, an operating system also brings some baggage. One bag contains security holes that the applications that run upon the operating system do not have.
The same is true for hypervisors. The more features that are integrated into the hypervisor such as management facilities, the more vulnerabilites arise. An operating system places a bunch of applications at increased risk and a hypervisor places a bunch of guests and even more applications at risk.