Didn't microsoft try that?
Moving video drivers out of the NT kernel and into user-space to speed things up?
Whatever happened with that ... Oh, yes, I remember now. Massive security holes.
And the MS attempt was under the watch of Dave Cutler at the time. If HE couldn't make it work, I'm absolutely certain Auntie Beeb and some wet-behind-the-ears kids paying rent in SanFrancisco (instead of a mortgage) can't make it work.
But what do I know. I've only been in the business for over 4 decades.