>they can get the code from the chip makers directly
Why would they need the code from the chip makers? They would give the chip makers an API and they would supply blobs that implement it just like they do now.
>(which they MUST provide to get their stuff working on Android in the first place),
They don't. At the moment AOSP provide reference implementations and vendors replace them with their own implementations.. this would not change if Android went closed. Google would provide vendors with a development kit that looks exactly like AOSP does. Google wouldn't suddenly be getting the source for a bunch of drivers which they have no hope of maintaining.
>going around the phone makers who aren't motivated due to market pressures.
If you think google would take source for proprietary GPUs etc and maintain it for vendors then you are a complete loon.