Re: "Linux UI"
> Linux doesn't have a UI, unless you count printk.
> The rest is actually completely unrelated to Linux,
Yes. But all of that stuff can be built for Android just as the Mac desktop could be built for PhoneOS. There's no reason that a phone couldn't run a conventional desktop if the right inputs are attached. If the device is fully in your control, that becomes a lot easier.
People already jailbreak their devices or install SSH servers on them in order to treat them more like old school Unix servers.
Running conventional desktop software is just the next logical step.
An X server is perhaps the wrong thing for an Android device. However, X libraries would allow any other X server to connect to your Android phone and login to it graphically from across the network.