Connecting things properly
I'd much rather see USB replaced by Ethernet with a connector that can take a decent power level, has an obviously directional shape, and is genderless (like an Anderson power connector, but smaller and with more than two contacts). Then you won't get distinctions like "A" and "B" connectors, or like USB-on-the-go being different from USB; things can just connect, and let the software/firmware sort out whether they have anything to talk about. NICs are cheap now; from a quick look at the protocols, I think they probably take less silicon to implement, too, and you'd probably get far better performance for most things without the grossly inefficient USB protocol.
For a little extra, make the connection reasonably mechanically robust and it can even hold devices onto each other (e.g. bluetooth/wifi dongles onto phones/computers) without worrying about bending connectors.
Vendors could add value to the products a bit, then we can let market forces push the prices down; tiny NAS units could take the place of USB flash drives, wifi dongles could be routers, and so on.