" It would be nice to have generic instructions"
It just hinders innovation (or can't use it) in the underlying hardware level. The translation between "generic instructions" and actual CPU ones could become really hard, and sub-optimal, when architectures becomes very different. They may require different higher-level programming patterns to choose the best approach for a given problem on a given architecture, not just some late compiler selection among different instruction sets.
LLVM is a "lazy" approach - perfectly fine with less demanding applications where it simplifies a lot - but it will hit roadblocks for more demanding ones where it's difficult to abstract fully the underlying architectures, the risk is to adopt a model where only common features are available - so you don't get anything better than Java...