It's still about
the bandwidth and power consumption. . .
The top end of the top500 HPCs has always been (and remains) something of a technological pecker-measuring contest. The marginal efficiency (that is the Rmax/Rpeak) for the HP Conjugate gradient benchmark for the top 3 in the Top 500 is interesting. The point is that if you really need the horsepower, you build a real supercomputer with a high bi-directional bandwidth, like RIKEN.
So the key questions are: What is Exascale computing really going to be able to do, and who in the world might want to do that?
Hint: The current population of the US is approaching 330 Million. Do the math and see how many MegaFLOPs per man, woman and child an exaFLOP comes to.
Then consider the aggregate computing power that folks like Google, Microsoft, Facebook. . . the list is damned nigh boundless, are devoting to what they do.
What could possibly go wrong?