Re: 3D XPoint is a new form of RAM, not SSD
Yes - if anything it reminds me of the merits of magnetic core, without the poor density and high power consumption.
The most interesting use, short-term, would be paging store. Systems already have a mechanism for paging stuff in and out of memory, but it's pretty useless these days as mechanical drives are too slow/thrashy and SSDs wear out.
If you view the paging mechanism as providing a RAM cache of the contents of the backing store (as if it were just a giant binary file that has been memory-mapped), the application is pretty obvious.
For HPC this means that the size of the data set can exceed the size of available RAM; and as RAM no longer limits the data set size, new RAM tradeoffs become possible. The amount of RAM can be chosen to suit the number of pages that are "hot" at any given time, thereby reducing cost/heat/size. The savings then become available for more processing power.
There is one element that would be a handy addition to the present virtual-memory model: a prefetch capability (analogous to that used in floating-point DSPs) that would extrapolate data-access patterns to identify data that should be brought into higher levels of the memory hierarchy before it is needed.
By "higher levels" I'm really thinking of processor caches rather than DRAM, which is pitifully slow in comparison to the performance of HPC compute engines.