Ahem. scuse me thinking aloud for a bit
The memristor stuff is essentially a memory technology. Allegedly something like persistent RAM - not sure if memristors really are as fast as today's DRAM or SRAM.
The photonics part is likely to relate to chip-to-chip interconnects. Not likely all-optical CPU's.
What does all of this boil down to?
The Machine is unlikely to be a whole new architecture, not something massively parallel or what. I would expect a NUMA with memristors for RAM. Did the article author mention DIMMs? The most straightforward way would be to take an x86 server (number of sockets subject to debate), run QPI/HT over fibers, and plug in memristors instead of DRAM. Or use Itanium (or ARM or Power) - the principle doesn't change much.
Is there anything else to invent? Any "massively parallel" tangent is possible, but is not new - take a look at the GPGPU tech we have today. Or the slightly different approach that Intel has taken with the Larrabee+. Are there any gains to be had in inventing a whole new CPU architecture? Not likely, certainly not unless you plan to depart from the general von-Neumannean NUMA. GPGPU's are already as odd and parallel as it gets, while still fitting the bill for some general-purpose use. Anything that would be more "odd and parallel" would be in the territory of very special-purpose gear, or ANN's.
So... while we stick to a NUMA with "von Neumann style" CPU cores clustered in the NUMA nodes, is it really necessary to invent a whole new OS? Not likely. Linux and many other OS'es can run on a number of CPU instruction sets, and are relatively easy to port to a new architecture. Theoretically it would be possible to design a whole new CPU (instruction set) - but does the prospect sound fruitful? Well not to me :-) We already have instruction sets, and CPU and SoC flavours within a particular family, and complete plaftorms around the CPU's, suited for pretty much any purpose that the "von Neumann" style computer can be used for, from tiny embedded things to highly parallel datacenter / cloud hardware.
You know what Linux can run on. A NUMA with some DRAM and some disks (spinning rust or SSD's). Linux can work with suspend+resume. Suppose you have lots of RAM. Would it be any bottleneck that your system is also capable of block IO? Not likely :-) You'd just have more RAM to allocate to your processes and tasks. If your process can stay in RAM all the time, block IO becomes irrelevant, does not slow you down in any way. Your OS still has to allocate the RAM to individual processes, so it does have to use memory paging in some form.
You could consider modifying the paging part of the MM subsystem to use coarser allocation granularity. Modifications like this have been under way all the time - huge pages implemented, debates about what would be the right size of the basic page (or minimum allocation) compared to the typical IO block size, possible efforts to decouple the page size from the optimum block IO transaction size and alignment... Effectively to optimize Linux for an all-in-memory OS, the developers managing the kernel core and MM in particular would possibly be allowed to chuck some legacy junk, and they'd probably be happy to do that :-) if it wasn't for the fact that Linux tries to run on everything and be legacy compatible with 20 years old hardware. But again, block IO is not a bottleneck if not in the "active execution path".
It doesn't seem likely that the arrival of persistent RAM would remove the need for a file system. That would be a very far-fetched conclusion :-D Perhaps the GUI's of modern desktop and handheld OS'es seem to be gravitating in that direction, but anyone handling data for a living would have a hard time imagining his life without some kind of files and folders abstraction (call them system-global objects if you will). This just isn't gonna happen.
Realistically I would expect the following scenario:
as a first step, ReRAM DIMM's would become available someday down the road, compatible with the DDR RAM interface. If ReRAM was actually slower than DRAM, x86 machines would get a BIOS update, able to distinguish between classic RAM DIMM's and ReRAM (based on SPD EEPROM contents on the DIMMs) and act accordingly.
There would be no point in running directly from ReRAM if it was slow, and OS'es (and applications) would likely reflect that = use the ReRAM as "slower storage". This is something that a memory management and paging layer in any modern OS can take care of with fairly minor modification.
If ReRAM was really as fast as DRAM, there would probably be no point in such an optimization.
Further down the road, I'd expect some deeper hardware platform optimizations. Maybe if ReRAM was huge but a tad slower than DRAM, I would expect another level of cache, or an expansion in the volumes of hardware SRAM cache currently seen in CPU's. Plus some shuffling in bus widths, connectors, memory module capacities and the like.
So it really looks like subject to gradual evolution. If memristors really turn out to be the next big thing in memory technology, we're likely to see a flurry of small gradual innovations to the current computer platforms, spread across a decade maybe, delivered by a myriad companies from incrementally innovating behemoths to tiny overhyped startups, rather than one huge leap forward delivered with a bang by HP after a decade of secretive R&D. The market will take care of that. If HP persists with its effort, it might find itself swamped by history happening outside of their fortress.
BTW, the Itanium architecture allegedly does have a significant edge in some very narrow and specific uses, from the category of scientific number-crunching (owing to its optimized instruction set) - reportedly, with correct timing / painstakingly hand-crafted ASM code, Itanium can achieve performance an order of magnitude faster than what's ever possible on an x86 (using the same approach). This information was current in about 2008-2010, not sure what the comparison would look like, if done against a 2014-level Haswell. Based on what I know about AVX2, I still don't think the recent improvements are in the same vein where the Itanium used to shine... Itanium is certainly hardly an advantage for general-purpose internet serving and cloud use.
As for alternative architectures, conceptually departing from "von Neumann with NUMA" and deterministic data management... ANN's appear to be the only plausible "very different" alternative. Memristors and fiber interconnects could well be a part of some ANN-based plot. Do memristors and photonics alone help solve the problems (architectural requirements) inherent to ANN's, such as truly massive parallelism in any-to-any interconnects, organic growth and learning by rewiring? Plus some macro structure, hierarchy and "function block flexibility" on top of that...
I haven't seen any arguments in that direction. The required massive universal cross-connect capability in dedicated ANN hardware is a research topic in itself :-)
Perhaps the memristors could be used to implement basic neurons = to build an ANN-style architecture, where memory and computing functions would be closely tied together, down at a rather analog level. Now consider a whole new OS for such ANN hardware :-D *that* would be something rather novel.
What would that be called, "self-awareness v1.0" ? (SkyOS is already reserved...)
Or, consider some hybrid architecture, where ANN-based learning and reasoning (on dedicated ANN-style hardware) would be coupled to von Neumann-style "offline storage" for big flat data, and maybe some supporting von Neumann-style computing structure for basic life support, debugging, tweaking, management, allocation of computing resources (= OS functions). *that* would be fun...
Even if HP were pursuing some ANN scheme, the implementation of a neuron using memristors is only a very low-level component. There are teams of scientists in academia and corporations, trying to tackle higher levels of organization/hierarchy: wiring, macro function blocks, operating principles. Some of this research gets mentioned at The Register. It would sure help to have general-purpose ANN hardware miniaturized and efficient to the level of the natural grey mass - would allow the geeks to try things that so far they haven't been able to, for simple performance reasons.