back to article Wow, still using disk and PCIe storage? You look like a flash-on victim, darling – it isn't 2014

For generations of PowerEdge, ProLiant, UCS and other x86 servers, the future has been fairly simple: more powerful multi-core processors, more memory, more PCIe bandwidth and shrink the space and electricity needs. For example, a Gen8 ProLiant DL3603 server had 1 or 2 Xeon E5-2400/2400 v2, with 2/4/6/8/10 cores, and 12 x DDR3 …

  1. bazza Silver badge

    Old idea, except Flash is far from ideal for this purpose. Old, because this is the ultimate use of something like memristor and has been discussed in that context before. Flash is non-ideal, you still have to do wear levelling, else it wears out.

    Now if HP ever did finish off memristor, or if any of the other players in that new-memory-tech game got their act together, that would be ideal. Faster than DRAM, great, non-volatile, check, wear-free lifetime, perfect. It'd be just like a SIMM that doesn't forget, ever.

  2. Storage_Person

    > "The disk-based IO stack in an OS takes too much time and is redundant"

    Yeah, who needs all that rubbish like metadata, consistency and replication?

    The problem with persistent memory at the moment is that it requires code-level changes (see for example) if you want to have this great performance.

    A thin filesystem-like layer would lose you a bit of speed compared to writing to bare memory but give you an awful lot more back in terms of reliability and management. Persistent memory-native apps will come, but not for a while yet.

    1. Paul Crawford Silver badge

      Memory mapping?

      "an OS takes too much time"

      For many cases you can memory-map a file and as you initially access it, it gets paged in to RAM by the virtual memory system.

      Downside is the rare occasions when it is flushed back to disk (typically only if you ask for that, or by properly un-mapping closing the file). So you gain speed but lose consistency/integrity.

    2. TheVogon Silver badge

      "The problem with persistent memory at the moment is that it requires code-level changes"

      No it doesn't. See

  3. Lusty


    This is going to push the need for resilience on DIMMS, and that's likely to be erasure coding. Unfortunately that'll then stomp on the CPU and we'll need another upgrade cycle :)

    This is like Violin when they emerged - a few niche environments will need it, the rest of us will salivate but ultimately make do with the normal technology which is still fine for running the mail server, just like the 486 box in the corner was...

    1. Pirate Dave Silver badge

      Re: resilience

      "still fine for running the mail server, just like the 486 box in the corner was..."

      What do you mean, "was"...?

  4. chrismevans

    Don't forget the processing effort

    Don't forget in these calculations that the CPU doesn't spend 100% of its time reading and writing data from external storage. In fact, with a decent amount of DRAM and clever data mapping, the processor might not read/write that often, depending on the application.

    Also, we have to bear in mind that when the processor does 'do work", it may be at the request of an external call (e.g. a website) or some other user interaction that takes time over the network.

    All this means the delay from storage I/O might not be that impactful, if we have enough other work to do. Hurrah for large memory caches, multi-tasking and parallel programming!!

  5. potatohead

    This is quite a simplification. There are plenty of situations where jobs are using remote storage (SAN say) to access large datasets, which can't possibly be replicated locally due to the data size. In these situations the local disk is hardly hit, and file access is normally sequential, so the criteria you are quoting (access speed, or basically random access latency) is totally irrelevant.

    I'm sure there are applications that this sort of architecture would help, but there is plenty of stuff where this is not the case.

  6. Nate Amsden

    tiny niche

    The stuff that needs this kind of tech is a tiny niche. Been supporting ops/dev type shit for 15 years now and at really no time has something faster than SSD been even thought as a "nice to have". Current org is a typical online e-commerce business doing well over $200M/year in revenue running standard web stacks (with mysql as the DB of choice). In excess of 90% of all I/O for OLTP occurs within mysql's buffer cache. Disk I/O to the LUNs mysql lives on is tiny, really only happens for very infrequent big expensive stupid queries but far more often for log writing (binary logs, relay logs etc). Storage array I/O workload is over 90% write under 10% read. We probably peak at 15-20% of the array's I/O capacity (meaning it will never be a bottleneck in lifetime we will be using it for).

    I was at another company several years ago who, many years before that wrote a proprietary in memory database for behavioral ad targeting. We had dozens of servers with massive amounts of memory(for the time) and clunky processes for loading data into these in memory instances. While I was there they eventually decided that in memory was too expensive so they re-wrote the stack so it did some layer of caching in memory(reducing memory usage probably by 80-90%) and the rest of the data sat on a NAS platform connected to a 3PAR storage array(with SATA disks no less). So they went from in memory to NAS on SATA, and the performance went UP, not down.

    I still believed it was a flawed architecture I would of preferred them use local SSD storage (at the time I wanted them to use Fusion IO). But they architected it so it HAD to reside on a shared storage NAS pool it wasn't going to work on local storage. Too bad, missed opportunity.

    So I've seen and managed traditional mysql (and Oracle) and vmware etc as well as in memory systems across now thousands of linux systems primarily across many companies over the past 15 years, and solutions like those presented in the article I believe are very very niche(relative to the size of the market as a whole). I could see things changing if the costs get down low enough that it makes no difference in cost/complexity if you are using DIMM SSD or SAS SSD(or even PCIe SSD), but I don't see that happening anytime soon.

    I do believe there is a market for this kind of stuff just much smaller one than articles like this seem to portray.

  7. Anonymous Coward
    Anonymous Coward

    It's Time

    Periodically in this industry we get a chance to combine some new hardware and software advanes. The we jump up and down hard on the software stack to remove many of the abstractions to get closer to the (new) metal. Then the layers begin accumulating anew. It has been ever thus.

  8. This post has been deleted by its author

  9. Anonymous Coward
    Anonymous Coward

    Nambiar's Law of IO


    Nambiar’s Law of IO


    Nambiar’s Law of IO

    BY Raghunath Nambiar.

    Two of of the most popular questions asked in the storage performance world are:

    – How do you estimate the IO bandwidth requirements?

    – How do you estimate the IOPS requirements?

    In other words how much IO can be processed by today’s data management and datastore platforms?

    Here is my simple rule of thumb, let me call it the “Nambiar’s Law of IO”, #NambiarsLawofIO:

    100 Mbytes/sec per processor core for every 7 years of software maturity for IO bandwidth intensive workloads. Examples: enterprise data warehouse systems, decision support systems, distributed file system based data processing systems etc.

    10,000 IOPS per core for every 7 years of software maturity for IOPS intensive workloads. Examples: transaction processing, key-value store, transactional NoSQL, etc.

    for up to 21 years, beyond which there is no significant improvement, that’s what we have seen historically.


    Platforms in the >21 years category:

    Traditional relational database management systems (Oracle database, Microsoft SQLServer, etc):

    – Data warehouse: 24 (number of cores in today’s popular 2 socket x86 servers) x 100 (IO bandwidth that can be processed the by software) *3 (>21 years) = 7.2 GBytes/sec

    – Transactional: 24 (number of cores in today’s popular 2 socket x86 servers) x 10,000 (IOPS that can be processed by the software) *3 (>21 years) = 720K IOPS (small block random r/w)

    Platforms in the < 7 years category:

    New generation distributed datastore systems:

    – Hadoop: 24 (number of cores in today’s popular 2 socket x86 servers) x 100 (IO bandwidth that can be processed by the the software) *1 (<7years) = 2.4 GBytes/sec.

    – Cassandra: 24 (number of cores in today’s popular 2 socket x86 servers) x 10,000 (IOPS that can be processed by the software) *1 (<7years) = 240K IOPS

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019