Sign in / up

back to article DataCore dominating SPC-1 benchmark on price–performance

DataCore has answered criticism of its Parallel Server SPC-1 benchmark [PDF] – that it was not applicable to enterprises, as it lacked high availability (HA) – by running an HA version that is the fifth-best SPC-1 benchmark performer ever. It is also the third-best ever price–performance score on the benchmark, with the top …

COMMENTS

Post your comment
House rules Send corrections
Add to 'My topics' unstar *
  1. Duncan Macdonald

    Parallelizing server IO

    Nice idea - however it requires either rewriting either the Windows kernel (a bit difficult without the sources!!) or the Linux kernel (possible but VERY non-trivial) depending on what the server runs on.

    Microsoft have very little incentive to rewrite the portions of the windows kernel needed for fully parallelizing the disk I/O and have a lot of incentive NOT to rewrite the kernel (cost and the certainty of introducing nasty bugs).

    Rewriting the Linux kernel for fully parallel disk I/O without breaking existing user programs would require a lot of first rate talent with a corresponding high cost and no guarantee of a profit at the end.

    The DataCore system is dedicated to I/O and does not need to support existing user programs (as they do not run on the box) so the company could write the fully parallel I/O code for a far lower cost.

    1 0 Reply
    1. Bronek Kozicki
      Reply Icon

      Re: Parallelizing server IO

      yes, and that was smart choice it would appear. Specialised software can get really good at performing one task only.

      1 0 Reply
  2. Anonymous Coward
    Anonymous Coward

    The issues with this

    1. The benchmark software is running on the Datacore servers. Not on client machines connected over a fabric. Every other submission I'm aware of has external clients.

    2. The current version of SPC-1 has a hotspot of about 7% of the total data. Datacore used around 3TB of space for the test data. 7% of that is under 250GB. The memory dedicated to Datacore cache in each server is 508GB...

    So most of this IO is really running in RAM cache. Which is what they've been doing with all their submissions.

    This is the easiest way to game the current SPC-1 benchmark.

    The SSDs they're using aren't capable of sustaining this performance.

    Nothing to do with parallelization.

    2 1 Reply
    1. Flammi
      Reply Icon

      Re: The issues with this

      Agreed. The whole thing is a joke.

      0 1 Reply
    2. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Serial misconceptions and distortions (was Re: The issues with this

      On DRAM, note that the current #1 Huawei used 4TB of DRAM cache and did 3 million IOPS (at 5 x the response time of DataCore), and it wasn't even H/A...so they'd have needed 8TB of DRAM to make it HA.

      I keep hearing you say that SPC-1 is 7% hot data -- you can say it as many times as you want on every SPC-1 article here on The Register and it will still be wrong. You don't understand SPC-1.

      SPC-1 is based on TPC-C and it is 100% hot data.

      0 0 Reply
  3. Anonymous Coward
    Anonymous Coward

    Nope...DataCore sw does not require application rewrites or client-side code.

    Duncan...if you contact DataCore, I'm sure they'll help you spin up a demo to confirm this for yourself.

    0 0 Reply
    1. Duncan Macdonald
      Reply Icon
      FAIL

      Sorry - my mistake

      I assumed that the SPC-1 benchmark was better protected against gaming - I was wrong.

      Reading the SPC-1 Benchmark full disclosure for this test - it is actually running on Windows Server with caching software providing a huge in memory cache (of over 500GB). I had thought (incorrectly) that SPC-1 required separate client and storage server systems.

      1 1 Reply
      1. Anonymous Coward
        Reply Icon
        Anonymous Coward

        Re: Sorry - my mistake

        Duncan...what made you think that SPC-1 required separate client and storage server systems?

        IBM ran this same identical architecture all the way back in 2009 (running the workloads on the "storage system"), this is nothing new...except that some now call this 'hyperconverged'.

        http://www.storageperformance.org/benchmark_results_files/SPC-1/IBM/A00083_IBM_Power-595_with_PowerVM_SSDs/a00083_IBM_Power595-PowerVM-SSDs_full-disclosure.pdf

        Meanwhile every storage system ever tested in SPC-1 going back to 2001 uses DRAM for cache, and the more the better -- also nothing new.

        Moreover, others have run the ENTIRE SPC-1 benchmark in DRAM going all the way back to 2008...without ANY non-volatile storage...

        http://www.storageperformance.org/results/a00063_TMS_RamSan400_full-disclosure.pdf

        I'm not sure what you're driving at here...

        0 0 Reply
      2. Anonymous Coward
        Reply Icon
        Anonymous Coward

        HA vs. No HA (was Re: Sorry - my mistake

        Duncan...while you're at it, you might also want to take note of the fact that the IBM system that ran the workloads on the storage server in was not HA. The new DataCore result IS HA.

        The SPC identifies HA results (like DataCore's new posting) as "Protected 2" while non-HA results like the IBM system are identified as "Protected 1".

        You'll find it on page 11 here:

        http://www.storageperformance.org/benchmark_results_files/SPC-1/IBM/A00130_IBM_Power780-SSDs/a00130_IBM_Power780-SSDs_SPC-1_full-disclosure-report.pdf

        Thought you would want to know...

        0 0 Reply
  4. Anonymous Coward
    Anonymous Coward

    Conspiracy Theorists

    I find it amusing that in spite of everything DataCore has done to flip the economics and storage performance upside down, their are always people who would prefer to find the flaw in the execution. If you haven't been through an SPC-1 audit personally, you would be amazed at the level of scrutiny and detail that the audit team, who are highly talented technical professionals, put the vendors through. There is a reason the SPC-1 is recognized as the premiere storage benchmark. And let me assure you that EVERY vendor that submits their product, their time, and their reputations to this process, puts their very best foot forward. Parallel IO is no gimmick - keep watching

    1 0 Reply
    1. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: Conspiracy Theorists (maybe not?)

      Your entire comment is well taken, and having worked for multiple storage vendors who have published an SPC-1, and for customers who used the SPC to make storage decisions I can personally attest that you are right on. SPC membership companies take the peer review process very seriously, to preserve the integrity of the benchmark.

      Yet I don't think these commenters are conspiracy theorists. Rather, I suspect they are mostly well intentioned storage professionals who are just struggling to understand how all of these 'Hardware Defined' storage architectures can possibly look so bad compared to DataCore's SDS stack on standard Intel x86 servers. They are struggling against disbelief.

      I think the entire 'naysayer/disbeliever' mentality can be summed up in one simple chart from David Floyer and the team at Wikibon (See Figure 6. at http://wikibon.org/wiki/v/The_Rise_of_Server_SAN)

      Software-Defined Server-SAN is the future...and old-school "hardware-defined" storage, like the Mainframes of the past, are rapidly declining in relevance.

      The SPC-1 results are proof of that.

      However...there are vast numbers of storage professionals who feel that their career-long investments in "hardware defined" storage are going down the tubes with the Rise of Server SAN. They are no different than the mainframe professionals who have watched their skills set decline in relevance over the last 20 years.

      Some of these folks will see the SDS vs. HDS writing-on-the-wall, get on board with SDS and thrive. Others will hang on and fight the tide, against the inevitable. And (yes) there are a few who will see it all as a conspiracy theory.

      These are just normal human responses to fear.

      1 0 Reply

      1. This post has been deleted by its author

POST COMMENT House rules

Not a member of The Register? Create a new account here.

Forgotten password ?

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like