back to article D-Wave disputes benchmark study showing sluggish quantum computer

Quantum computing device manufacturer D-Wave is disputing a recently published study that claims the Canadian firm's systems aren't reliably faster than more-conventional computing systems. On Thursday an international team of computer scientists published a widely previewed paper in Science detailing a series of benchmark …

COMMENTS

This topic is closed for new posts.
  1. ROlsen

    I think I know what's up

    The computer is simultaneously fast and slow

    1. stizzleswick

      Re: I think I know what's up

      I just hope it's not Schrödinger's computer.

      1. Tubs

        Re: I think I know what's up

        @stizzleswick

        Maybe that's the problem: it can't do the calculations while you're looking at it.

    2. Gannon (J.) Dick
      Pint

      Re: I think I know what's up

      I wanted to simultaneously vote this up and down but obviously El Reg's website is under the influence of superposition (rather early in the day isn't it guys ?)

    3. Anonymous Coward
      Anonymous Coward

      Re: I think I know what's up

      The New Uncertainty Principle, does it work or not, damn it?

      D-Wave should be permitted to define their own test, the 'Catch Me If You Can' problem. But they need to be at least 10x faster than anyone else, otherwise FAIL. That would more or less prove that it's working.

      Unless they're found cheating with a custom FPGA to solve that one problem. Too easy to be caught, so not likely.

      1. ShortLegs

        Re: I think I know what's up

        "The New Uncertainty Principle, does it work or not, damn it?"

        I'm not sure

        Hat, coat, taxi

      2. Anonymous Coward
        Anonymous Coward

        Re: I think I know what's up

        D-Wave should be permitted to define their own test, the 'Catch Me If You Can' problem. But they need to be at least 10x faster than anyone else, otherwise FAIL. That would more or less prove that it's working.

        actually, I thought they did; when they announced the two I recall it was something like 3000x faster than the digital computer in their testing. and this is hardly the first time their claims have been disputed: some swiss researchers found similar results a year ago and they were testing with simulated annealing algorithms (which the d-wave is supposedly optimized for).

  2. Destroy All Monsters Silver badge
    Headmaster

    A bicycle with a Ferrari exterior?

    Google also issued its own response to the Troyer paper, noting that in its tests the firm is getting computational speeds of over 35,500 times that of conventional computing systems.

    Yeah, but is that speedup outside of the error ranges? And what problems are they solving? Movie recommendations?

    1. John Tserkezis

      Re: A bicycle with a Ferrari exterior?

      "And what problems are they solving? Movie recommendations?"

      The usual things. How to download porn faster.

      1. illiad

        Re: A bicycle with a Ferrari exterior?

        huh, most large vid & pic sites have limited their access speed, to prevent acces problems..

        only *proper* way to check the d/l speed is to download a linux distro...

      2. Anonymous Coward
        Anonymous Coward

        "And what problems are they solving?"

        >"And what problems are they solving? Movie recommendations?"

        >The usual things. How to download porn faster.

        Actually Google does have a class of hard problems that defy algorithmic solutions:

        If customer A07f5DE7C4 watches latex granny porn, and customer B79F6E3C8 prefers vinyl doggy style, should Google display ads for crunchy or smooth peanut butter?

    2. TheVogon

      Re: A bicycle with a Ferrari exterior?

      "And what problems are they solving?"

      How to spy on more aspects of your life at once and target appropriate adverts presumably. What else would The Borg be doing?.

    3. Michael Wojcik Silver badge

      Re: A bicycle with a Ferrari exterior?

      And what problems are they solving? Movie recommendations?

      In all seriousness, the majority of the problems Google has to solve - from maintaining their huge network of computing resources and workloads, to modeling user preferences - can be transformed into various graph problems. And many of those can be converted into questions of energy distribution in a closed system. As such, annealing approaches, which is what the D-Wave seems to be doing1, should be applicable in many cases, and could offer significant speedups over classical discrete calculation.

      Of course that's true of other analog mechanisms, "quantum" or not, as any number of people have noted.

      1I take no position on whether it's "true" adiabatic quantum computation (AQC), or "true" quantum annealing, etc, or just a hybrid classical analog-digital machine. I haven't looked at it and I'm not qualified to make that determination anyway.

  3. Paul 87

    So basically, bad code causes slowdowns?

    Who'd have thought it!

    1. Destroy All Monsters Silver badge
      Holmes

      So basically, bad code causes slowdowns?

      No, because there is no code.

      It's a "quantum annealing" machine, which is something like a cooling metal, only with configurable strengths between the configuration elements (think analog computer doing neural networks or spin glass annealing), and with more quantum sauce.

      I also notice that there are two Google Guys/Gals in the list of authors:

      > Troels F. Rønnow - Theoretische Physik, ETH Zurich

      > Zhihui Wang - Department of Chemistry, University of Southern California

      > Joshua Job - Center for Quantum Information Science and Technology & Department of Physics, University of Southern California

      > Sergio Boixo - Google, 150 Main Street, Venice Beach, California

      > Sergei V. Isakov - Google, Brandschenkestrasse 110, Zurich

      > David Wecker - Quantum Architectures and Computation Group, Microsoft Research, Redmond

      > John M. Martinis - Department of Physics, University of California Santa Barbara

      > Daniel A. Lidar - Department of Chemistry & Center for Quantum Information Science and Technology & Department of Physics, University of Southern California

      > Matthias Troyer - Theoretische Physik, ETH Zurich

      1. Zane

        Not programmable

        So do I get this right: it's not programmable, i.e. it's not a computer at all, it's just a machine that is optimized for some special problem? Than what is the problem that it solves so fast?

        /Zane

        1. Anonymous Coward
          Anonymous Coward

          Re: Not programmable

          I'd rephrase to a machine optimized to handle a certain class of problems, not any one particular problem. What you do is construct the experimental setup so that when the system "relaxes" you end up with the optimal solution to that particular problem. It's all about getting the preconditions right. There's a certain quantum-mechanical sauce (retrieving the 'solution' without collapsing the wave-function) but that's the idea.

          Trying to compare it to a system setup with a highly specialized, highly optimized GPGPU system really doesn't do it, in my not so humble opinion. My problem here, and as I recall this was pointed out before, is that you have a hell of a time trying to identify when those problem domains actually overlap between the two methods and for which you should detect a measurable (measure, quantum, uhhh...) set of results that positively (p<.01 I believe) demonstrates that the D-Wave is a quantum computer.

          Instead what we end up with is some problem types that demonstrate the D-Wave is ideal for that one particular problem and I mean one, not the whole class of problem, a lot of experiments that show equivalence between classical and quantum, and some for which the D-Wave doesn't come up with a solution. Personally, I am not surprised. First, if this were easy, we'd have done it decades ago. Second, thinking how to properly 'program' the D-Wave would give most anyone migraine's, ala Dr. Feynmann and understanding QM. Lastly, if you don't believe in a personally malignant universe, you will now.

          1. phil dude
            Boffin

            Re: Not programmable

            A nice post! I am still waiting for my slides from D-Wave for the curious presentation given at SC13... I was given the nudge not to discuss "certain uses". Go figure, it is only a hobby for me...

            However, buying a supercomputer to program your quantum computer (that was the presentation's thrust, though it was mentioned it was part of an iterative process), is marketing genius at a computing conference - no really! Buy 2!!

            I have a number of colleagues who really hope this works, but so far the resounding lack of evidence suggests there is much more research to do understanding exactly what the machine they have built *is*. Apparently, if you agree not to use the QC for anything "useful" you can apply online to have a go.....see what I did there?

            In another thread I wrote "it's still a really good fridge". A friend of mine who makes deuterium bullets, thought is was a cryogenics research project on first hearing "microKelvin" mentioned....

            P.

            1. Anonymous Coward
              Boffin

              Re: Not programmable

              If you're looking for interesting coincidences (of which I am not certain exist), go read my comment over in Pond Scum:

              http://forums.theregister.co.uk/forum/1/2014/06/17/mutation_governs_algaes_quantum_mechanics/

              Same class of problem, interesting how nature achieves it, and without any reference to micro-Kelvins.

  4. John H Woods Silver badge

    So are these D-Wave boxes fast ...

    ... until you observe them?

  5. bigtimehustler

    So, seems to me everyone who buys one sees their benefits but a group of scientists who didn't have to justify the business case for buying it don't think its fast? I know who I believe, and frankly that is the people who laid their jobs on the line recommending the purchase of the machines and use them in real world situations, not a fake lab test that no one will ever run on the system ever.

    1. Anonymous Coward
      Anonymous Coward

      Emperor's clothes

      "everyone who buys one sees their benefits but a group of sceptics who didn't have to justify the business case for buying it don't think its fast? "

      By any chance have you worked for Autonomy in the past?

    2. Anonymous Coward
      Anonymous Coward

      " I know who I believe"

      The D-Wave machine may be perfectly genuine, but surely we all know people who've bought some immensely expensive piece of junk and have then managed to spend long enough promoting it internally before the shiny wore off, to polish the CV, and find a job somewhere else? It may not be that common, but it does happen.

      I'd mention two or three cases from my own experience but sadly the details might create enough identification to be libellous; though I can say that two of them involved the purchase of electron microscopes to solve problems that, in fact, could not be tackled in this way.

  6. David Dawson

    We live in a world with quantum computers!

    I'd still quite like my jetpack, but this is really cool.

  7. illiad

    historically....

    when the PCI-E bus came out, it seemed too complex and slow.... and everyone said all the other new buses were better, PCI-E was more expensive, sometimes even slower....

    then CPU speeds increased, memory got a lot better, and they improved the PCI-E architecture...

    I think you know who the winner is today..... :p

    when they manage to get a 'pc-standard' basic test going on the D-Wave, then we will get something believable...

  8. Tsunamijuan

    The old game of benchmarks

    Every time I discuss benchmarks with people its sorta like discussing politics. You have to take into account whose doing what. What initial bias they have if its intentional or not. Then you have to take into account who ran the benchmarks and how versed they are in what is being tested.

    Most of the time the person running the benchmarks has more experience running one over the others. Specifically if they had an extremely limited amount of time with what they are benchmarking. Then there are in house benchmarks. These will always be skewed somewhat in favor of the companies own software. In part cause they know how to optimize and tweak their software better. Then the other item is left pretty much out of the box or sometimes even unconfigured.

    To put it mildly benchmarking and benchmarks mean nothing unless you have decided on the ruleset before hand that is going to be the standard, and constantly check to make sure that the standard has been adhered too. Even then there is a enough of a margin of error that can throw this stuff of.

    1. Anonymous Coward
      Anonymous Coward

      Re: The old game of benchmarks

      And provided that someone sneaky hasn't programmed that the machine in question detects you running the benchmark. Samsung anyone?

      1. Anonymous Coward
        Anonymous Coward

        Re: The old game of benchmarks

        Not only Samsung. Wasn't there once a benchmark where the compiler identified empty loops and short circuited them on one maker's processors, and iterated them fully on another?

  9. DropBear
    Facepalm

    Ok so deciding once and for all exactly how "quantum" the machine is looks like a hard job, and I even get that we're not quite sure exactly what sort of thing it might be good at. But from a purely pragmatical point of view evaluating its usefulness is not hard at all - take a Hard Problem you're interested in, get the best conventional hardware you can, put it side by side with the D-wave kit and see which gives you more answers per second, by how much, and how accurate / optimal each is. That's it! If you really observe 35000 answers from one while a single one from the other, it really is faster, quantum or not - unless it's 35000 time more expensive too, it's a win. If not, we're just playing "The emperor's new clothes" all over again...

    1. Anonymous Coward
      Anonymous Coward

      Apart from all the other factors, it's necessary to take into account problem preparation time - an issue identified as far back as the days of one A M Turing, and the driving force behind early programming languages.

      It's like the interpreted versus compiled languages issue - if I can develop code to solve a particular problem much faster in, say, Python, than in C++, the slower execution time only matters if I need to solve the same problem many times with different parameters. Your argument would be correct if the class of problem we are talking about is like that. But if the problem is a few-off only, getting 35000 answers from one and one from another may not matter at all.

  10. Anonymous Coward
    Anonymous Coward

    A suggestion

    Feed Mr 503 qubit machine a real world problem such as simulating the behavior of a simple superconductor at variable temperatures.

    A regular PC would struggle and even a supercomputer would take ages to do the calculations.

  11. James Loughner
    Coat

    But does it

    run flight simulator??

    Had to be said

    Mine is the one with the joy stick in the pocket

  12. John Smith 19 Gold badge
    Unhappy

    Seems they are playing the old school definition of a "computer"

    IE Something that computes (something).

    The RL10 rocket engine contains a similar "computer" for mixture ratio control.

    Most people today would call it a hardwired analog computer. Operational amplifiers, temperature sensors, no actual ADC/DAC in sight etc.

    So what kind of problems do Google get a 1:35000 speed up on?

    1. Jellied Eel Silver badge

      Re: Seems they are playing the old school definition of a "computer"

      What do you get if you multiply six by nine?

      1. illiad

        Re: Seems they are playing the old school definition of a "computer"

        well however much you like Douglas, it is NOT the right answer.....:P :/

        the real problem is 22 divided by 7.. how long does it take, to get 1 million digits..... :P :D

  13. DerekCurrie
    WTF?

    Judging an Experiment as a Commercial Product is Rarely Helpful.

    The overwhelming sense I have, reading about this situation, is that quantum computing is still in an infantile stage. But here we have it being judged in public as if it's a viable product ready for public consumption, when it obviously is not. The enormous Clunk Factor going on at D-Wave is inevitable at this point. To have them shoved into defensive mode over an obviously Not Ready For Prime Time machine is ridiculous. I hope this wrong attitude toward the technology doesn't kill the baby before it has a chance to grow up.

    And then there's the possibility that D-Wave really does have a dog that won't hunt. But this clearly is not the time for that judgement.

    1. andre 2

      Re: Judging an Experiment as a Commercial Product is Rarely Helpful.

      Considering the QPU is cooled down to millikelvins, in a special room similar to the shielded one used for MRI scanners and shielded from everything including ambient radiation I would say that D-Wave do indeed have a quantum computer.

      calling is a "physics experiment" is like calling Fat Man a "Sunshine simulation device"... Doesen't do it justice and D-Wave deserve serious funding whether their system is fully tested or not.

      We can't afford to be left behind on this, China has its own version in the pipeline and South Korea even has a prototype under construction.

      503 qubits is incredible progress in such a short time even if they are flux ones, ten years ago this would have been science fiction.

      1. Destroy All Monsters Silver badge
        Holmes

        Re: Judging an Experiment as a Commercial Product is Rarely Helpful.

        Given that ... I would say that D-Wave do indeed have a quantum computer

        Although your biggening up is noted, your criteria for QC-ness are frankly lacking in salience.

        You are of course free to send your own money to D-Wave (far be it from me to have a negative opinion on that), but please don't go all "we this" and "we that" here.

    2. John Smith 19 Gold badge
      Unhappy

      Re: Judging an Experiment as a Commercial Product is Rarely Helpful.

      "And then there's the possibility that D-Wave really does have a dog that won't hunt. But this clearly is not the time for that judgement."

      I recall a company that put out some of the first Josephson Junction superconductor components. Made in Nb and LHe cooled.

      They were commercial.

      Were they "Ready for prime time?"

      Probably not.

      But they cost their early adoptors a shedload of cash to find that out.

This topic is closed for new posts.

Other stories you might like