back to article OnePlus accused of installing cheat codes for benchmarks with new handset

On Tuesday OnePlus launched its latest smartphone with much hoopla, and the handset has already found notoriety. The Middle Kingdom manufacturer sent out review units to a few publications before the launch (El Reg isn't on their lovey list) and coder blog XDA Developers analyzed a unit. According to its research, the new …

  1. Steve Aubrey

    Where have I heard that before?

    "I didn't do it, and I promise not to do it again."

    1. Destroy All Monsters Silver badge

      Re: Where have I heard that before?

      O.J. Simpson when the cops turn up for wife beatings?

  2. Anonymous Coward
    Anonymous Coward

    5%

    Personally I don't consider 5% to be substantial. It's not like the days when CPU performance decided whether the compile would have completed by the time you got back to work next morning. But as I understand it they aren't gaming the benchmarks by the good old techniques of recompiling with loops whose results are discarded simply removed, but by running at full speed.

    If the claim that their OS skin does this for intensive applications is true, it isn't precisely dishonest. It just demonstrates that benchmarks are, indeed, more and more artificial.

    1. Nate Amsden

      Re: 5%

      Less than at least 25% and I can't tell a difference (I probably could in a game though I don't really play games).

      On the same note I can tell zero difference between 6gbps sata and 10gbps nvme (both samsung pro) on my quad core skylake lenovo p50 laptop. (Outside of benchmarks anyway).

      My last phone upgrade was galaxy note 3 (still using it), quad core 2.3ghz probably at least 350% faster than the hp pre3 it replaced (single core 1.4ghz) and 600% more ram, and 36X more storage(currently 256gb sd+32gb vs 8gb on pre3, though when I originally bought note 3 it had 96gb total).

      I have looked at benchmarks for newer phones and have not noticed anywhere close to a similar leap in specs. Note 3 works fine so no interest to change it.

      Maybe when quad core 5ghz comes round with 12G of ram and 1tb of storage(with comparable battery life) I'll be willing to drop android 4.4. Not holding my breath though.

    2. DropBear

      Re: 5%

      I didn't look into it extensively but I seem to recall what they did was disable thermal throttling or some such, artificially pegging the pedal to the metal in a way that isn't really happening anywhere else. That said, again, I'm not familiar with the exact details...

    3. Sil

      Re: 5%

      You don't find 5 % substantial, but seeing a chart with a name on top, won't you get influenced by the ranking ?

      1. Anonymous Coward
        Anonymous Coward

        Re: 5%

        "You don't find 5 % substantial, but seeing a chart with a name on top, won't you get influenced by the ranking ?"

        back in the old days review sites used to do loads of benchmarking of different PC motherboards ... but as more and more of the functionality became part of standardized chpsets then these turnied into pages of charts with everyone having (about) the same resutls .. and as a result some manufactures became known for increasing their bus frequency by 1-2% to try to move them to be the "first among equals" position .... there's a similar phenomenum now in graphics cars where "factory overclocked" cards up frequencies by tiny amounts but it will get them higher up the list of in all other respects identical cards

  3. TonyJ

    Oneplus... no compromises. Or security updates.

    They do good marketing, I'll say that much.

    1. Boothy

      Oneplus 3 here (original version, not the updated one), and I get OTA updates every few weeks.

      Current Android version is at 7.1.1, at a security patch level May 2017. It's rarely more than a month behind Google themselves.

      Any other manufactures better than that? I'm genuinely interested to know?

      1. W4YBO

        "Any other manufactures better than that? I'm genuinely interested to know?"

        Google's Project Fi, but I'd bet that's pretty much it.

      2. goldcd

        Yup

        Pixel owner here, and getting my updates 2-3rd of each month since I bought it.

        1. This post has been deleted by its author

      3. Anonymous Coward
        Anonymous Coward

        > Current Android version is at 7.1.1, at a security patch level May 2017. It's rarely more than a month behind Google themselves.

        > Any other manufactures better than that? I'm genuinely interested to know?

        A Spanish company called BQ, apparently (I chose a phone at random).

        I believe their stuff is very similar to this OnePlus thing, though I am not particularly familiar with either.

      4. cb7

        Galaxy S7

        Security patch level installed within the last week bringing it up to May 2017.

        Android version still 7.0 though.

        1. Doogie Howser MD

          Re: Galaxy S7

          Not strictly the same thing, but I have an old Samsung SM-P600 tablet that I update regularly with LineageOS builds, so I've always got the monthly security update on it.

        2. iron Silver badge

          Re: Galaxy S7

          My S6 Edge is exactly the same, up to date on monthly security patches but still 7.0. Not bad for a phone that is 2 years old.

      5. TonyJ

        OnePlus 2 owner from new before swapping. In the 10 months of owning it, I got three OTA updates including the one to Android 7 - so TWO...yes, TWO security updates in almost a year... and I would check manually at least weekly as well as the automatic updates.

        Contrast that to my Galaxy S7 (that was bought for me as a Christmas present - ironically the reason I'd moved away from Samsung to the OP was they committed to regular updates) which is on the May security patch and has patched monthly since December.

        It is yet to be seen of course, at what point Samsung decide to stop patching it but so far it's been far superior to OP.

        1. TonyJ

          Just had a popup - downloaded and installed the June 1st security update

      6. Chz

        Huwaei Honor 8, similar to Samsung. Up to the April patches, but still 7.0.

        To be honest, that's way more than I expected out of them. I bought the phone resigned to the fact that it may not get much in the way of patching, but since Goog's moved a lot of the Android core into apps update-able through Play I didn't care much.

        I was even more shocked when they actually released source a few weeks back so that we now have drivers for the Kirin SoC.

      7. CrazyOldCatMan Silver badge

        Oneplus 3 here

        Likewise. Mine is, however, rooted and running a custom ROM. Sure, that means I can't use Android Pay but, since I wouldn't anyway, I've not felt the lack.

        The ROM I'm using is this one: https://forum.xda-developers.com/oneplus-3/oneplus-3--3t-cross-device-development/op3-op3t-unofficial-lineageos-14-1-t3588696 from the developer Sultanxda - he provides regular updates based on the upstream changes from both LineageOS and OnePlus.

        I would heartily recommend his ROM (and his Android 6 version for the OnePles One)

      8. Anonymous Coward
        Anonymous Coward

        Any other manufactures better than that? I'm genuinely interested to know?

        Just to throw an odd one in: Windows Phone, and it can't run the update because the pretty much empty phone doesn't have enough storage, which pretty much says it all, really. I'm glad it's not mine, but I have the displeasure to be unavoidably acquainted with two idiots who bought it despite warnings not to, and are now complaining that it doesn't work.

        It is my sincere hope that me telling them (a) beforehand not to do it and (b) (when they started complaining) that my fees would probably be more than a new phone will eventually rid me of these people, but they've been surprisingly clingy. They must be masochists..

      9. VinceH

        "Oneplus 3 here (original version, not the updated one), and I get OTA updates every few weeks.

        Current Android version is at 7.1.1, at a security patch level May 2017. It's rarely more than a month behind Google themselves."

        WileyFox Storm. Version 7.1.1 and 1st June 2017.

        1. Anonymous Coward
          Anonymous Coward

          OnePlus One. I got regular updates and have now moved to LineageOS and am still getting regular updates. I'm on the latest Android version thanks to that.

          Other phones in the family almost never get updates.

          I don't really care about the benchmarks. I like my current phone and can't see any reason to change it until it breaks. I doubt I'll go to the OnePlus 5 though, just because of the price.

    2. TheDillinquent
      Trollface

      Oneplus... no compromises. Or security updates.

      I take it you don't own an OP then?

      My OP3 got a whole updated OS last week, with May '17 security patch.

  4. Planty Bronze badge
    FAIL

    HTC

    No mention of HTC getting caught doing the same....

  5. Anonymous Coward
    Anonymous Coward

    Chinese spec writers and honesty

    Not easy bedfellows

  6. Anonymous Coward
    Anonymous Coward

    "El Reg isn't on their lovey list"

    Neither will be XDA, by the looks of it. I thought when you got a review unit you were supposed to sing the praises of it. :-)

  7. Kevin McMurtrie Silver badge

    Meh

    Usually the cheat is no different than selecting the "Performance" mode in the Power settings. It changes the governor up/down thresholds so that there's less chance of a pause after one system is done waiting for another to finish.

    I'm just happy if the OEM tuning doesn't maximize stuttering or overheat running 2D games that alternate between CPU and GPU use.

  8. msknight

    Tainted?

    "This is an inexcusable move, because it is ultimately an attempt to mislead not just customers, but taint the work of reviewers and journalists with misleading data that most are not able to vet or verify."

    You're kidding me... right? As if that world isn't already tained to the tune of an extra large pot of purple Dulux gloss all over it?!

  9. Anonymous Coward
    Anonymous Coward

    I don't honestly see they've done anything wrong. If the OS would also detect a game being played, and up the CPU power, then this is just demonstrating a realistic performance level that you'd expect under heavy use.

    If CPU power for benchmarks was increased, but not for games, that would seem a bit more cheaty.

    1. Anonymous Coward
      Anonymous Coward

      If the governor worked properly it would boost all demanding titles including benchmarks and not need app specific exceptions. But these benchmarks aren't boosting as much when hidden from the cheat, the assumption has to be no other app will get these special settings either. That's cheating.

  10. Bucky 2

    I have questions...

    If benchmarks were of no use, wouldn't there then be no point in trying to subvert them?

    No matter how I look at it, it seems like their intention was to mislead. How is that ever okay?

    1. Anonymous Coward
      Anonymous Coward

      Re: I have questions...

      I think their point is that the benchmarks are no good but some people use them so they want a "fair" ranking.

      You may question if what they've done is about fairness or cheating but I can understand why.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon