back to article Egad! Could Samsung be cheating in Galaxy benchmark tests?

Samsung has reportedly been cheating in benchmark tests, artificially boosting the scores of its latest and greatest system-on-chip, the Exynos 5 Octa, on those performance-ranking number generators so beloved by reviewers and product evaluators. "Oh hell Samsung, shame on you!" wrote a Beyond3D forum member in a posting on …

COMMENTS

This topic is closed for new posts.
  1. ecofeco Silver badge

    Large corproations fudge their numbers?

    Shocking. Just shocking.

    Not.

    1. LarsG
      IT Angle

      Re: Large corproations fudge their numbers?

      So the bottom line is this,

      You're not getting the performance you are paying for

      You're not getting the battery life you have been promised

      Nothing new really is there

      1. GettinSadda

        Re: Large corproations fudge their numbers?

        Next you'll be telling me I can't expect the achieve the mpg figures in my car that are listed in the brochure!

  2. Grikath

    Shocking....

    And absolutely no-one in any business has, of course, "positively represented" their relevant stats to the General Public.

    Getting caught out is embarassing, but really is this news?

    1. Anonymous Coward
      Anonymous Coward

      Re: Shocking....

      They get caught with their hand in the cookie jar and it just okay - they must have been hungry. What treatment would Apple have got if this was reversed??

      1. Anonymous Coward
        Anonymous Coward

        Re: Shocking....

        "What treatment would Apple have got if this was reversed??"

        Well for a start the headline would, for sure, not have been represented as a question.

        And for seconds they wouldn't have had The Register try to re-level the playing field already tipped so underhandedly in their favour with a statement like;

        "And if they're cheating, there's a fair chance that others are, as well. But Samsung got caught."

        Third, they would have got a "journalist" deliberately designated with a trolling brief to write the article and use only childishly disparaging language throughout, with copious references to "fanbois" (Jasper Hamill, before him Anna Leach)

        Might I suggest The Register adjust their strap-line when publishing articles on Samsung to "licking the arse that shits on IT"

    2. AndyS

      Re: Shocking....

      Of course this is news - how else do you expect to enforce a level playing field, if not by journalists reporting when companies cheat? What do you think qualifies as news, simply a new shiny thing?

    3. This post has been deleted by its author

  3. Anonymous Coward
    Anonymous Coward

    My WiFi Access Point has a 400MHz processor in it.

    I thought advanced phones would have a faster chip inside them.

    1. Steve Knox
      Facepalm

      They do

      The processors are 1.2GHz. 480Mhz is the GPU speed.

      Oh, and clock speed is not the sole predictor of performance. The processor architecture and design have a great deal of impact.

      1. Anonymous Coward
        Anonymous Coward

        Not just "not the sole predictor"

        Clock speed comparisons across different architectures are completely meaningless.

      2. I think so I am?
        Facepalm

        Re: They do

        You do know their all 64bit right.

        1. Anonymous Coward
          Anonymous Coward

          Re: They do

          "You do know their all 64bit right."

          *they're.

          Top marks for ironic handle and use of icon though.

        2. Anonymous Coward
          Anonymous Coward

          Re: They do

          No, actually all the current generation of ARMs are 32bit. Not that this has anything to do with comparing CPU performance by clock speed (several very different ARM designs in current use as well as independently designed ARM compatible cores.) It has even less to do with comparing CPU and GPU perf/clock. GPUs are wide -- often very wide -- SIMD machines that get a lot of work done in a single clock.

    2. Giles Jones Gold badge

      Mhz is just the oscillator speed. It's how much a CPU can do per cycle that matters.

      Just like in a car where CC doesn't really matter, it is power to weight ratio.

  4. Anonymous Coward
    Anonymous Coward

    Proper mobile benchmarking

    While Samsung's cheating is inexcusable, I hope the people running these benchmarks learn something from this and expand their repertoire a bit from choosing a handful of stupid benchmarks that don't measure anything useful versus what a person uses their phone. These type of toy benchmarks were mostly exterminated from articles about PCs about 10 years ago, but you still see people referring to crap like AnTutu, Geekbench and even Dhrystone for chrissakes.

    Maybe there aren't any good mobile benchmarks out there, hopefully someone will fix that.

    1. Monty Burns

      Re: Proper mobile benchmarking

      "These type of toy benchmarks were mostly exterminated from articles about PCs about 10 years ago..."

      You must be reading VERY different webpages to me! Tomshardware, for example is often covered with them.

      1. Simon Harris
        Happy

        Re: Proper mobile benchmarking

        Time to bring back the Sieve of Eratosthenes from Byte and the 8 benchmark tests from the early days of Personal Computer World!

        1. EddieD

          Re: Proper mobile benchmarking

          Or even the list in ZD publications in the era of Whetstone and Dhrystone that included such gems as:

          slyandthefamilystone - a measure of how funky your PC is

          rollingstone - a measure of how well your aging PC can perform compared to modern units

          1. Dave 126 Silver badge

            Re: Proper mobile benchmarking

            >Tomshardware, for example is often covered with them [synthetic benchmarks]

            Yeah, But Tom's put their synthetic benchmarks in context, and always alongside 'real-world' tasks.... so if they are testing Workstation GPUs, for example, they run tasks in a variety of CAD and transcoding applications. If they are testing CPUs, they might run standard tasks in Photoshop and other productivity apps. Gaming hardware is tested on popular titles such as Crysis and Skyrim, since different games tax CPUs and GPUs differently. Seems reasonable.

            1. Monty Burns

              Re: Proper mobile benchmarking

              Agreed! And they usually do!

              But his statement is still plainly wrong.

    2. Giles Jones Gold badge

      Re: Proper mobile benchmarking

      The benchmark should specifically set the Mhz/Ghz on the CPU and GPU to stop this being bodged.

    3. Malcolm Weir Silver badge
      WTF?

      Re: Proper mobile benchmarking

      Ah, but are they really cheating? The handset is actually running the benchmark, and if the battery life is worsened while benchmarks are being run, then surely it is a failure of the individual performing the tests that they assume stupid things about the linearity of current draw under varied conditions.

      "Cheating" would be if the system didn't do the work, e.g. by optimizing out routines that it had been told weren't important to the benchmark.

      All this is is a system that is optimized to run those benchmarks. Whoopie. If that really matters to you, is there anything fundamentally preventing you from applying those same optimizations to whatever app you care about (i.e. stuff the name of the app into the tweak file)?

  5. Busby

    Cheating, what you mean like BT Broadband does with speedtests? Shouldn't even be surprising theres a lot more of this goes on than people realise.

    1. Badvok

      In the interest of fairness I'd also like to point out that Virgin Media have been suspected of the same. Brilliant speed test results but try to stream a movie - no chance. A pity it is so hard to find out the truth in that market though.

      1. Captain Scarlet Silver badge
        Unhappy

        But this depends on the links your ISP has, how utilised they are, quality of the exchange etc...

  6. Homer 1

    re: bootnote

    Unlike iCultists, I'm disinclined to defend bad behaviour, or "evangelize" for any faceless corporation, for that matter.

    1. Joseph Lord
      WTF?

      Re: re: bootnote

      So that is why your comment disparages the customers of another company instead of making any statement about the unethical and possibly illegal (Unfair Commercial Practices Directive may stretch this far and could it even constitute fraud) behaviour of Samsung. Well that's OK then because you didn't defend them.

      1. danbi
        Thumb Up

        Re: Unfair Commercial Practices Directive

        This is an interesting idea. However, Samsung are not *that* stupid...

        What Samsung did was to trick the reviewers and it was the reviewers who outright "lied" to the public, in part by not actually checking the "facts" of their findings. Will these reviewers be punished? I doubt that very much.

        If Samsung were stupid enough to actually refer to those benchmarks in their advertising materials and thus directly deluded consumers, that would merit punishment. Otherwise, they are safe.

        Let's hope this story will actually make the reviewers more wary and less prone to praise whoever vendor gives them new gadgets to play with.

  7. Steve Knox
    Holmes

    Of course it is.

    Here's a li'l fairness v. bias test we suggest you might find personally illuminating. Read the story above one more time, except each time you see the word "Samsung", substitute "Apple".

    Then ask yourself: "Is my response any different?"

    Sure is.

    With Samsung, my response is "Poor Samsung got caught doing what all major Android manufacturers are doing these days, because their market is saturated with people who take these benchmarks seriously. Stupid but understandable."

    With Apple, my response would be "WTF Apple? Your target customers don't read benchmarks! They come to you for shiny and cool. Why'd you waste resources trying to game a system you don't depend on!?"

    1. Karlis 1

      Re: Of course it is.

      How many 'major' android manufacturers are still out there?

  8. Anonymous Coward
    Anonymous Coward

    WTF?

    They didn't bother to disassemble the code? Sigh. Kids today...

  9. Zola
    Facepalm

    Something else the 5410 contains

    A defective CCI-400, or Cache Coherency Interconnect, which means the 5410 can only operate in "Cluster Migration" mode, which is big.LITTLE's least useful mode of operation.

    Thanks to the broken CCI, the 5410 will only ever be running all A15s at once, or all A7s, but never a mixture of the two cores ("CPU Migration" mode), and when the 5410 migrates all of the tasks from one cluster to the other it has to flush all of the caches to main memory resulting in a significant power and performance penalty.

    See Anandtech.

    Basically, the 5410 is a bit of a cluster f**k, pardon the pun, and hopefully the 5420 has allowed Samsung to correct this glaring mistake. I wonder if they rushed the 5410 to market to meet the schedule of the international Galaxy S4 in which it is used (avoid the international GS4!)

  10. Karlis 1

    Meh

    I'm a consumer. Well - an ex mobile developer that used to care about this profoundly. No more.

    There isn't all that much difference between Galaxy S2 and 3 or JesusPhone 4S and 5 to warrant bothering at this level of detail. Whoever is 'benchmarking' phones these days is probably a rather misguided effort. Nobody cares. It has been a PIII-733 with a GeForce 1 card by default for at least 2 years now.

  11. Anonymous Coward
    Anonymous Coward

    This is nothing new

    Intel benchmarks have been inflated for decades but only in recent years have reviewer discovered and confirmed this conspiracy to dupe the gullible. If you want to see how Intel and AMD processors actually perform, test them with real applications not benchmarks that look at what processor brand is being tested.

    1. xyz Silver badge
      Paris Hilton

      Re: This is nothing new

      I think it's only men who would be shocked by this. Women automatically take about 30% off any measurement given to them in respect of performance or size.

  12. plrndl

    Apple

    Apple doesn't sell its products on specs, but "user experience", so it wouldn't bother to fake the benchmarks.

    1. g e

      Re: Apple

      Surely an Apple benchmark would olny be really relevant to another Apple product unless anything else uses the same chips+architecture ?

      And all Appleists need anyway is to be told it's faster so yep, no point benchmarking.

    2. Anonymous Coward
      Trollface

      Re: Apple

      Is that sarcasm? Because it's hard to tell.

    3. Anonymous Coward
      Anonymous Coward

      Re: Apple @plrndl

      "prndl" surely?

  13. Phil O'Sophical Silver badge

    Nothing really new here

    Years ago I remember hearing how one now-gone computer manufacturer had a tweak in their compiler which was able to recognise the code for a common benchmark (SPECmark?) and, when it found it was compiling that code, it inserted an optimised block of machine code that could just all fit in the processor cache. It flew on that benchmark! Cheating or clever optimisation? You decide.

    1. Steven Raith

      Re: Nothing really new here

      I suppose it really depends on whether that optimisation works in the main application the benchmark is based on - if there is one.

      Optimising for Antutu for example, is a dirty trick as it's used as nothing but a performance benchmark.

      Optimising for a benchmark based within an industry standard app, like Maya - If you use Maya, and the CPU is optimised for it, that's a good thing from the Maya users point of view.

      Of course, if it only works in the benchmark within Maya and doesn't work for the main program in day to day tasks, then it's a dirty trick.

      Context is king. But this instance *appears* to be dirty. It seems to be a workaround for the 'wait' as Exynos ramps up to it's big cores from the little ones (As flushing them out takes time that would skew the benchmark) but if that's what they were doing, they:

      A: Should have said so

      B: Should have implemented it in userland as a feature.

      Otherwise it just looks underhand.

      All IMHO.

      Hoping someone (ideally AT with their deep dives) can try to replicate this on other devices to see who else is doing this, be interesting to see how widespread it is...

      Steven R

      1. Anonymous Coward
        Anonymous Coward

        Re: Nothing really new here

        Well, it is implemented in userland and managing DVFS frequencies from userland on Linux is a normal thing to do. Shockingly, you'll probably also find similar code in other 'optimised' android releases from most OEMs. You won't see such things in AOSP. Generally you'll require root to change DVFS governor or do similar things yourself.

        Plus of course, there is a more general mechanism through power HAL where max frequencies are limited when the screen is off etc.

        The fishy bits are that you can't achieve the same GPU clock yourself and that the increased thermal headroom is only allowed for specific apps. If both of those were time-limited features available on-demand to any app subject to device conditions, it would be all above board.

        At least with Android being so open (don't laugh!) you can see what's going on. We have no clue what other OS' are doing in this regard.

        1. Steven Raith

          Re: Nothing really new here

          I agree generally - although to clarify, when I said userland, what I really meant was 'exposed to the user as a 'turbo boost' feature. My bad :D

          That GPU clock thing is definitely shady, though.

          Steven R

          1. Anonymous Coward
            Anonymous Coward

            Re: Nothing really new here

            And it seems confirmation that HTC use the same technique is already out there if anyone looks.

            http://forum.xda-developers.com/showpost.php?p=36033515&postcount=744

            I suspect all the Android OEMs will be doing something like this for their optimised ROMs, DVFS is not perfect and a lot of the mobile benchmarks have an execution model which is just about the worst case for a usage-based frequency monitor - sleep for a while, wake up and calculate for a short time, go back to sleep. Repeat for each sub-test.

            Shitty benchmarks given too much weight by shitty reviewers leads to shitty optimisations.

  14. Anonymous Coward
    Anonymous Coward

    meh - it's Shamsung

    it goes with the territory.

  15. Anonymous Coward
    Anonymous Coward

    bootnote

    thanks for the bootnote, a fairly good test, if my moral benchmarking hasn't been rigged I'm relieved to think, that I would have felt no more amused / outraged, had the hero of this article been Apple.

    BUT wait! - this test is flawed - if I substitute Samsung with Apple, I already know it's a just a test, and the real (...) are Samsung. So... perhaps my internal benchmarking is rigged with anti-Apple strings anyway! :)

    I think, the true test of "are we biased against Apple" (hell, of course we are! ;) would be to run such article claiming it was Apple - and THEN telling people to compare how they felt, as the real culprit is Samsung, as people's reactions to Apple are, generally, more... emotional and strong, than of any other company. But then, oh dear, Apple response to such a "test" article would have been swift and brutal, with the whole Register probably taken down by lunchtime, and claim for 100 billion £ in damages lodged by end of business day...

    p.s. can, we, in the wake of the findings, expect an industry-wide wave of "minor" firmware updates coming up in the next few days from all major and minor phone manufacturers? ;)

    1. danbi

      Re: bootnote

      Yet, Samsung stays mum... that has to tell us something.

      But you are correct. If Apple were caught to do something like this, that would enrage a lot of people -- for no other reason, than the fact that Apple does not have to make such tricks to sell their stuff. Nor do Samsung, but for some reason they don't seem to understand it.

  16. Anonymous Coward
    Anonymous Coward

    Analogy

    Although if you did an analogy of a Formula One car.

    If there were benchmarks for the engines and pre season every car was showing off that their new engine was the fastest ever. They might set up a benchmark that measures, power and torque. Now for the pre-season benchmark they would run it at maximum for the three minutes to do the test (end then bin the engine or resign it to the test car). However during the season they run the cars at slightly lower performance due to the fact the engine has to last for a few races.

    This doesn't mean the engine can't run at that benchmarked speed, just that they choose not to as other factors need to be considered.

    Similar to a benchmark, it is saying that the processor is capable of running at that high speed and can be software activated to do it - for instance if a specific piece of software needed it, however for day-to-day use they optimise it for other factors. They aren't inserting fake results into the benchmark or writing over memory addresses with different stats for instance.

    If you look at the battery stats for a device and it says 4 days - they have obviously optimised very few if any applications running, and closed down any background data or unneeded radios. In real-life however YMMV.

    1. Tom 38
      FAIL

      Re: Analogy

      That is a shitty analogy. At any point the F1 team can choose to turn up the wick and use the full power of their engines. With the phone, Samsung decide that you can only use the full power of the GPU if you are running benchmarks.

      1. Anonymous Coward
        Anonymous Coward

        Re: Analogy

        Samsung ("the team") enabled the higher performance for other apps as well if they are needed. They can up the maximum when required by certain apps. Same as the F1 team could run it for the last lap of the race when the engine is to be retired that race. The user ("driver") can't exceed predefined limits.

        1. Anonymous Coward
          Anonymous Coward

          Re: Analogy

          Samsung ("the team") enabled the higher performance for other apps as well if they are needed.

          Worse than an iTard. Keep sucking the teat.

    2. danbi

      Re: Analogy

      The morale: never agree to buy an reviewer's (Samsung) phone second hand, it's CPU might have not much life left.

      There is a reason why the 24 hours of Le Mans race is so important for car manufacturers.

    3. Mike Dimmick

      Re: Analogy

      Cars in need of extra sponsorship often perform significantly better in pre-season testing than in race trim. In pre-season, they don't need to pass scrutineering, so can be below minimum weight, and don't have to provide a fuel sample after qualifying, so can run on just barely enough fumes to get round a lap.

  17. ukgnome
    Joke

    I benchmarked my phone

    And that's the last time I put it on a bench

  18. Sensi

    Hmm, every cpu, gpu or browsers are optimized against/for popular synthetic benchmarks whatever are the merits in real use, cheating would be to fake a processing power it can't achieve.

  19. heyrick Silver badge

    Difficult situation

    People, and the company, are going to want their product to look great in the artificial benchmarks despite real world requirements rarely needing so much power. It is a bit of a braindead way to do it, as opposed to something like monitoring system load and switching up when the extra oomph is required.

    And yet, people would bitch and moan if their battery life sucked, because not only do we want it faster and better, we want it smaller/thinner/lighter, meaning a big chunky battery to run the thing is out of the question.

    And yes, my response would be the same for Apple; although with slightly more sarcasm.

  20. Cuddles

    What cheating?

    The benchmark results are entirely real. Samsung simply made sure that they actually measured the maximum possible computing power of the device, which is the whole point of benchmarks in the first place. The fact that it's necessary for Samsung to do this isn't a problem with them, it simply shows just how badly benchmarks suck at doing what they're supposed to. Hell, just look at one of the "problems" -

    "When running v.2.7.0, however, the Exynos 5 Octa switched over to its less-powerful Cortex-A7 cores."

    I'm seriously supposed to be upset that Samsung made sure a performance benchmark used the high power cores instead of the low power battery saving ones? Why the fuck was the benchmark trying to use the wrong cores in the first place? Unless you're suggesting that in the real world the phone will never use the high power cores at all, any benchmark that doesn't use them is fundamentally broken.

    As for replacing Samsung with Apple, I'm with Steve Knox there. Apple simply have no need to do this because they compete on different terms. An Android phone is an Android phone. They can't compete based on ecosystem, apps, and so on, so performance is a meaningful and potentially important metric. Apple phones run an entirely separate OS with an entirely different ecosystem built around it, so trying to compare raw power would be completely pointless. I still wouldn't see anything wrong with Apple doing this, I just wouldn't see the point.

    1. danbi
      WTF?

      Re: What cheating?

      The difference is, if Samsung chose to let their CPU run at these speeds all the time, battery life would be severely impacted (up to unusable), the handset temperature would increase (up to frying or catching fire), the chips inside would melt etc. Most chips have limited lifetime if exposed to high temperatures, so do most of the other components.

      Also, you erroneously assume that the benchmark has any control over what cores it runs on. This is all done "transparently" by the OS. Samsung obviously decided to do this trick, because otherwise if left to the default core scheduling they would reveal just how bad the real-world performance is.

      They were however not smart enough to account for newer versions of the same benchmarks.....

      1. Dave 126 Silver badge

        Re: What cheating?

        Is is sustained high temperature that shortens the lifespan of components (other than the battery), or is it the cycling between hot and cool that does the damage? (moot point, I know)

      2. Philip Lewis
        Holmes

        Re: What cheating?

        "They were however not smart enough to account for newer versions of the same benchmarks....."

        Prediction is a tricky task, especially of the future.

    2. jz100

      Re: What cheating?

      I agree, but this world is full of idiots. The people trashing Samsung are most likely Apple fanboys because they have a hatred towards Samsung.

      I'm more angry with Anandtech, because they KNOW that all companies do this, but yet they chose to publish this defamation article anyways.

  21. rvt

    Does non of these so-called test labs own a ecent spectrum analyser? Then they could have seen ages ago if the devices under test is cheating or not.

  22. jz100

    I can see there are a lot of idiot posting here, who obviously are computer illiterate.

    Next time you get a driver update from AMD or Nvidia, look at the patch notes and see how performance was improved for certain games AND benchmark programs.

    Samsung makes their own cpu and gpu, so they are the ones who write their own drivers for it. Your an idiot if you think they don't have the right to improve performance. And personally, I hope they do sue Anandtech over this.

    1. Philip Lewis
      Pirate

      "And personally, I hope they do sue Anandtech over this."

      Since truth is an absolute defence in a libel case, I am wondering on what basis you think they might successfully sue?

  23. jz100

    I have lost ALL respect for Anandtech over this. Anandtech is the one playing dumb, pretending they didn't know that ALL companies do this. ALL companies tweak their drivers to improve their benchmark scores in games AND benchmark software. You are living in a cave if you didn't know that, either that or you are computer illiterate.

    Anandtech knows this also, and they chose to trash Samsung anyways. Which means they probably could be sued for defamation.

    1. heyrick Silver badge

      Posting it twice won't make it happen...

      "Which means they probably could be sued for defamation."

      Samsung might be able to sue Anandtech for providing skewed reports if Samsung can find just one other manufacturer that does the same that was not picked on by Anandtech; however I very much doubt that Samsung will have much in the way of grounds to sue for defamation if Anandtech's little revelation is entirely truthful (although depending on where Anandtech are, Samsung might be able to clobber them for poking around the binaries and some measure of disassembly).

      If I was a Sammy guy, I'd be inclined to say: It's simple. When you are running a benchmark you want to know what your device is capable of. When you are using it normally, you'll want to trade processing power for battery life - you don't need to run everything at full speed. (duh.)

      And, if I was a Sammy guy, I'd be inclined to tell the techs to write a better resource scheduling thingy that responds to system load instead of specific known signatures.... and to stop hiding incriminating-looking crap in the binary blob. ;-)

      1. Anonymous Coward
        Anonymous Coward

        Re: Posting it twice won't make it happen...

        So, by that logic, if I called Al Capone "a crook" because he did crooked things which I could prove, he would be able to mount a successful defence against my slanderous comments, if I had failed to also call Frank Nitti a crook.

        Bizarrely you seem not to require me to have evidence that Frank Nitti was indeed a crook, but in order to escape from Big Al's slander defence, I need to knowingly slander Frank Nitti, opening myself to Frank's defence.

        A little late in the night for coherent thought?

        1. heyrick Silver badge
          Happy

          Re: Posting it twice won't make it happen...

          Are you replying to me?

          "So, by that logic, if I called Al Capone "a crook" because he did crooked things which I could prove, he would be able to mount a successful defence against my slanderous comments, if I had failed to also call Frank Nitti a crook."

          No. That isn't what I said. I do not expect Samsung to be able to claim defamation (at all) if the reports are entirely correct. Who or what has been defamed by reporting the provable truth? It is possible, however, that Samsung could try claiming journalistic bias and unfair reporting; a consideration for discrediting the report itself (akin to the "We All Do It, Why Pick On Me?" - or The Lance Armstrong Defence, perhaps?).

          [update: it looks like Anandtech may have picked up upon similar behaviour from others; re. the updated post linked below; if so, this'll close that avenue of approach and Samsung will just have to lump it; although I must say I found Samsung's response to be...bizarre.]

          Likewise, I wouldn't expect Al Capone to claim slander if he can be proven to be a crook (though given his line of work, just shooting you might have been more productive than a pointless legal tussle...).

          "A little late in the night for coherent thought?" - and a little late in the night for reading what was actually said instead of drawing (incorrect) conclusions?

      2. danbi

        Re: Posting it twice won't make it happen...

        "And, if I was a Sammy guy, I'd be inclined to tell the techs to write a better resource scheduling thingy that responds to system load instead of specific known signatures.... and to stop hiding incriminating-looking crap in the binary blob. ;-)"

        Those poor tech employed by Samsung were obviously asked "Guys, what can we do to have our stuff perform better", and the folk likely answered:

        - we could do proper research, better design, implementation and testing: this will take X years;

        - we could attempt to write better scheduling software to make our product perform better (*): this will take X months;

        - we could write a small utility, that detects when a benchmark is being run and push up the limits of our stuff. That could fry the thing, but we know people typically run benchmarks for a short time and observe their things during that times: this will take X weeks.

        Guess which option the Samsung management chose?

        (*) By now we know, that the original 5410 "Octacore" has defective CCI and that better scheduling was in fact impossible. Samsung engineers might or might not have known this at the time. Most likely they did.

        In any case, I believe the stupidest thing Samsung can do is to attempt suing Anandtech. Going to court might force Samsung to reveal things they better keep secret.

  24. Suburban Inmate

    Update here

    http://www.anandtech.com/show/7192/update-on-gpu-optimizations-galaxy-s-4

    A response from sammy and a lot more digging by the folks at anandtech. Make of it what you will.

  25. This post has been deleted by its author

    1. danbi

      Re: Is this more political than technical?

      It is technical.

      The Samsung's chip (Exynos 5410) has severe design bug in that it contains a non-working CCI. That means, that it

      - cannot run all the 8 cores at the same time;

      - cannot run the big.LITTLE cores, as intended by the architecture (each big core has a LITTLE pair);

      - when switching from "low power" to "high power" (A7 to A15 cores) it has to flush/reload cache which makes for abysmal performance.

      All this means that it's "performance" is severely impacted and way under the promised values. About the only way to "fix" this is to lock all CPU cores running at the maximum frequency, no power management, no gating, no big.LITTLE switching. In short: a big lie.

      It is as if you were offered an car engine, with "exceptionally high elasticity and dynamics, low fuel consumption, high power etc"... only to find out that under normal usage, it becomes unresponsive for a while when switching from low power to high power more, the low power and high power modes are very distinguishable and boringly not smooth and you can never, ever reach the promised performance. But, when the engine detects it is being performance tested, it would run full throttle, maximum power and no consideration whatsoever on fuel consumption -- with the risk of burning in flames. Would anyone buy such a car? I doubt very much.

      1. heyrick Silver badge
        WTF?

        Re: Is this more political than technical?

        Hmm, just looked that up, so I'll withdraw my comment. The question I have now is why are they using a known defective SoC in a production device, and given this, who'd want to buy one? It seems...silly.

        I get that the mobile phone market is cutthroat and all, but when you're shooting yourself in the foot, might be time to rethink?

  26. xQx

    It ain't cheating, it's putting your best foot forward over lazy journalists!

    Really, I don't think it's cheating. It's just putting your best foot forward. The chips are capable of that speed, so why not turn all power saving off when someone launches a cpu-test application.

    Obviously if you jailbreak the phone you are going to be able to tweak it to get the same CPU figures for your favorite apps.

    Maybe this will force tech journalists to do a real job when reviewing a product by actually testing how it performs in the real world, not just loading some fancy-pants benchmark application and cut'n'pasting the results into their articles.

    It's not like they're cheating on compliance tests like LG does:

    http://topnews.co.uk/23081-lg-cheats-energy-efficiency-testing-launches-801-liter-dios-refrigerator

This topic is closed for new posts.

Other stories you might like