back to article Intel didn't tell CERTS, govs, about Meltdown and Spectre because they couldn't help fix it

Letters sent to the United States Congress by Intel and the other six companies in the Meltdown/Spectre disclosure cabal have revealed how and why they didn't inform the wider world about the dangerous chip design flaws. Republican members of the House Energy and Commerce Committee sent letters to the seven in January, to seek …

  1. JimmyPage Silver badge
    Boffin

    It makes you wonder ..

    what else could be uncovered with a little basic kit.

    It's been many years since I worked in a lab, but a few logic analysers, protocol analysers, and oscilloscopes ...

    Here's an El Reg challenge: what tools would *you* equip your exploit research lab with ?

    It's much more likely there are more exploits than there aren't, I'd wager.

    1. boltar Silver badge

      Re: It makes you wonder ..

      "what else could be uncovered with a little basic kit."

      If these flaws could have been exposed with basic kit or simple proof of concept code then they would have been discovered decades ago since speculative execution has been used since at least the pentium pro and possibly earlier. The fact that it took so long shows how complex these faults are and how hard they are to exploit.

      1. Anonymous Coward
        Anonymous Coward

        Re: It makes you wonder ..

        Roughly speaking, one can summarise Spectre by saying that to keep the pipelines full, security boundaries are ignored, and no-one cared because it was thought that pipeline contents could be not read. They found a really clever way of indirectly reading those pipelines -- h/w kit not needed.

    2. Anonymous Coward
      Anonymous Coward

      Re: It makes you wonder ..

      SDR, laptop running the oldest OS ideally with all the bugs patched, and clean-boot-via-CDR enabled, storage 'scope ideally Tek, some analogue gear like meters etc, spectrum analyzer, circuit tester, frequency generator, Geiger counter/dosimeter, neural net system dual 16C/32T/256G Threadripper+8 TitanX + lots of storage, standalone backup system w/8192 bit AES, quantum annealing system.

      I'd also throw in a good old pile of screwdrivers and a 12 pound sledge etched with "CLUE" on the working face.

    3. Nick Ryan Silver badge

      Re: It makes you wonder ..

      If you've had much exposure to vulnerability exploitation at higher levels, e.g. website URLs and query/SQL timing responses, then these chipset exploits are largely an extension of similar processes. This kind of thing is very obvious after the fact, but somebody needed to make the connection and apply very similar techniques to a very different part of the execution stack.

      I nearly feel sorry for Intel, however Intel have dominated their market position and have such resources that not considering these factors is pretty unforgivable given the world's reliance on their technology. I'm not a conspiracy nut therefore I don't believe that were any malicious intents in the implementation, just that the chip designers did not know enough and were not exposed to enough security techniques to consider their implemenation. After all, in all normal operations the data is "safely protected", it's just that there are timing side effects due to execution operations and optimisations that while they don't directly expose data, the timing difference between a cache hit and a cache miss can be used to infer the protected data value.

      Did Intel engineers cut corners in the name of security? In many ways, they didn't. Their job was to make the chips execute instructions as fast as possible however their failure was in not appreciating that the difference in timing between a cache hit or a cache match which could be used to infer the actual data value. AMD chips generally performed the security check before the data comparison, Intel chips generally performed the security check after the data comparison and it's this execution difference that explains some speed differences between the two vendor's chips and the timing exploits that can be used to derive the content of otherwise protected memory.

      Now that these techniques have been exposed, there is focus on exploits at this level and I expect to see many more rear their ugly heads. Meltdown and Spectre are just for a single chip exploit, the checks become even more complicated and, potentially expensive in execution times, to rectity when cross-chip exploits come into play.

      The only real, long term fix, is a fundamental reconsideration of processors and processor design taking into account security from the start rather than as an afterthought tacked on at the end - this never works for any system.

  2. K Silver badge

    Nothing to do with .GOV assistance..

    They did it this way, purely because they do not trust them.. and with good reason.

    The Intel services would rub their hands in Glee, and force Chipzilla to sit on this for as long as possible.

    1. Alan Brown Silver badge

      Re: Nothing to do with .GOV assistance..

      "The Intel services would rub their hands in Glee, and force Chipzilla to sit on this for as long as possible."

      They probably _did_ rub their hands with glee.

      Forcing Intel to sit on it would have lead to leaks. It was better to say nothing and let them be ignorant of the hole they'd created.

  3. jaywin

    That meant the cabal felt none of the US government, the United States Computer Emergency Readiness Team or the Computer Emergency Readiness Team Coordination Center would be useful in preparing a response to the mess it made.

    Well, they're probably right. Government reaction would either be "ooh, more exploits we can use to spy on the public, er, dangerous actors" or "PAANNIIICCC!!!! TELL EVERYONE RIGHT NOW". Neither being constructive, or helpful.

    1. Loyal Commenter Silver badge

      Government reaction would either be "ooh, more exploits we can use to spy on the public, er, dangerous actors" or "PAANNIIICCC!!!! TELL EVERYONE RIGHT NOW".

      The US govt. response would most likely be, "Don't fix it, we'll exploit this thank you very much, and definitely don't tell anyone else, or there will be Consequences."

  4. Shadow Systems Silver badge

    Didn't Intel tell China though?

    I thought Intel had told the Chinese government long before the American one, so what does that say about Intel, those governments, or the likelihood that we can believe anything ELSE that comes out of Intel's spin doctors?

    1. The Man Who Fell To Earth Silver badge
      Black Helicopters

      Re: Didn't Intel tell China though?

      Since Intel has facilities in China, more likely the Chinese government could have known because it's undoubtedly thoroughly infiltrated Intel's corporate network.

    2. Commenter44655

      Re: Didn't Intel tell China though?

      I believe the issue was they told Chinese *companies*, who no doubt shared the information with the Chinese government.

      1. eldakka Silver badge

        Re: Didn't Intel tell China though?

        They also told ARM as it one of the "gang of seven".

        ARM Holdings is a British Corporation, which is 75% owned by SoftBank Group, a Japanese company.

        Therefore does this mean they told the British and Japanese governments before they told the US?

  5. BinkyTheMagicPaperclip Silver badge

    Note that they didn't bother with open source operating systems

    The only open source operating system that got a heads up was Linux, because of Intel. They didn't bother with any of the BSDs or other operating systems.

    1. John Sager

      Re: Note that they didn't bother with open source operating systems

      I suppose someone might have considered impact, in that *BSD et al are far less deployed in critical systems than is Linux (downvote farming here...). Versus the more who know the quicker it leaks, as events have shown.

      1. BinkyTheMagicPaperclip Silver badge

        Re: Note that they didn't bother with open source operating systems

        You're very charitable, I suspect they just didn't think.

        OpenBSD has been included in embargoed fixes before, and slipped the changes into the codebase so users were protected on day one of the public notice.

        Knowledge probably wouldn't have leaked from the BSDs unless someone was watching commits for all four of the main BSDs (fourth : DragonflyBSD), but yes, if every single OS out there is informed sooner or later it will leak.

        1. Orv Silver badge

          Re: Note that they didn't bother with open source operating systems

          OpenBSD has been included in embargoed fixes before, and slipped the changes into the codebase so users were protected on day one of the public notice.

          Pretty sure their history of doing that is why they WEREN'T included. If you slip something into the code repository, people can diff it and figure out what you're up to. They don't need a commit message that says SECURITY HOLE HERE to catch on.

          Kinda by definition open-source projects can't keep secrets.

    2. OldCrow
      Holmes

      Re: Note that they didn't bother with open source operating systems

      There's a reason for that.

      The flaws can only be exploited on platforms that run untrusted code. I.e. javascript, Flash, et.al. . BSD variants see mostly server use, so are not that much affected.

      If I'd been in Intel's shoes, I would have included Adobe and Mozilla on the short list. But that's the only change I'd make (off the top of my head).

      1. tom dial Silver badge

        Re: Note that they didn't bother with open source operating systems

        Omitted here: Amazon, Microsoft, and Google cloud services run enormous numbers of virtual machines that run code that is, to them and other customers on the same servers, untrusted and would make their cloud operations direct and immediate targets of both criminals and signals intelligence agencies world wide. BSDs, not so much, although possibly significant on web and storage servers where these vulnerabilities could add to the tool kit and allow malefactors to avoid the need for privilege escalation after gaining access using other vulnerabilities.

      2. Orv Silver badge

        Re: Note that they didn't bother with open source operating systems

        The flaws can only be exploited on platforms that run untrusted code. I.e. javascript, Flash, et.al. . BSD variants see mostly server use, so are not that much affected.

        IMHO the biggest risk is to shared hosting platforms, which is why Amazon was involved. This breaks down the barriers that are supposed to exist between virtual hosts. Now if you can get your VM on the same machine as a sensitive VM, you can potentially read stuff from their memory space. That's why Amazon was included, and why this has had VMware scrambling. Linux is also frequently used as a VM host. FreeBSD often runs as a guest, but is rarely used as a VM host on a commercial scale.

        `

    3. Anonymous Coward
      Anonymous Coward

      Re: Note that they didn't bother with open source operating systems

      Neither IBM nor Oracle on that list, either.

    4. bombastic bob Silver badge
      Unhappy

      Re: Note that they didn't bother with open source operating systems

      Well, according to the article, the 'insiders' comprised "Google, AMD, Arm, Apple, Amazon and Microsoft."

      Apple has at least SOME connection with FreeBSD, but there's no motive to let any of the BSD engineers know about something in the KERNEL.

      And, there MAY be a 'profit motive' in which "ONLY THEY" have fixes available, so if you don't want a vulnerable system, you "go to them" [specifically Micro-shaft and Apple].

      Had IBM been involved, I think we'd have seen more efforts in Linux, and being open source, it would 'trickle' into the BSD's pretty quickly.

      The better path is to be as open about this problem as possible. "We F'd up" is a better path than scrambling for the hills and covering your tracks.

      1. Jamie Jones Silver badge
        Devil

        Re: Note that they didn't bother with open source operating systems

        FreeBSD were notified under NDA in December, still far too late, but before the expected public release date:

        https://lists.freebsd.org/pipermail/freebsd-security/2018-January/009719.html

  6. TrumpSlurp the Troll Silver badge

    Nobody has yet

    Explained the benefit of revealing the fault as soon as it was discovered.

    No doubt there would have been class action lawsuits from shareholders becaus the share price had been damaged by premature release of price sensitive information.

    1. bombastic bob Silver badge
      Devil

      Re: Nobody has yet

      "Explained the benefit of revealing the fault as soon as it was discovered."

      Here's the alternate universe timeline that I think should have happened:

      1. Intel and AMD announce that they have discovered some flaws in their CPU design that could lead to "side channel' attacks, something that is difficult to anticipate, and NOT disclose the details.

      2. Intel and AMD work out fixes for this problem, and share 'mitigations' with all operating system vendors.

      3. Intel and AMD release new CPUs that are designed NOT to have these flaws, and provide well tested working microcode for older systems.

      4. After all of the fixes are in place, the details are released so people understand what happened.

      This would result in Intel and AMD looking VERY good. Initially they get a small hit, but being THAT HONEST about the cock-up will eventually turn around to help them. And NOT disclosing the details helps prevent 0-days from emerging.

      The thing about Meltdown and Spectre is that it's not intuitively obvious since they're side-channel attacks [for the most part]. So exploiting them without a really good explanation of what the flaw is would be "hard to guess", a sort of 'security by obscurity' that can last long enough to patch it.

      So if Intel and AMD had simply bitten the bullet and and admitted the existence of the flaw the moment it was discovered, they MIGHT have INCREASED THEIR SALES overall as a direct result, as people replace old hardware [which was previously considered 'good enough'] to avoid any slowdowns or potential un-patched vulnerabilities.

      In any case, this "alternate universe" scenario didn't happen. That made the original cock-up WORSE.

      1. Nate Amsden Silver badge

        Re: Nobody has yet

        That is just about exactly what happened. I mean maybe el reg or someone else snuck the news out hours or a day or two before Intel and others came out about it I'm not sure.

        I'm sure you mean Intel and AMD should of come out immediately following them learning of the problems(months before the public was originally alerted).

        But saying there is a big security bug and we have no mitigations right now doesn't really do most folks any good. Just look even now vendors are struggling to get fixes out.

        And in probably 98% of times you can't release mitigations for such issues without the root cause being discovered quite quickly. So Point #4 really can't happen.

        #3 is the only thing that hasn't happened yet, but Intel at least says they will have a hardware fix later in the year, and I'm sure AMD (and others) are working on it as well.

        I think the approach the people involved was a fine one. Have to balance the risks of the information leaking out before mitigation of some kind is possible, and also balance the risk of how many people know that could help, the more that know the more likely it leaks out.

        If anything with regards to OpenBSD I'm kinda-sorta surprised they didn't do this kind of thing years ago in the name of security. Someone posted a link from a mailing list where Theo was ragging on Intel (and I believe other) errata going back almost a decade, saying things like he wouldn't be surprised if it was exploitable. Sure there is a performance hit but in OpenBSD's case performance isn't their top priority.

        If the government is so upset(I really think it's only a few that want news) then they should worry much more about all of the very outdated systems they are running, many of which will never see another update.

        So much of the press and comments on this thing makes it seem like these are trivial bugs and should be easily fixed(or at least spotted a long time ago). Obviously that is not the case.

  7. Dr Mantis Toboggan
    Stop

    Easy

    The more people you tell, the more chance it leaks early before any countermeasures are ready.

    Google and Intel did EXACTLY the right thing, informed a small set of trustworthy companies that could help is addressing the technical issues, and avoiding red tape government who only involvement will be "is it ready yet" and "who is to blame"...

    1. Paul Shirley

      Re: Easy

      It would leak seconds after informing government.

  8. Primus Secundus Tertius Silver badge

    Hacks?

    "…sharp-eyed Reg hacks…"

    Question: hacks as in journalists or as in computer accesses?

  9. Anonymous Coward
    Anonymous Coward

    I think ARM was included even though they didnt have any effected chips in production...

    ...to spread the sh*t on as large a surface as possible, perhaps in the hope that the stink will be spread out sufficently that "who dunnit" is less obvious and if they are really lucky that some will stick.

    That the guilty party has spent the time since pushing everyone else in front of them as they try to siddle away from their miasma is made worse in that they don't even seem embaressed.

  10. mark l 2 Silver badge

    Another reason Intel spoke with Microsoft, Google and Amazon first is that these are some of the biggest purchasers of Intel CPU's and Intel didn't want to piss them off by not giving them the heads up, just in case they switched their datacentres to use AMD chips or even switch to ARM based SOC because they were not made aware of the flaw with advance notice.

  11. Anonymous Coward
    Anonymous Coward

    Specter / Meltdown were not mistakes...

    The 3 letter agencies have a very deep relationship with chipzilla. These flaws were not mistakes, and these exploits were not unknown. They were there purposefully to give undisclosed access to every machine running an intel proc, which at current count is about 92% of the market. You could not ask for a better intelligence gathering back door. The exposure does put Intel in the crosshairs, but really, the blame is probably more on the nsa / cia. Even if they fix this vulnerability in the next gen, I can guarantee there will be a new exploit hidden in the next architecture to give them the functionality back.

    1. Adrian 4 Silver badge

      Re: Specter / Meltdown were not mistakes...

      I might agree with you, except that I don't think they'd bother when they've got the management engine.

      1. bombastic bob Silver badge
        Black Helicopters

        Re: Specter / Meltdown were not mistakes...

        "I don't think they'd bother when they've got the management engine"

        ack

        Seriously, though, there should be a jumper for enabling or disabling the 'management engine' or its equivalent - ONLY allow it when the jumper is set to allow it, so it can be disabled electrically. even a BIOS setting to disable it could theoretically be circumvented.

        home users don't need this. Only large organizations that have an IT department with tools that can leverage 'management engine' MIGHT actually need it. But the NSA _can_ use it. That's the problem.

        And I doubt Meltdown/Spectre would be something the NSA would want anyway. side channel attacks are too inefficient.

    2. Orv Silver badge

      Re: Specter / Meltdown were not mistakes...

      I'm pretty paranoid, but I don't really buy that in this case. These security holes are a direct result of using caching and speculative execution together; they're fundamental to how the chips get the performance they do, not something that was grafted on to make them less secure. It's a bit like saying that Chevy is in a conspiracy to make me late for work because they designed cars that drive on streets and have to stop at traffic lights.

      Now, if you want to complain about the management engine, you might have an argument there; it's largely superfluous to how the CPU works, and was designed so insecurely that it would almost require special effort.

      1. bombastic bob Silver badge
        Devil

        Re: Specter / Meltdown were not mistakes...

        "These security holes are a direct result of using caching and speculative execution together"

        an interesting point. So a mitigation MIGHT be to NOT physically access anything that isn't in the L0 cache during speculative execution? And, of course, to NOT physically access any memory address for which memory flags would cause a protection fault or page fault [aka 'Meltdown']

        Doing these things in microcode would END the vulnerabilities as I understand them, with only a limited effect on execution speed.

        To mitigate those changes, a serious increase in L0 cache size would also help.

        compilers might need also need updating to help minimize the number of RAM pages that a "tight little loop" would need in order to function efficiently with out-of-order and speculative execution.

        and that 'ret-poline' mitigation COULD become an option, for people who want to do that.

        thinking of that last part, I've used 'ret-poline' techniques before. It's the easiest way to jump to a 'long' address on an ATXMega processor (and probably an ATMega also, specifically the ones with >64k of NVRAM). The alternative actually takes more assembly instructions. So you push 3 bytes and do a 'ret' instead. Yeah, Arduinos and microcontrollers. But not a new idea for a "ret-poline".

  12. FordPrefect

    Well call me a cynic but you tell the US government about exploits, the NSA will be writing exploit kits based on it. Someone finds that exploit kit and figures out what its targeting and suddenly you are in the middle of a massive sh*t storm.

  13. Henry Wertz 1 Gold badge

    screw 'em

    So, really, I'm no believer in so-called "responsible" disclosure, I'm for public exposure of flaws. I'm all for some class action lawsuits for the time between when they found out about these flaws and when they disclosed, selling known-flawed chips without so much as an errata notice sure opens one up to liability.

    That said, once they decided to hide this flaw, screw CERT, they wouldn't be helpful.. and Congress? F--- 'em. I've seen no sign of anyone in Congress ever being even vagvuely technologically competent. Intel is right, telling them all would have been zero help.

  14. Anonymous Coward
    Anonymous Coward

    Nothing new here

    Intel's only concern is the legal liability for their willful decision to violate security command execution standards. Intel could care less about U.S. national security or it's customers. That's why prosecution of Intel Corp. should include prison time for Intel execs.

  15. wownwow

    The CPU God is talking ...

    The CPU God said that it did inform other technology companies that use its chips of the issue because they are its breads and butters and determine its sales.

    The CPU God said it did not inform government officials because it’s on the need-to-know basis determine by the CPU God itself, not anyone else.

    The CPU God said:

    1) Not following the privilege levels defined by itself is the intended design and needing OS kernel relocation is a feature that others' don't have, not a bug!

    2) People who already had the "Meltdown inside" chips just enjoy the feature, no replacement!

  16. Anonymous Coward
    Anonymous Coward

    hahahahaha

    This was after Intel waited to tell OEM about Spectre and Meltdown, until the CEO's share deal to sell ticked over. that was really nice of them

    Maybe the powers to be need to look at that.

  17. Anonymous Coward
    Anonymous Coward

    @ "Maybe the powers to be need to look at that."

    Given how much else there is to look at already then a CEO abusing insider knowledge is clearly way down the list of priorities.

  18. razorfishsl

    It was Fraud pure and simple.

    Bet the stock price got shorted as well.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019