back to article Dan Kaminsky is an expert on DNS security – and he's saying: Patch right God damn now

Dan Kaminsky, the man who could have broken DNS but fixed it instead, is warning that the glibc bug found by Red Hat and Google could be much worse than anyone has predicted. "I've seen a lot of bugs, but this bug was written in May 2008, right at end of my own patching effort on DNS," Kaminsky told The Register on Friday …

  1. Tromos

    Buffer overflows in 2016 are an embarrassment

    Given that buffer overflows were an embarrassment decades ago, it amazes me that processor memory management hasn't stepped up to take care of this and take it out of the hands of the coders. You can't mend stupid.

    1. Richard 12 Silver badge

      Re: Buffer overflows in 2016 are an embarrassment

      They can't.

      The OS can do something - and does with ASLR and killing a process that tries to access memory the OS doesn't think it should.

      The next line is the standard C/C++ runtime libraries, such as glibc, msvcrt etc.

      These do the allocation and bounds checking.

      If there is a bug in OS or standard libraries, then any application can have trouble.

      That's before considering bugs in actual applications.

      Memory management is a very hard problem in general.

      Recently I've been banging my head against a memory management bug in a commercial hardware driver - which glibc detected.

      I can't fix it because it's closed source.

      1. This post has been deleted by its author

      2. Warm Braw

        Re: Buffer overflows in 2016 are an embarrassment

        They can't

        Actually, they can, and some have. There have been commercial capability-based systems, though the additional complexity means that they have not really caught on.

        I can't help feeling that if you've got the spare die space for invisible operating modes running opaque binary blobs for "system management" purposes, the argument for omitting bounds, type and access checking is looking pretty weak.

        1. Charles 9

          Re: Buffer overflows in 2016 are an embarrassment

          They can't if high performance or tight memory is a simultaneous and conflicting issue. Bounds checking creates both time and space overhead.

          1. Warm Braw

            Re: Buffer overflows in 2016 are an embarrassment

            Bounds checking creates both time and space overhead

            So does virtual memory - we've managed to live with it...

            1. Anonymous Coward
              Anonymous Coward

              Re: Buffer overflows in 2016 are an embarrassment

              Not in high-performance or memory-tight (like embedded) situations, we haven't Both of them have to work pretty much to the metal (due to time constraints with the former and space constraints for the latter).

            2. Geoffrey W

              Re: Buffer overflows in 2016 are an embarrassment

              Moving blocks of memory around locations is such a basic operation down at the machine level, which is performed in almost all higher level operations, that adding bounds checking at processor level has a huge impact on performance and so is left to the programs to perform at their own discretion. Or not, as the case may be.

    2. Michael Wojcik Silver badge

      Re: Buffer overflows in 2016 are an embarrassment

      it amazes me that processor memory management hasn't stepped up to take care of this and take it out of the hands of the coders

      Done decades ago, with capability architectures. The AS/400 did it in 1988,1 as did the Flex system at roughly the same time (though I don't think the Flex was available commercially). The Intel iAPX 432 did it in 1981, though poor performance killed it after a few years in production. The System/38 did it in 1978. The Plessey 250 did it in 1970.

      Of those, only the AS/400 (now IBM i) is still available. But it is available. Most people just choose to use processors with much coarser memory-management models because they're cheap.

      1Wackypedia claims otherwise, citing a book by Frank Soltis; but having written OS/400 code on both CISC and RISC implementations, under the OPM, EPM, and ILE models, and with the MI and SystemC "low-level" languages, I rather disagree. For all practical purposes, application code ran under a capability-addressing model. Storage could only be addressed through 128-bit "space pointers" which could only be created by the MATPTR instruction, were invalidated if modified, and had range information. Overflowing a buffer produced an immediate trap (a machine check or program check).

  2. Ole Juul

    Party like it's 2008

    I have most machines patched now. Interestingly, one server provider sent me an e-mail saying not to worry, they're going to automatically patch for all clients. I would have done it myself but that's probably a wise move on their part.

  3. Ken Hagan Gold badge

    Lingering in the cache

    "But here's the kicker: let's say the attack doesn't work, but the payload lingers in the ISP's DNS cache."

    For most broadband customers there is probably an additional cache in their home router. Such devices are rarely restarted, and in the vast majority of cases never patched, so it *will* linger there.

    1. Ian 55

      Re: Lingering in the cache

      Except that the router probably doesn't use glibc.

      Probably.

      1. Anonymous Coward
        Anonymous Coward

        Re: Lingering in the cache

        Also what I read, but it isn't touched on why or how, but most now run dnsmasq (as I do on my home network using a PI, not my router (all that does is route!)) and that somehow fixes the head on attack:

        From here:

        http://dankaminsky.com/2016/02/20/skeleton/

        "Local resolvers are popular anyway, because they mean there’s a DNS cache improving performance. A large number of embedded routers are already safe against the verified on-path attack scenario due to their use of dnsmasq, a common forwarding cache."

      2. Chris King

        Re: Lingering in the cache

        In which case, the router itself won't be affected, but clients behind it could be. The router would be a carrier of the nasty, rather than a victim itself.

      3. Michael Wojcik Silver badge

        Re: Lingering in the cache

        Except that the router probably doesn't use glibc.

        It seems to me that Linux-based SOHO routers are quite common. Presumably most of them use GNU userland, including glibc.

        A better question might be "under what circumstances does my home router call getaddrinfo for an IPv6 address?". Logging, perhaps. I can see that attack surface being pretty large.

        Now, all that said, the problem Kaminsky is pointing to doesn't depend on glibc being used on the system with the DNS cache - it's glibc used on any system that might receive a DNS response from said cache. So a router that uses glibc might itself be vulnerable, and any systems that use that router as a DNS server might be vulnerable.

        (And it might be worth pointing out that the use of glibc on Windows systems is not unknown.)

    2. CliveS
      Unhappy

      Re: Lingering in the cache

      "For most broadband customers there is probably an additional cache in their home router. Such devices are rarely restarted, and in the vast majority of cases never patched, so it *will* linger there."

      Unless you're a Virgin Media customer with their Superhub. In which case you're probably used to rebooting the wretched thing on an almost daily basis. But your point stands.

      1. Down not across

        Re: Lingering in the cache

        Unless you're a Virgin Media customer with their Superhub. In which case you're probably used to rebooting the wretched thing on an almost daily basis.

        I can't remember when I last had to reboot the SuperHub. Then again it has been in modem-mode since it arrived, with a router (which only routes and doesn't do DNS) behind it.

  4. Christoph

    The malware is sent every time someone looks up evildomain, and lingers in the ISP caches.

    Is it not possible for ISPs to write something to scan their caches, and for security firms to scan through domains checking for malware?

    If anything is found, tell everyone to block that domain.

    1. Anonymous Coward
      Anonymous Coward

      This is what I don't understand - if the DNS chain has been patched all the way through, how can a dodgy request hit the end user? It's not like people query the dodgy DNS server directly (and secondly, to set up a DNS server, you need a upstream server to AUTH you, so how does that work in the real world?).

      1. Anonymous Coward
        Anonymous Coward

        So the problem is 1) the dodgy DNS response is RFC compliant. 2) It's not easy to solve server side as caching the dodgy response for the appropriate time, is correct behaviour for all *patched* cache servers.

        As far as I can see the issue is a field in the DNS response is specified as being capable of holding N-bytes, it's rarely bigger then Y bytes so it's been special cased in glibc, likely for (premature optimization / measurable performance) reasons.

        The bug was about the improper handling of the transition between the fast path and the slow path, e.g. stack memory and dynamic memory.

        Now take away here is accepting a long value for the DNS fields is correct behaviour as per the RFC.

        So a conforming DNS Cache implementation will store this request with long fields.

        An unpatched DNS Client will pull this response choke on the large field value and still overflow, the magic of which is that eventually given enough runs, you should strike it lucky and endup running into an executable address.

        1. Anonymous Coward
          Anonymous Coward

          OK, I understand that, but you still haven't explained how a blackhat controlled DNS can get to answer queries down the chain?

          1. SImon Hobson Bronze badge

            > OK, I understand that, but you still haven't explained how a blackhat controlled DNS can get to answer queries down the chain?

            OK, so you (through whatever means) get a client to lookup some url - perhaps you manage to embed it in compromised web sites, put it in spam emails, whatever. The client looks up the url, say screwme.evildomain.com using it's internal mechanisms. The software stack on that client will pass the query up to it's configured name servers, which will pass the query on up until a recursive resolver which finds the nameservers for ervildomain.com and asks one of them for the answer to "where is 'screwme.evildomain.com' ?". The authoritative nameserver will give an answer that is carefully crafted to trigger the bug, and this will be faithfully passed back to the client - and cached by any nameservers handling it.

            A typical chain for a home user would be : user's machine -> home router -> ISPs resolvers -> scrote's authoritative servers for the query, and the reverse chain for the answer.

            So the scrote trying to use this bug doesn't need to intercept anything, he just needs to get the client to query a name in a domain for which he controls the nameservers - the standard DNS resolution mechanisms take care of getting the query to his nameservers, and the answer back to the client.

      2. Tom 13

        The way I read this is that if ALL the servers are patched, it can't because the flaw was a mismatch between allocated and used memory. The problem is that ALL of the servers have to be patched, and at least some people are normally slow to patch DNS servers since 1) they are critical infrastructure pieces, 2) historically they've been assumed to be innately more secure because they've narrowly focused at the key points. And until the LAST DNS server is patched you are still potentially vulnerable.

  5. Anonymous Coward
    Anonymous Coward

    Actually, I more confused now.

    Reading this from the CVE:

    "Multiple stack-based buffer overflows in the (1) send_dg and (2) send_vc functions in the libresolv library in the GNU C Library (aka glibc or libc6) before 2.23 allow remote attackers to cause a denial of service (crash) or possibly execute arbitrary code via a crafted DNS response that triggers a call to the getaddrinfo function with the AF_UNSPEC or AF_INET6 address family, related to performing "dual A/AAAA DNS queries" and the libnss_dns.so.2 NSS module."

    Right. I build my own kernels - no IPV6 built-in. dnsmasq, running on my network PI DNS/DHCP server is built with with no IVP6 support [-DNO_IPV6] (IVP6 disabled on Pi kernel boot line anyway).

    So I am confused as to what the 'dual A/AAAA DNS' queries are - if I don't have dual, how can I?

    1. Anonymous Coward
      Anonymous Coward

      Re: Actually, I more confused now.

      A record is Give me an IPv4

      AAAA record is Give me an IPv6

      modern stacks use getaddrinfo(3) which returns a linked list of addresses which may be either IPv4 or IPv6.

      This either IPv4 / IPv6 result is a "duel stack" query.

      getaddrinfo(3) takes a flag which can restrict which type of address to query for, the "correct" thing in new code is to accept any address so you can be ready for IPv6, hence the duel stack "A/AAAA" queries.

      1. Anonymous Coward
        Anonymous Coward

        Re: Actually, I more confused now.

        But, but, but - if my kernel isn't built with IPV6, then surely the request isn't processed but dropped? Reading the REDHAT stuff, they state disabling IVP6 doesn't mitigate it, but that is surely because their Glibc and kernels are built with IVP6?

        1. SImon Hobson Bronze badge

          Re: Actually, I more confused now.

          > But, but, but - if my kernel isn't built with IPV6, then surely the request isn't processed but dropped?

          This has nothing to so with the stacks compiled into your kernel. The client programs will probably still make dual-stack queries, and get dual-stack replies for services with both A and AAAA records. When your client program (browser, email client, whatever) get the reply, it'll see that there are no IPv6 interfaces it can use and so will ignore any AAAA records it's given.

          But the DNS lookup and result will still be the same, so still capable of triggering the bug.

          It's possible that the client may see that it has no IPv6 interfaces and so only request A records - but I suspect most clients won't bother doing this. In a way, while it would be more efficient on DNS, it's redundant since AAAA records will get ignored later when the code (which must be there) is looking to see which interfaces it can use and selecting one.

  6. Anonymous Coward
    Anonymous Coward

    I have a retina iMac

    I'm smug but am I safe?

    1. Anonymous Coward
      Anonymous Coward

      Re: I have a retina iMac

      iSmug, I would say.

    2. Crazy Operations Guy

      Re: I have a retina iMac

      OS X uses the same glibc as the vast majority of open source OSes, so its quite likely you are vulnerable. Also, what the fuck does your type of monitor have to do with the software running it?

      1. Anonymous Coward
        Anonymous Coward

        Re: I have a retina iMac

        I was being smug.

      2. stephanh

        Re: I have a retina iMac

        OS X does not use glibc, it uses the Darwin libc, see http://www.opensource.apple.com/source/Libc .

        This is ultimately derived from the BSD project, like the rest of the core OS X (Darwin) system. (Somewhat simplified, Darwin = OS X without the GUI stuff, or alternatively, the parts of OS X which are open source since they derive from BSD.)

        1. Anonymous Coward
          Anonymous Coward

          Re: I have a retina iMac

          Me too!!

    3. AlbertH

      Re: I have a retina iMac

      I'm smug but am I safe?

      Not if you're smug!

      However, you are an iMug for paying outrageous money for cheap commodity hardware with an expensive badge stuck on.

      1. david 12 Silver badge

        Re: I have a retina iMac

        >outrageous money for cheap commodity hardware with an expensive badge stuck on

        Still living in the 00's ? In 2016 Apple OSX laptops are not cheap commodity hardware: they are top-line hardware at a reasonably competitive price. I wouldn't put Win8 on one, but that's because Apple provides crap Windows drivers for the hardware. Running OSX, you'r paying top price for a top quality laptop.

  7. Tim Brown 1

    It's the nature of security consultants to big-up the problem

    Not that I'm complacent, I patch the Linux servers I manage at least every week.

    However security consultants like to make the latest bug sound like the end of the world, when really it isn't and isn't anywhere near. Well-managed servers will get patched in a timely fashion, some badly managed servers will get deservedly bitten, need to be rebuilt, and in the process we may get to learn who the IT-incompetent companies are (I'm looking at you Talk-Talk).

    The world will keep turning and a few more cowboys will go to the wall.

    1. ecofeco Silver badge

      Re: It's the nature of security consultants to big-up the problem

      Up voted. I've met those cowboys.

  8. Crazy Operations Guy

    Porper use of malloc()

    Am I the only person that when a function accepts input from a network, I have the function do malloc( NET_INTERFACE_MTU * INPUT_SEGMENTS) for any unknown-length inputs? I then copy a specific number of bytes out of that variable into the variable I'll actually work with. I do the same when working with files; malloc(FS_BLOCK_SIZE * FS_BLOCKS).

    RAM is cheap, and with the typical packet being a mere 1.5 KB each and filesystem blocks being 512-1024 Bytes, I don't mind using extra RAM if it means preventing buffer overflows, especially since nowadays even the higher-end RAM is less than $10/GB...

    1. Anonymous Coward
      Anonymous Coward

      Re: Porper use of malloc()

      Ah, but did you write the Glibc libs? Maybe easy on a 2Kb lib.

      1. Crazy Operations Guy

        Re: Porper use of malloc()

        This is in my programs, I'm not modifying glibc. My point is that when you are accepting input, never assume that its the size you expect, so plan for it being way too large. A buffer under-flow can be safely handled by an exception in whatever piece of code is handling the data, a buffer overflow, on the other hand, could mean that the system is fully compromised.

  9. Anonymous Coward
    Anonymous Coward

    Oh DNS

    Such a BIND

    1. ecofeco Silver badge

      Re: Oh DNS

      You funny. Up voted.

  10. Anonymous Coward
    Anonymous Coward

    I could tell you a DNS joke. But it would take you 86400 seconds to get it.

    1. ecofeco Silver badge

      I'd have to look it up.

  11. Anonymous Coward
    Anonymous Coward

    Please spare us the whoring

    "I've seen a lot of bugs, but this bug was written in May 2008, right at end of my own patching effort on DNS,"

    This bug has a lot more to do with linux's glibc than with the DNS poisoning vulnerability he worked on in 2008.

    1. Anonymous Coward
      Anonymous Coward

      Please spare us the boring

      Anything of relevance to contribute, or just inexplicably pissy that someone did some work?

  12. Leeroy

    CVE-2015-7547

    For anyone running WHM \ cpanel the fix was posted a few days ago and takes 30 seconds to implement plus a reboot.

    https://forums.cpanel.net/threads/cpanel-security-team-glibc-cve-2015-7547.527651/

    I thought this was a new one but checked just in case :)

  13. Anonymous Coward
    Anonymous Coward

    Memory management is a very hard problem in general.

    @Richard 12: "Memory management is a very hard problem in general."

    So, fix it !

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like