back to article Internet-wide security update put on hold over fears 60 million people would be kicked offline

A multi-year effort to update the internet's overall security has been put on hold just days before it was due to be introduced, over fears that as many as 60 million people could be forced offline. DNS overseer ICANN announced on Thursday it had postponed the rollout of a new root zone "key signing key" (KSK) used to secure …

  1. This post has been deleted by its author

    1. macjules

      Re: Who does this really affect, its hard to tell....

      Trump followers: 32m on Twitter plus his POTUS account of 18m and 22M on Facebook

      I think that we can afford to lose 60m users.

      1. Alan Brown Silver badge

        Re: Who does this really affect, its hard to tell....

        "Trump followers"

        How many once you remove the bots?

    2. bombastic bob Silver badge
      Devil

      Re: Who does this really affect, its hard to tell....

      I suspect it only affects DNS resolvers running on ISP servers, and individual users won't be affected at all, even when you're running an ancient version of windows...

      In order to state the query as 'authoritative' you'll need that key stuff mentioned in RFC 8145 is for the conversation between the 'resolver' (running on the client) and the server, using DNSSEC. I don't believe that DNSSEC is actually _REQUIRED_ though, and older servers should still work.

      I would expect older clients NOT using DNSSEC to work just fine, also.

      If you're trying to resolve the queries yourself, and NOT use an ISP server for DNS, then maybe this will affect you. Or not.

      If you're using a forwarding server or cacheing server from your ISP (or 8.8.8.8 for google's DNS server) then I'd expect it to work just fine and not break anything.

      but worst case you could temporarily turn off DNSSEC [though I doubt it would be necessary]

      The question here is MOST likely what the ISP cacheing servers will be doing, and whether those would all need to be updated. And yeah, it could cause a BIG problem if they can't resolve DNS queries any more...

      1. DougMac

        Re: Who does this really affect, its hard to tell....

        > If you're using a forwarding server or cacheing server from your ISP (or 8.8.8.8 for google's DNS server) then I'd expect it to work just fine and not break anything.

        Thats not right. If your ISP enabled DNSsec resolvers, and their system doesn't follow the automatic KSK addition mechanism that is required for the 2017 KSK key roll, *all* their lookups will fail when the old 2011 KSK stops signing the responses from '.'

        The client doesn't request DNSsec (well, it could and check itself), but all the resolvers upstream need to be able to follow the KSK addition into their keystores via the proper method if they do DNSsec resolving themselves. (which most ISP servers do, unless your ISP is a small podunk one that doesn't follow current standards). All client lookups will fail of the ISP resolver is broken.

        Since Government users typically demand that due to their standards they have to follow, most large ISPs have followed suit.

  2. mathew42
    Childcatcher

    A curated list of servers with outdated software?

    The organization is planning to publish a full list of resolvers that listed having only the 2010 KSK key, and then ask the internet community to help identify where they are and figure out what the problem is, and how to update them.

    Did anyone else read this and think that these resolvers are likely to be running out of date software with known issues? I expect certain sections of the internet community will be very quick in identifying the servers and updating them, but I'm not sure their actions will be considered 'helpful'.

    1. Voland's right hand Silver badge

      Re: A curated list of servers with outdated software?

      Most of these are incumbent telcos outside the developed world.

      No stick short of kicking them off will make them upgrade. They do not care as most of them are monopolies too.

  3. Sureo

    "60 million people could be forced offline"

    Most of them would be facebook users, so doing them a favor.

    1. FuzzyWuzzys

      Re: "60 million people could be forced offline"

      Hey, all the time they're sharing pictures of cats on Zuck's cash-cow, they're not there getting in everyone else's way. Count your blessings!

  4. Terafirma-NZ

    The problem?

    I say just do it.

    Then all the companies running DNS servers and not maintaining them will get an in-flood of angry customers some will leave and the company will learn it's lesson to not leave stale infrastructure out there.

    Heck doing it is likely to also halt a number of DDoS attack vectors due to loads of DNS servers suddenly being updated.

    1. Charles 9

      Re: The problem?

      Unless they're servers for entire regions or even companies, meaning nowhere to defect. Remember, DNS is hierarchical in nature, and we don't know how far up the tree the rot has settled.

      1. Martin Summers Silver badge

        Re: The problem?

        "DNS is hierarchical in nature, and we don't know how far up the tree the rot has settled."

        And you'd hope they'd quickly identify and fix it. Sometimes it takes things to break completely before someone does something about creaky infrastructure, in fact that's how a lot of companies work with their IT budget!

      2. Not also known as SC

        Re: The problem?

        If the servers are 'unidentified' then the only real way to find them is to implement the changes. If they pussyfoot around this then they'll never be in a position to make the switch. The broken link will soon be discovered and whoever owns/provides the DNS server can then explain to their customers why service was lost,.

      3. Alan Brown Silver badge

        Re: The problem?

        "Unless they're servers for entire regions or even companies, meaning nowhere to defect. "

        That kind of defection is accompanied by a P45 or a monopolies investigation.

        Back in the days when open mail relays were a major problem, getting japanese admins to fix their boxes was quite hard, with most either claiming they were required to keep the things open, were fully standards compliant or simply blocked complaints from people who'd been hit, making open japanese relays an seemingly intractable problem...

        ...Until some bright sparks in Tokyo hit on the idea of notifying japanese media about the problems and the TV channels delighted in naming and shaming companies which were assisting hackers/spammers by not securing their computer networks(*) - which wasn't so much loss of face as being publicly kicked in the ballsack as far as management was concerned. As a result it usually took less than a day from the time that reporters started asking questions to the time that the servers either went offline permanently or were fixed.

        (*) The public had been sensitised to the problem due to massive spam campaigns targetting mobiles.

        Management facing adverse publicity and/or investigations in western countries has about the same reaction. Embarrassment is a fantastic teacher/persuader when dealing with refuseniks (either the management who refuse to let changes be made or the admins who refuse to do it)

        1. Charles 9

          Re: The problem?

          I meant to say entire COUNTRIES. Plus you can't tell if it's YOUR country that would be affected or not since the servers could be upstream of you. Sounds fun to say, "let 'em suffer" until you discover YOU'RE suffering...AND can't change ISPs because ALL of them were affected at the same time.

    2. Doctor Syntax Silver badge

      Re: The problem?

      "I say just do it.

      Then all the companies running DNS servers and not maintaining them will get an in-flood of angry customers some will leave and the company will learn it's lesson to not leave stale infrastructure out there."

      Maybe a half-way house. Roll it out for a short period then roll back (they did have a roll back facility in their plans, didn't they? Then announce "Did you have a problem? If so you've got a month to fix it, then it's permanent."

      1. Hans 1
        Joke

        Re: The problem?

        Roll it out for a short period then roll back

        Hm, ok, I hear the Windows admins screaming already: "Could you please do it once more and wait at least 5 minutes to allow our Windows servers to reboot to apply the change, thanks!"

      2. Charles 9

        Re: The problem?

        "Roll it out for a short period then roll back."

        They're already at that point. BOTH keys are accepted. What's being pushed back is the point of no return which happens when the old key is revoked (key revocation is one-way).

    3. Anonymous Coward
      Anonymous Coward

      Re: The problem?

      "I say just do it."

      Yeah, just fuck over millions of customers and businesses just to make a point to a few ISPs. Great plan.

      1. Charles 9

        Re: The problem?

        And note some of those blacked out could be entire regions or countries.

        1. Roland6 Silver badge
          Pint

          Re: The problem?

          >And note some of those blacked out could be entire regions or countries.

          Like the USA? Go for it?

      2. Anonymous Coward
        Anonymous Coward

        Re: The problem?

        "Yeah, just fuck over millions of customers and businesses just to make a point to a few ISPs. Great plan."

        I can only assume the 10 thumbs down I got for this post mean there really are 10 people too stupid to spot blindingly obvious sarcasm, or there are 10 people who really are such bastards that they think entire communities should suffer just to make a point to some DNS admins at an ISP. Either way, you're morons.

        1. LaeMing

          Re: The problem?

          Could they introduce lag to processes using the old cert? Start with 1 minute and increase the lag by a minute each day?

          1. Charles 9

            Re: The problem?

            "Could they introduce lag to processes using the old cert? Start with 1 minute and increase the lag by a minute each day?"

            No, because most give up and time out before then. You'd be doing a wholesale block the hard way.

    4. Aodhhan

      Re: The problem?

      If you come to realize the number of corporations who contract out things such as payroll with SaaS software, you then can understand a huge problem.

      You know, that you may not get paid.

      Still want them to go ahead?

  5. john.jones.name

    Root KSK roll

    actually to quote Matt Larson :

    " Historically there has been no way to determine which trust anchors DNSSEC validators have configured, making it difficult to assess the potential impact of the root KSK rollover. "Signaling Trust Anchor Knowledge in DNS Security Extensions (DNSSEC)" (defined in RFC 8145) is a recent protocol extension that allows a validator to report which trust anchors it has configured for a zone to that zone's name servers. The protocol was only finalized in April, 2017, and only the most recent versions of BIND (9.10.5b1 and 9.11.0b3 and later) and Unbound (1.6.4 and later) support it. This protocol was not expected to have sufficient deployment to provide useful information for the first root KSK rollover."

    updating the DNS infrastructure has a side effect that finally DNSSEC with ECC specifically Ed25519

    ed25519 is a public-key signature system invented by Bernstein et al. that is standardized for use in internet protocols as RFC 8032. In RFC 8080, ed25519 (and ed448) were standardized for use in DNSSEC in February 2017.

    DNS software supporting ed25519

    An incomplete list of DNS tools and servers that support or will support ed25519:

    BIND 9 (unreleased) has support based on the upcoming OpenSSL 1.1.1

    Knot DNS (unreleased) will support ed25519 in 2.6 using GnuTLS 3.6.0

    Unbound (since 1.6.4) supports ed25519 via the unreleased OpenSSL 1.1.1 and since version 1.6.6 via libnettle

    PowerDNS Authoritative Server (since 4.0.0) has support for signing with ed25519 using libsodium

    PowerDNS Recursor (since 4.0.5) has support for validating ed25519 using libsodium

    ed25519 signature is 64 bytes long, compared to 256 bytes for an RSA 2048 signature. These smaller signatures ensure that DNS amplification attacks are less severe than before, without sacrificing the security of DNSSEC.

    1. Bronek Kozicki

      Re: Root KSK roll

      I do not quite understand. It does not appear that the new KSK is somehow linked to the new signing algorithm ed25519 , or at least the article is silent on this. From what I read, it is a new DNSSEC signing key for the root servers, so as long as your (new or not so-new) DNS server has the new DNSSEC keys installed, it should just work. The RFC 8145 seem to be about enabling the root (and not so root) DNS servers to build up a knowledge about who has the new key and who has not, and it does not need to be installed everywhere. Neither seem related to the new key signing algo.

      Unless the requirement of the new key is the use of the new signing algo, which would be a pretty important point to make in the article (and yet it is no there)?

    2. Bruce 11

      Re: Root KSK roll

      Umm BIND is currently releasing 9.11.2, 9.10.6, 9.9.11

      https://www.isc.org/downloads/bind/

      dnssec looks like it was added about 9.9 - see matrix

      https://kb.isc.org/article/AA-01310/109/BIND9-Significant-Features-Matrix.html

      Additional info - addition of ED25519 alg is in proto 9.12 - in beta release:

      https://www.isc.org/blogs/bind-9-12-0-alpha/

  6. Anonymous South African Coward Bronze badge

    It wibbles, it wobbles... and it is the Wibbly Wobbly Web!!!

  7. Wolfclaw

    Just do it, the knob heads that haven't implemented the upgrade will soon face the wrath of customers going offline, unless your a VM customer and probably couldn't tell the difference.

  8. Prst. V.Jeltz Silver badge

    So is this something that only ISPs have to know about - as they are the ones running DNS servers mainly?

    Could an affected end user with a crappy ISP just tell his computer to use google's 8.8.8.8?

    It would be fun to email my ISP for confirmation thy are aware of this upcoming change.

    Are there more layers of DNS ( like managers) . some group of "master servers"?

    EE might say to me - Yeah we're ready but unable to get comfirmation from our DNS chain wholsaler that they are prepared.

    1. Charles 9

      "Are there more layers of DNS ( like managers) . some group of "master servers"?"

      Yes. DNS is hierarchical in nature with varying levels. The "master servers" are the 13 root servers at the top of the hierarchy. It's THEIR keys that are being changed, and each layer down needs to copy or everything below them can go dark.

  9. patrickstar

    How about rolling over, or rather rolling back, to not using DNSSEC at all? Across the global Internet, it adds exactly zero security for all practical purposes.

    1. Charles 9

      Oh? What about spoofing? Redirection attacks?

      1. patrickstar

        All of those, and more, are solved at other levels of the protocol stack (TLS, SSH, etc). No need to get DNS involved. Or benefit from doing so.

        If you haven't protected the actual data stream, you need to do so regardless since there are many ways to direct it into the arms of an attacker apart from the few venues DNSSEC protects against. And once you have done so, you no longer need DNSSEC with its downsides.

        1. Charles 9

          What significant downsides are there apart from some smaller organizations needing to update their tools to more modern ones (which can have the benefit of less drudgery)?

          1. patrickstar

            DNS just about barely works as it is. The average quality of zones and delegations is simply appalling. Adding a PKI on top of it certainly won't help it work better. And considering adding said PKI has no benefits whatsoever *, why even consider doing it, much less do it on a global scale?

            * At most it would stop something along the lines of Web sites getting defaced because of DNS hijacks, like after compromising hosts in the same subnets as the DNS servers. Not exactly an issue worth remodeling fundamental parts of the Internet for... If you can have much more fun than that, the issue is with the actual service (read: cleartext authentication, no HTTPS, etc), not with the DNS infrastructure.

            PS. The proper "upgrading of tools for DNSSEC" in a small organization consists of making sure it's disabled everywhere.

  10. Matt34
    Facepalm

    What a load of cow pats

    Having just completed a training course which covered DNSSEC, the Root Key Signing Keys are using algorithm 8 which RSA/SHA256 and is compatible with Bind 9.10.3 which is available for MintLinux and Ubuntu Mate for Raspberry Pi. The new KSK works just fine. After five years of giving training on DNSSEC, 99% of students who were familiar with DNS had never heard of DNSSEC (despite it being 16 years old).

    For those unfamilar with DNSSEC at present there is a double Key Rollover, both the KSK and Zone Signing Key (ZSK) are rolling over to a new key.

    dig @8.8.8.8 . dnskey +multiline

    ; <<>> DiG 9.10.6 <<>> @8.8.8.8 . dnskey +multiline

    ; (1 server found)

    ;; global options: +cmd

    ;; Got answer:

    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5983

    ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

    ;; OPT PSEUDOSECTION:

    ; EDNS: version: 0, flags:; udp: 512

    ;; QUESTION SECTION:

    ;. IN DNSKEY

    ;; ANSWER SECTION:

    . 166290 IN DNSKEY 257 3 8 (

    AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQ

    bSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh

    /RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWA

    JQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXp

    oY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3

    LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGO

    Yl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGc

    LmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0=

    ) ; KSK; alg = RSASHA256 ; key id = 19036

    . 166290 IN DNSKEY 256 3 8 (

    AwEAAYvxrQOOujKdZz+37P+oL4l7e35/0diH/mZITGjl

    p4f81ZGQK42HNxSfkiSahinPR3t0YQhjC393NX4TorSi

    TJy76TBWddNOkC/IaGqcb4erU+nQ75k2Lf0oIpA7qTCk

    3UkzYBqhKDHHAr2UditE7uFLDcoX4nBLCoaH5FtfxhUq

    yTlRu0RBXAEuKO+rORTFP0XgA5vlzVmXtwCkb9G8GknH

    uO1jVAwu3syPRVHErIbaXs1+jahvWWL+Do4wd+lA+TL3

    +pUk+zKTD2ncq7ZbJBZddo9T7PZjvntWJUzIHIMWZRFA

    jpi+V7pgh0o1KYXZgDUbiA1s9oLAL1KLSdmoIYM=

    ) ; ZSK; alg = RSASHA256 ; key id = 15768

    . 166290 IN DNSKEY 256 3 8 (

    AwEAAcRIZfxskdElMKgjwvWQO2bQe7EGAvX6zgIaqmbs

    aMqmMrIpd1+bP7nyULLuL8jWnKAqcaVfal2yJD50gg5z

    Fl5yW/F9dKNXXEFI7VEcGrPyG6/OrA9RBU8pGWm0qxps

    Nm5UIgTU5IX7pb/0rBj67c/R7qln8sjH1ylsr4f1Y3R6

    p/druiEalKasEjGKA9L2w9jzUQusWxM7fQx/T8c/3x3b

    sjveD1dleQ6MJaCx4bpPXYZpqXmSvGn+T2v5350cBVAF

    qVKhGbjxEyXAweem8cTU4L1p+DV7Ua11a1tMf0Tlu8pk

    pLwh7NQIggIEhJwEhPeXE3E4C6Q2/PFENcoFERc=

    ) ; ZSK; alg = RSASHA256 ; key id = 46809

    . 166290 IN DNSKEY 257 3 8 (

    AwEAAaz/tAm8yTn4Mfeh5eyI96WSVexTBAvkMgJzkKTO

    iW1vkIbzxeF3+/4RgWOq7HrxRixHlFlExOLAJr5emLvN

    7SWXgnLh4+B5xQlNVz8Og8kvArMtNROxVQuCaSnIDdD5

    LKyWbRd2n9WGe2R8PzgCmr3EgVLrjyBxWezF0jLHwVN8

    efS3rCj/EWgvIWgb9tarpVUDK/b58Da+sqqls3eNbuv7

    pr+eoZG+SrDK6nWeL3c6H5Apxz7LjVc1uTIdsIXxuOLY

    A4/ilBmSVIzuDWfdRUfhHdY6+cn8HFRm+2hM8AnXGXws

    9555KrUB5qihylGa8subX2Nn6UwNR1AkUTV74bU=

    ) ; KSK; alg = RSASHA256 ; key id = 20326

    ;; Query time: 9 msec

    ;; SERVER: 8.8.8.8#53(8.8.8.8)

    ;; WHEN: Fri Sep 29 19:51:13 BST 2017

    ;; MSG SIZE rcvd: 1128

    DNSSEC is used to digitally sign DNS records by adding a RRSIG record which is cryptographically signed hash of a set of records within a Zone. So the NS records inside a Zone would have a single RRSIG to allow authentication and validation of all the NS records as if you query the NS records all will be returned.

    In terms of handling the new KSK, if the DNS Server supports RFC5011 the KSK rollover is handled automatically BIND has support this for a couple of years. RFC7344 allows the automatic update of the Child Zone Delegated Signer Record using the CDS Record and CDNSKEY record both supported in Bind 9.11

    Unfortunately many commercial DNS providers do not support RFC5011, nor do they support RFC7344.

    The Root KSK should have been replaced in June 2015 under the contract ICANN operated under here we are seven years later and the its going to be delayed again.

    Personally the root key change should not be delayed... A competent DNS Architect/Administrator should have planned for this two years ago.

    1. Charles 9

      Re: What a load of cow pats

      You incorrectly assume all such positions are filled by competent personnel. Consider nepotism, tight budgets, and basic barrel-scraping.

  11. Anonymous Coward
    Anonymous Coward

    Am I missing something? I believe I was promised “naming and shaming” and yet I can find a single name...

    Tease!

    1. Anonymous Coward
      Anonymous Coward

      sparticus

  12. DougMac

    Just look at IPv6?

    > Just look at IPv6

    I'm looking at IPv6. Mobile really made it a slam dunk use-case.

    56-60% of all my email users come in over IPv6.

    I'm not a large web content provider, so I can't show the same stats there, but I'd bet that Facebook is showing numbers equally impressive.

    Look at the ISPs or companies like Facebook that are 100% IPv6 internal with only IPv4 gateways now.

    Look at the IOC 2017 IPv6 report for more evidence of ISPs considering dropping IPv4 native in the next *handful* of years.

    The one case where everybody is dragging their feet?

    Enterprise.

    Enterprise fears IPv6, buried their heads in the sand, even though they probably have significant IPv6 traffic internally traversing their network. They need to figure out that those OSs running internally are all doing IPv6 native now, and learn how to properly secure it (a single external breach could setup a IPv6 RA and proxy, and funnel all the Enterprise traffic out beyond the firewall in a heartbeat) and embrace it. IPv4 is going away, Enterprise needs to learn that.

    1. Charles 9

      Re: Just look at IPv6?

      Well, enterprise happens to be most likely place to find IPv4-ONLY equipment: acquired before IPv6 was a thing yet too indispensable and/or too expensive to replace. You can't roll out IPv6 selectively because the old stuff will lose touch with complicated bodges, becoming a case of "If it ain't broke..." best not to rock the boat internally and address external IPV6 needs via specialized structures: dedicated subnets, gateways, proxies, etc.

  13. mark l 2 Silver badge

    Some ISP won't let you use DNS servers other than their own, I connected to the BT WIFI at an Airbnb property i was renting. My laptop is set to use Google DNS and got a message that I had to use their DNS when i tried to get online.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like