back to article Hardwired crypto certificate FAIL bricks Juniper router kit

Sysadmins with older Juniper Networks kit have been left scrambling to keep their networks running after a security certificate expiration bricked their boxen. The issue has been keeping mailing lists like AusNOG and J-NSP busy as users tried to work out whether it was a deliberate strategy to force people off the EOL gear – …

COMMENTS

This topic is closed for new posts.
  1. Anonymous Coward
    Anonymous Coward

    And if the equipment isn't under a support contract, I'm sure Juniper will be more than happy to sell a software upgrade for it.

    Nothing should ever be hard-coded especially a certificate. At least if the cert was replaceable, the companies running this could either use a self-signed, an in-house PKI infrastructure or place their own CA issued cert on the equipment. Juniper should be offering free software updates on this kit mainly because this problem was a decision Juniper made in regards to the hard-coded certificates.

    1. Ole Juul

      It is only from the customer's point of view that this was a mistake. Juniper, no doubt thinks that all's good.

      1. Anonymous Coward
        Anonymous Coward

        The customer could start to look at other vendors too, sometimes all a customer needs is a good reason to look. I know a customer that decided to switch to Juniper switches and within a year, they replaced them. After two power outages and with some of the switches came back without a config each time, they sold the Juniper switches to their previous equipment supplier in exchange for buying new switches. Sure one could say a UPS should have been on the closet switches, it still wouldn't have fixed the underlying issue.

        1. Anonymous Coward
          Anonymous Coward

          "Sure one could say a UPS should have been on the closet switches, it still wouldn't have fixed the underlying issue."

          Saving the running config to startup config would have though ...

          1. Anonymous Coward
            Anonymous Coward

            They were saved, it was a hardware fault in the flash that Juniper used. Juniper was replacing switches and the next time there was a power event, more switches came up with no config. Switches that survived the first power event and thus obviously had a saved config.

            These switches also had auto-commit mode enabled. So as soon as you make a change, the config is saved.

  2. ecofeco Silver badge

    FAIL everywhere

    Is it just me or does it seem as if the entire computer industry cannot get its shit together?

    1. Anonymous Coward
      Anonymous Coward

      Re: FAIL everywhere

      This is not unique to the IT industry.

      1. Brewster's Angle Grinder Silver badge

        Re: FAIL everywhere

        "This is not unique to the IT industry."

        Filed under "When accountants rule the world and profit is the only metric that matters..."

        1. Grease Monkey Silver badge

          Re: FAIL everywhere

          "This is not unique to the IT industry."

          Too right. For example, GM failed to build an ignition lock barrel that can cope with the weight of a bunch of keys safely. It's not an IT industry problem. Nor is it down to the bean counters. It's simply a lack of forethought. And that is just human nature.

      2. ecofeco Silver badge

        Re: FAIL everywhere

        You're right, it isn't unique to IT.

        But IT now controls in some way or another, every industry.

    2. Anonymous Coward
      Anonymous Coward

      Re: FAIL everywhere

      "as if the entire modern computer industry cannot get its shit together?"

      Things were different back in the days when computer kit was designed by engineers for engineers.

  3. Anonymous Coward
    Mushroom

    This is rather mean.

    I mean at least Microsoft gave everyone years of advance notice with respect to Windows XP's impending End-of-Live. Even then, though, it's technically still possible to use the Operating System.

    If I were one of Juniper's customers and all of my kit suddenly died with zero advance notice what-so-ever I'd be rather miffed. Especially so if the fail-over routers were also old and thus also affected.

    Routers especially tend to last rather long and for most part if you're utilizing nothing beyond core routing functionality it's not uncommon to be able to get away with not patching the router for years. I've seen tons of Cisco routers in the wild running on ancient versions of IOS. But hey, if it works, why not? Imagine if every single Cisco router in excess of five years old suddenly "deactivated" overnight... half the internet would probably collapse.

    1. Anonymous Coward
      Anonymous Coward

      Re: This is rather mean.

      "I've seen tons of Cisco routers in the wild running on ancient versions of IOS. But hey, if it works, why not?" I dunno, maybe the massive number of security holes that would be accessible to any man and his dog?

      " Imagine if every single Cisco router in excess of five years old suddenly "deactivated" overnight... half the internet would probably collapse."

      If every single Cisco router on the internet wasn't patched/updated like you suggest, half the internet would have already collapsed.

      1. Grease Monkey Silver badge

        Re: This is rather mean.

        The thing is that routers like this should be maintained by proper technical bods and proper technical bods know how important it is to keep things patched up to date. The problem is that managers don't.

        Management don't like the expense of paying for software upgrades and even when they are available free they don't like the expense of paying people to perform the upgrades. They don't like the idea of downtime in working hours, but they don't like the idea of paying for overtime to do the work OOH.

        Then of course there are those managers who decided there was no need to pay for that expensive maintenance on all the network hardware. I knew one manager who decided he didn't need maintenance on a large estate of Cisco kit. He decided it would be much cheaper to get hardware replacement cover only. Of course he then started complaining when he needed an IOS upgrade for a 6500 and he no longer had access to the downloads he used to have. He wanted to know if the hardware support company could get him the IOS image. He started trying to blame his tech people when the supplier pointed out exactly what "hardware replacement only" means and how software cover would cost a lot more.

        Managers are the biggest reason why there are so many routers, firewalls, switches and other network hardware out there running horribly outdated software.

  4. Joe Montana

    Licence enforcement code

    Another example of where licence enforcement code causes a denial of service to paying customers... All of this licence enforcement crap is basically companies distrusting and screwing their own customers, these functions provide no benefit whatsoever to the actual customers and they don't harm the pirates who will simply apply a crack to remove them.

    The fact that companies will go to significant extra effort to implement functions purely for their own benefit and to the detriment of their paying customers is ridiculous. If only they spent that time fixing bugs instead.

  5. wyatt

    The company I work for resell equipment/software which the manufacturer decided to start using certificates with. They failed to put a reasonable expiry date on said certificate and left us to upgrade the affected customer sites at our expense.

    They've also just released a version of software which was certified against specific telephony versions, we installed on a customers site to find that without some hot fixes it failed to work. Something they knew about the service pack previously and hadn't fixed or told anyone about (they knew we were installing it and didn't tell us, we didn't ask as it was a specific release for this purpose).

  6. Eku

    Computer Misuse Act

    As I recall when the computer misuse act was introduced in 1990, there was a section on 'time bomb' software being an offence. That is to say to put a time bomb that disables the software was an offence. Haven't kept upto date, but my view would be that juniper would have to fix or replace the devices at their own expense.

    1. Nigel 11

      Re: Computer Misuse Act

      Was about to post the same thought, that this is illegal.

      Moreover, is this purely a matter of civil law? IANAL but I hope that Juniper executives take legal advice about the possibility that they are guilty of a (currently ongoing) criminal offense?

      1. Grease Monkey Silver badge

        Re: Computer Misuse Act

        That would surely depend on whether evidence of intent is required for that offence to be proven.

  7. DropBear
    FAIL

    Appliance level software...

    ...that one just turns on and expects to just work should not be dependent on any external checks. There should be no "if" - power up and work, dammit: end of story. Admittedly, no other software in general should do this sort of thing either, but Photoshop is one thing, a router is quite another...

  8. This post has been deleted by its author

    1. Version 1.0 Silver badge

      Re: at best...

      Check your Terms and conditions, it's very likely that there's some wording in there that allows Juniper to walk away from this ...

      It's the way business works these days, you are a customer. you are to be plucked, diced and quartered and boiled up with some vegetables to make a nice soup stock. So get over your self importance and jump in the pot.

      1. Mark 65

        Re: at best...

        In the UK it is impossible to sign away statutory rights. T&C are thus irrelevant if an offense has been committed.

        1. Grease Monkey Silver badge

          Re: at best...

          Ah, but the law also requires that the owner takes reasonable steps to maintain the goods in working order. If you bought a car and drove it without servicing until the cambelt snapped you would not have a claim against the manufacturer or retailer. I suspect that Juniper can sidestep this by saying that reasonable maintenance means keeping the software patched up to date.

        2. Anonymous Coward
          Anonymous Coward

          Re: at best...

          "In the UK it is impossible to sign away statutory rights"

          Applies, afaik, only to "business to consumer" transactions ("consumer" = mass market, not a professional).

          Junipers wouldn't usually be classed as stuff a 'consumer' would buy.

          IANAL.

  9. Christian Berger

    Systems enforcing licenses...

    ...should not be considered highly available. It's a completely unnecessary part that can fail, and that can even be a security issue.

  10. tin 2

    Licencing: causing more outages than actual faults since 1990

  11. Crazy Operations Guy

    Why do the licenses for routers even need to exist?

    How much does it take to produce the software on it? I don't think it could possibly bankrupt any of the manufacturers to just give it away, especially when they charge anywhere between $5000 for a basic router all the way up $2 million for the high-end stuff. Hell they might end up saving money by cutting down on the versions they have to maintain, shutting down the authentication servers and laying-off all the account managers responsible for the software support accounts.

    HP's networking division seems to be doing well enough despite giving away the software.

  12. Anonymous Coward
    Anonymous Coward

    FUD Man

    If you read the AusNOG post and the actual Juniper KB article it is clear that the licensed features will stop working and not the entire device. Admittedly this can be an issue if one is relying on these licensed features.

    Saying the device will brick is a bit of a stretch.

    The AC talking about Juniper EX switches must have had some wonky gear as Junos does not auto-commit anything as far as configuration changes go.

    Early EX were prone to flash corruption due to unclean shutdown but there was a way to workaround this. Juniper introduced redundant flash images in Junos 11.x and I haven't seen or heard of any EX running 11.x and beyond having issues like this.

This topic is closed for new posts.

Other stories you might like