back to article University DDoS'd by its own seafood-curious malware-infected vending machines

A US university saw its network traffic slow to a crawl thanks to an IoT malware infection that hit, among other things, its vending machines. The unnamed university had its story told by Verizon Enterprise in a sneak preview [PDF] of its 2017 Data Breach Digest report. The story, as told by an also unnamed senior IT staffer …

  1. Anonymous Coward
    Anonymous Coward

    Lesson Learned

    There's a valuable lesson for them to have learned: don't put crap on your net. We have wifi vending machines here at work, they take credit cards and NFC, and I have no idea what net they are on, but I know my security people would not allow any device on the network without checking it out thoroughly. But then, we have a dedicated security team and run online and normal retail ops, so there's no way an infected vending machine is going to appear on my work production networks. At that mysterious university? Yes, there is a way. The staff are not equipped to do anything or say anything about those fancy new vending machines until AFTER they cause a problem. 100% this is "hey, just install the ice cream machines and don't worry about the wifi security, kid!" One. Hundred. Percent.

    1. Flocke Kroes Silver badge

      Re: Lesson Learned

      Don't use plain text passwords to control your botnet.

    2. Stevie Silver badge

      Re: Lesson Learned

      All your Mars Bar are belong to lightbulb.

  2. Anonymous Coward
    Anonymous Coward

    1. Put the IoT (Internet of Trash) behind a firewall.

    2. Change the frickin default passwords.

    3. Put the IoT (Internet of Trash) on a VLAN separate from everything else. If you need to make several IoT VLANS, do so.

    4. Write a script that scans all of your VLANs looking for IoT devices that allow default credentials. If you find one, take action to hunt the vermin down.

    http://www.csoonline.com/article/3126924/security/here-are-the-61-passwords-that-powered-the-mirai-iot-botnet.html

    1. Mage Silver badge

      Also

      Disable uPNP on firewalls / routers.

      The article is a catalog of incompetence.

      1. hmv Bronze badge

        Re: Also

        uPNP is unlikely to be enabled on a University firewall - we don't run SoHo gear.

        As to incompetence, well perhaps, but universities are weird places where IoT devices spontaneously appear on the network without the (in)security team having a say. This is not necessarily a bad thing in itself (academics are supposed to play with weird shit - it's in their job description).

        What is also possible is that the IoT vendor agreed to comply with security best practices, and then ignored that contractual requirement whilst deploying the gear.

        1. Anonymous Coward
          Anonymous Coward

          Re: Also

          Our university (hence the AC) is planning a network upgrade and one of the great new ideas is to do away with subnets-per-department so all machines can talk to all others.

          Sure there are plans for trusted/untrusted IP ranges but nothing so far that indicates this sort of thing has been taken in to account. Apparently your single sign-on for windows will be enough to make sure you get the appropriate access...

          1. Adam JC

            Re: Also

            Crikey, won't you run into IP allocation exhaustion issues with regards to IPv4 pretty quick?

            And aren't your core-routers able to do inter-VLAN routing with ACL's? Surely that's a better way of going about it.

            1. Bandikoto

              Re: Also

              16.8 million hosts aren't enough for even a large land-grand college? Sure, the public hosts will likely be on a Class B network, but everything else (e.g. IoT, phones, student laptops) will be on a private network and those are invariably on net 10.

          2. bombastic bob Silver badge
            Devil

            Re: Also

            "Apparently your single sign-on for windows will be enough to make sure you get the appropriate access"

            I _HOPE_ they're not excluding Linux and FreeBSD (or even mac) by requiring a 'single sign-on for windows'... or maybe I'm just reading too much into this.

            Anyway, separately firewalled subnets for IoT might help. Or not.

            1. Anonymous Coward
              Anonymous Coward

              Re: Also

              I don't think they will exclude them, many in the arts department use Macs, as do a few of the IT folk themselves, and many of the sciences use Linux boxes and the occasional (very occasional now) Solaris box.

              I just think they have not thought it through. Or are thinking of using the underpants+profit approach to design...

  3. Anonymous Coward
    Anonymous Coward

    Internet of Tat

    That is all

    1. Stevie Silver badge

      Re: Internet of Tat

      Your "w" key isn't working, AC.

  4. cd

    I was just reading today about an IOT water heater. "Only" $1000 for a 50 gallon. Which would send me updates about how it was doing. And potentially, according to the description, save me $4444 over its projected 10 year lifetime. Be curious how they arrived at that one, other than just holding the 4 key down, then Return, and going to lunch.

    What would Pirx do if his water heater texted that it was in jeopardy?

    1. DNTP

      water heater in jeopardy

      Probably find out that the IoT company that made the water heater had gone out of business five years ago, or that their "warranty" only covered hardware malfunctions and they had no interest in supporting a software or security problem.

  5. Youngone Silver badge

    Reads like marketing nonsense

    The linked PDF reads like an Ad for some Verizon service or other.

    I'm going to choose to be skeptical about whether the described events even happened.

  6. razorfishsl

    Seriously WTF was this not on a separate VLAN with Firewalls.

    Typical China like mentality, just increase the subnet mask and stick it all on the same network with the same DHCP.

    As one of my staff asked today,

    "I've been reading up on some internet tutorials, can you give me a config dump of the CISCO core switch & FW, I want to improve it"

    .......

    "NO"

  7. Doctor Syntax Silver badge

    As far as I can see this network was specifically set up for the purpose, presumably by the University's own network team. Did it never occur to them to change the passwords?

  8. Denarius Silver badge
    Unhappy

    Ritchie and Kernighan again

    Reflections on Trusting Trust. ITIRC ASCM decades ago. Trust nothing, trust no-one until at least basic verifications done. Techno-utopians could benefit from understanding the doctrine of the total depravity of humans from the Reformation. It at least gives a sensible starting point for risk assessment such as, "how would a baddie abuse this kit/process/document?"

    1. jake Silver badge

      Re: Ritchie and Kernighan again

      ACM, not ASCM. Written by Thompson. No Kernighan or Richie.

      http://dl.acm.org/citation.cfm?id=358210

      I'm not certain it supports your thesis. Perhaps elaborate?

      1. Steve the Cynic

        Re: Ritchie and Kernighan again

        The point, I think, is about how you can know that a particular thing is trustable, and the conclusion is that unless you can trust the person (or people) who created it, you can't trust the thing.

  9. Anonymous Coward
    Anonymous Coward

    I can't help wonder...

    If this is the same kind of university which would also easily hook up a students network with the internal administrative network and the teachers network, only to end up surprised that students managed to gain access to their study results and more...

  10. John Smith 19 Gold badge
    Unhappy

    I'm guessing the slightly tough part.

    starts once you've written a script to scan for the Mirai botnet list of default passwords.

    You find 1000 devices all with the same password because this is the IoS we're talking about.

    Do you a) change them all to new standard password (get one device, get them all) or b) Create and give them all unique passwords and keep them in an encrypted field in a database.

    a) Is cheap and probably quite simple, but once it's compromised you're back where you started. b)Is more work to start with but you can leverage the result for all you future IoS devices.

    Maybe I'm paranoid but a layered defense seems a pretty good idea to me. As others have said, firewalls and subnets are good. Limit the access. Find a way so once a device starts spewing packets at some ridiculous rate for no obvious reason (other than it's been infected) it chokes something and there's an obvious marker to investigate.

    1. Doctor Syntax Silver badge

      Re: I'm guessing the slightly tough part.

      "Do you a) change them all to new standard password (get one device, get them all) or b) Create and give them all unique passwords and keep them in an encrypted field in a database."

      Because you want to act quickly, go for a) on the first pass. This gives you breathing space to implement b) when you've worked out a suitable strategy.

      As this installation seems to have been intended to apply down to the level of every light-bulb in the place for easy of maintenance (yup, sure made life easy!) there might need to be a lot of people who needed access to the list so implementing b) might not be straightforward. It might even include a review of whether all the devices needed to be "smart".

      1. Jess--

        Re: I'm guessing the slightly tough part.

        Trouble is on a lot of these things you can't change the passwords.

        for example on a cctv dvr manufactured by dahua (and heavily resold under many brands) you can change all of the passwords apart from 2, the 1st is a remote view login (33333333 / 33333333) the 2nd is a root login (88888888 / 88888888)

        the box itself is a cracking bit of kit but you can't deploy it because there is no way of making it secure.

        1. Stoneshop Silver badge
          Mushroom

          Re: I'm guessing the slightly tough part.

          the box itself is a cracking bit of kit but you can't deploy it

          ... on a public network

          because there is no way of making it secure.

          Pickaxe, blowtorch, C4.

        2. RSteer

          Re: I'm guessing the slightly tough part.

          I wonder if it's possible to put such devices behind a bastion or gateway that requires a strong login, making the individual device weaknesses not easily accessible?

      2. John Smith 19 Gold badge
        Unhappy

        "It might even include a review of whether all the devices needed to be "smart".

        Possibly the most fundamental question of them all.

        And probably the least frequently asked until some poor fool has to support this s**t

        The IoT is coming like a slow mo car crash. I could not believe that circuit breaker modules already have internal microcontrollers in them. WTF they do I have no idea. They certainly aren't net connected and they are robust against being zapped by high currents and voltages (by the standards of digital equipment). Maybe other systems could learn from them?

        Hard coded accounts is a lazy, incompetent way to cope with dumb users. I do hope it won't take an actual law to stop this stupidity being included.

  11. Michael H.F. Wilkinson Silver badge

    Seafood curious?

    The IoT devices caught the crabs?

    1. Stoneshop Silver badge
      Coat

      Re: Seafood curious?

      They misinterpreted "squid proxy"?

      1. Korev Silver badge

        Re: Seafood curious?

        They misinterpreted "squid proxy"?

        These puns are awful, put a SOCKS in it...

  12. Hans Neeson-Bumpsadese Silver badge

    requesting seafood-related subdomains

    Obviously some sort of phishing attack then

    1. Stoneshop Silver badge

      phishing attack

      In this case, wireshark will probably have caught it.

  13. zaax

    And there is the problem. Vendling machine owners do not care about your network.

    1. Paul Crawford Silver badge
      Trollface

      Set your IoT networking rules to only allow access to the vending machine companies network addresses then.

      1. Anonymous Coward
        Anonymous Coward

        Erm yes, Dave Lister's boss Rimmer has a static IP and the knowledge to tell you what it is...

        Most vending M/C repair vans/companies I've seen are Mum (+ Pop) type companies, probably running out of a lockup/from home with BT Internet as the ISP (because their kids got fed up with the "who is best calls" and then arguing with the anything but BT answer)

    2. Infernoz Bronze badge
      Meh

      Simples, just block DHCP requests from unknown devices, or otherwise restrict network access of unknown devices.

      A 1st layer of controls could use a MAC address white-list (devices registered by authorised people) and/or MAC address to IP address mapping, extra control layers could include on-line auth. e.g. encrypted user auth.

      I use MAC address to IP address mapping at home, to simplify device use and as a 1st security layer, I can then limit access via the IP address and encrypted user auth.

  14. EnviableOne Bronze badge

    separate vlan, separate interface on firewall, no access to the rest of the network, Bandwidth limited on the infrastructure and v short ACL

    iotVlan_Access_In

    permit IotVlan VendorNetwork

    outside_Access_in

    permit VendorNetwork IotVlan

    there IoS their Problem

    1. Afernie

      Yes, but...

      It would not surprise me in any way if the Vending Machines lacked VLAN support.

      1. EnviableOne Bronze badge

        Re: Yes, but...

        The machine doesn't have to as long as the switch does.

        the vending machine is plugged to the switch the port assigned to the VLAN, the network virtually isolated from the rest. 802.1q its a great standard

        1. Afernie

          Re: Yes, but...

          Yes, 802.1q is a great standard, and no you don't need tagging support. I've nonetheless seen some very interesting behaviour with kit lacking it. Weird shit with bridging firewalls, for example.

  15. Anonymous Coward
    Anonymous Coward

    Internet scanned devices default passwords

    Shadowserver reports supplied by your friendly hosting network should be supplied and investigated.

  16. Primus Secundus Tertius Silver badge

    My new car

    My new car is an IoT on wheels. I am moderately confident, but no more than that, in its integrity as it emerged from the factory.

    But there is a USB socket by the gear stick. One clueless service mechanic wanting music with their tea-break....

  17. Anonymous Coward
    Anonymous Coward

    Coke machine...

    The first IoT machine ever was a soda dispenser that could read its thermostat, so the students knew when the Coke truck had refilled it and when its contents were cold.

    There is irony somewhere in there.

  18. Stevie Silver badge

    Bah!

    Makes you wonder how they sold machine-vended chocolate bars before the IoT doesn't it?

  19. zen1

    If that happened where I work, I would have the individual, who authorized the decision to stick that crap on my production LAN and NOT behind a fw, hunted down and lectured. And any monies spent mitigating that mess would come out of their budget. That's inexcusable, even if it was a university.

    Secondly, I would be having a nice long talk with the vendor about them reimbursing my company for the expense, as well as what their plans were to prevent events like these from happening again OR a when I could expect them to get their shit out of my buildings.

  20. David Roberts Silver badge
    IT Angle

    Money talks

    Vendors sponsor education establishments to use their vending machines.

    The vendor turns up and plugs the machine into power and the nearest convenient LAN socket. IT involvement?

  21. W4YBO

    Lamp post?

    "Short of replacing every soda machine and lamp post, I was at a loss..."

    Why would a lamp post have a network connection? What practical value could that have?

    1. Andytug

      Re: Lamp post?

      The "saving" as always seen by the PHB and their bean counters, is manpower.

      No need to send a person out to check any readings, usage, how many choc bars are left or whether the bulb is gone, it'll tell you over the internet.

      See also smart meters and a heck of a lot of IoT tat.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019