back to article Google rushes in where Akamai fears to tread, shields Krebs after world's-worst DDoS

Google has provided free distributed denial of service attack (DDoS) mitigation services to security publication Krebs on Security, stepping in after Akamai withdrew support. The information security site was last week hammered with a 620Gbps DDoS attack, widely rated one of the world's largest by volume of junk data. …

  1. Brian Miller

    Big pipes are the only protection

    The only way to really "protect" against attacks like this is to have the bandwidth to eat the attack traffic. Otherwise, you'd have to be somebody like Netflix, and I don't think they do pro bono stuff like this. Nice to see Google take up the challenge.

    1. Alan Brown Silver badge

      Re: Big pipes are the only protection

      "Nice to see Google take up the challenge."

      Google are connected enough to be able to work out who's behind the attacks.

      This is the only long-term solution.

      The (totally non-ironic, honestly) part of this story is that the vast majority of the traffic is coming out of IoT "security" devices such as cameras. I'm surprised that El Reg didn't pick up on this part.

      1. VinceH

        Re: Big pipes are the only protection

        "The (totally non-ironic, honestly) part of this story is that the vast majority of the traffic is coming out of IoT "security" devices such as cameras. I'm surprised that El Reg didn't pick up on this part."

        Yes. Brian's post on the subject was quite long and covered a lot of related issues, but it would have been worth mentioning that - just a single sentagraph since this is a Darren Pauli post. Another thing worth mentioning would have been Akami's estimated cost of continuing to defend against the attack, and the estimated value of the free protection Akami were providing:

        "In an interview with The Boston Globe, Akamai executives said the attack — if sustained — likely would have cost the company millions of dollars. In the hours and days following my site going offline, I spoke with multiple DDoS mitigation firms. One offered to host KrebsOnSecurity for two weeks at no charge, but after that they said the same kind of protection I had under Akamai would cost between $150,000 and $200,000 per year."

      2. Martin Summers Silver badge

        Re: Big pipes are the only protection

        "I'm surprised that El Reg didn't pick up on this part."

        They did in an earlier article they posted about this.

        http://www.theregister.co.uk/2016/09/26/brian_krebs_site_ddos_was_powered_by_hacked_internet_of_things_botnet/

    2. Anonymous Coward
      Anonymous Coward

      Re: Big pipes are the only protection

      I read that as bagpipes and it still seemed like a perfectly sensible idea.

    3. NonSSL-Login

      Re: Big pipes are the only protection

      The attacks on Krebs also had valid GET HTTP requests so even if there was enough bandwidth, without enough load balancing the web servers could have exhausted their CPU's. One assumes that HTTP POST requests were also made to the form for the mailing list too.

      Interesting half the traffic was made up of GRE routing protocol traffic too. Still unsure as to why, but these guys knew what they were doing beyond the usual spoofed IP UDP floods and easily scanned and scripted amplification attacks for various services/protocols.

  2. Ole Juul

    Thanks Google

    But this is just not going to be a fix in the long run. This situation is really a heads-up on the vulnerability of the internet and we're going to need the cooperation of a lot more companies to get a grip on this. Let's hope everyone rallies.

  3. allthecoolshortnamesweretaken

    Read about it on BB (link below). Apparently insecure IoT devices (are there any other) were used to boost the attack.

    http://boingboing.net/2016/09/25/the-democratization-of-censors.html

    1. ecofeco Silver badge

      Some of us have been warning this would happen. I didn't think it would happen this soon.

  4. ecofeco Silver badge

    Here's the scary part...

    According to other news sites, the DDoS used IoT devices.

    http://www.csoonline.com/article/3123785/security/largest-ddos-attack-ever-delivered-by-botnet-of-hijacked-iot-devices.html

    A giant botnet made up of hijacked internet-connected things like cameras, lightbulbs, and thermostats has launched the largest DDoS attack ever against a top security blogger, an attack so big Akamai had to cancel his account because defending it ate up too many resources.

    1. LDS Silver badge
      Devil

      Re: Here's the scary part...

      So Google just turned off its Nest thermostats to stop the DDoS?

  5. Anonymous Coward
    Anonymous Coward

    Who gains?

    All you traffic are belong to Google.

    1. Ole Juul

      Re: Who gains?

      The gain is to whoever is profiling the infrastructure through this.

    2. Dan 55 Silver badge
      Black Helicopters

      Re: Who gains?

      All those peering agreements that Google has, have finally they flipped the switch and turned Google Edge into the Internet?

  6. Anonymous South African Coward Silver badge

    Rise of the IoT

  7. lglethal Silver badge
    Go

    The Only Way...

    The only way we are going to see the end of DDOS attacks is to start holding the companies that produce or control the unprotected services responsible.

    I hate the American way, but until the various IoT developers start getting sued for the costs of protecting against, or the downtime caused by, DDOS attacks, they aren't going to start spending the time and money to implement the sort of security which can make sure their devices cant be used this way.

    1. DJ Smiley

      Re: The Only Way...

      I'm thinking about a car analogy for this.

      The Tesla ships with autodrive, it's great, it generally works, but it might kill you (or someone else).

      Your responsibility as a driver, is to remain in control of the device you are using. Tesla's is to try and make sure you do.

      Obviously, with IoT devices that you can't configure then ISP's need to outright block this traffic. This is something that's actually discussed in the original article (or one of the many I've read about this attack since). I'm starting to think we need a dual layered internet where the vast majority of users only have access to websites (and maybe force them to use https?) and anything more than this is an extra you opt into. Of course as soon as we did this then ISP's would jump on the chance to charge you for all the extras you need, vpn? charge for that, you host a server? charge for that... etc

      I don't know what the fix is :(

  8. Anonymous Coward
    Anonymous Coward

    be very very scared

    where krebs is now, and what google are doing, is not the point.

    the crims have have found a big enough weapon to make the infrastructure of the biggest DDOS protection provider start to run hot. Akamai moved krebs on, to protect their paying customers.

    I have no insider knowledge, but its very likely that akamai engineering will have been working 24x7 since then to increase their defensive firepower. Its also likely that various suppliers of internet transit, high end routers, ddos cleaning appliances and servers, will have all just received large orders on a "rocket-ship" expedite request from them. They may already have had they extra infrastructure queued up ready to go, in which case they will have spent the weekend turning it up.

    until that next step in the arms race is taken, the crims are temporarily ahead, and they know this. So they have a window in which they can shoot at a site that does something that people really rely on and cause real pain. so you should worry, that its not your govt/stock market/bank that they choose.

  9. winterswift

    Is it not obvious that a "booter" or "stresser" service charging $5 for a decent sized DDoS attack for "stress" testing is a bot-net?

    Use DDoS professionals like NimbusDDOS who can provide the traffic, tiered delivery, emergency shutoff, pause during the attack for settings changes, et. al.

    Don't mess around to save a few pennies.

    1. DJ Smiley

      Does it matter if it's obvious?

      These guys were making money from doing the DDoS's, from a business pretending to be a testing service.

      I'd find it hard to believe anyone thought these guys were legit.

  10. AJNorth

    Since moving on from the Washington Post to form his own company, Mr. Krebs has been the victim of several — and wide-ranging — attacks (including threats to himself and his family). He is one of today's [relatively little-known and often unsung] heroes.

  11. Anonymous Coward
    Anonymous Coward

    Plastic bag charge

    As the problem is mostly consumer internet devices providing a large number of small and un/underdefended attack points it feels to me like this could be solved fairly quickly by crafting a mandatory charge for outbound bandwidth.

    A sensible free allocation of a reasonably large amount to cover video messaging would be needed, but after that charging for outbound bandwidth might tip the balance for people who are currently very poorly incentivised to secure non-critical systems.

    Forcing service providers to charge a fee for outgoing bandwidth would be easier to implement than a mandatory security framework and would focus peoples' minds on not being part of the problem.

    1. Mage Silver badge
      Facepalm

      Re: Plastic bag charge

      People do pay for their connection already.

      People often do have a traffic cap, even if hidden in "fair use".

      Why should everyone pay for idiots that don't setup their router/firewall properly for their IoT?

      We could perhaps fine makers $500 per vulnerable device.

      Broadband is already expensive enough without everyone subsidising IoT. The proposal would not help as it's the aggregated data of thousands or even millions of PCs, IoT, tablets, phones and even routers/modems etc.

      1. Anonymous Coward
        Anonymous Coward

        Re: Plastic bag charge

        I don't envisage everyone paying. I envisage everyone whose outgoing bandwidth on a consumer internet connection exceeds a sensible limit paying.

        Let's say it's 3GB per month of outbound traffic (for the sake of argument). The right number would be to discourage the current 'not my problem' attitude of the owners of these consumer devices/computers that get hacked without affecting otherwise normal behaviour.

        The fact that it's the aggregated data of thousands/millions of PCs, IoT, tablets etc. is precisely why a solution influencing the behaviour of the thousands/millions of people who own those PCs, IoT, tablets etc. is required.

        1. Dan 55 Silver badge

          Re: Plastic bag charge

          So you've fixed the problem a billing cycle or two into the future. What about today's DDoS?

          1. Anonymous Coward
            Anonymous Coward

            Re: Plastic bag charge

            Today's Ddos is a problem that is handled by commercial solutions and, when at scale on underequipped targets, pro-bono protection like Google's.

            No macro-policy solution will be quick to implement, but I believe that one that influences behaviour will work better than one which tries to define a set of technical specifications which equipment makers have to follow.

            My claim isn't that it will make a lot of people magically more technical over night, it's that when their technical inadequacy costs them money they'll learn a little about the basics and/or get technically competent advice in and/or voluntarily buy from equipment makers that do a decent job.

            The plastic bag tax doesn't work because all of the people paying understand or agree with the reasons they're paying. It works because putting a cost on something makes people value it.

        2. Esme

          Re: Plastic bag charge

          @AC - wouldn't work - the very people whose kit will have been hijacked are those who are liable to have least idea of how to stop their stuff being hijacked. It needs for manufacturers to really suffer if they sell such insecure kit, and for product recalls where sensible. Neither of which is likely to happen anytime soon, more's the pity.

      2. Milton Silver badge

        Re: Plastic bag charge

        "Why should everyone pay for idiots that don't setup their router/firewall properly for their IoT?"

        I understand the sentiment, because we're mostly technically minded here.

        But seriously, how many ordinary Joes buying anything from a consumer WiFi router to a {enter latest thingternet appliance fad here} knows or cares about the kind of security we're talking about? He is not going to spend any longer on that device than plugging it in and seeing it do something. That's human nature, right there, plus the fact that (guessing here) 75%+ of the population doesn't understand why a dotted-quad IPv4 address looks as it does. You can't blame them, any more than you can blame a modern car owner for not having a clue what "firing order" means.

        I think the solution to this has to consist of at least two major initiatives.

        One: revisit some fundamental design and functional aspects of the internet. There is a painful, expensive and absolutely necessary process of reinvention needed here, to fix some decisions which in retrospect were naive, not least to do with DNS routing, authority and who controls what. The price of not doing this could be to lose a critical cyberwar before we even know it's started.

        Two: impose a readily and automatically updatable, cryptographically sound layer of protection into every single device that may be net-connected. There would be a big technical, political and economic task, and I don't suggest it lightly, but again - can we afford the alternatives?

        Maybe it's time for the internet to grow up. Even if that proves painful.

  12. phillupson

    I think what puzzles me most is the cost to ISPs on managing their network, if they start dropping faked IP addresses and malformed packets then most of this could be solved at source, after all, most have enough resources to monitor and QoS traffic they don't like. I have a feeling if subscribers connections were dropped in the event that they started flooding traffic then most customers would learn how to install a free AV as well.

    1. Cris E

      But there were 150K IoT devices with real IP addresses, not thousands of fakes. And when you get the A/V software ready for my camera be sure to let me know. (Edit: number was high.)

  13. Grommet

    BCP 38

    I just wish more network operators including ISP's would implement BCP 38 to stop these kind of attacks.

    I am not one to jump to legislative answers quickly, But if they won't implement this protection out of self interest maybe it is time to do so. A bit like forcing businesses to install basic safety equipment in buildings like fire extinguishers.

    1. Matthew Elvey

      NO - Re: BCP 38

      No. BCP 38 is a good thing, but a BCP38 implementation won't block legit-seeming DNS queries to or from a DNS server.

  14. Anonymous Coward
    Anonymous Coward

    Premium Rate IP's

    What if you could op-out of some IP ranges at the ISP and one of those categories is "Current DDOS" targets. If that was the default a company could flag itself under attack and then many IOT devices fall off the team at the ISP.

    The real user could then be redirected to a proxy so valid requests to important sites under attack can be made but rate limited (to avoid the DDOS achieving it's aim to taking the website down).

    1. Ian Ringrose

      Re: Premium Rate IP's

      Just being redirected to a page explaining that to access the given website you must first log into your ISP billing system and confirm you wish to do so would help greatly.

  15. Anonymous Coward
    Anonymous Coward

    Well, I just got a 502 for Googles protected krebson site.

    Amazing cloud services are available to you all.

  16. Anonymous Coward
    Anonymous Coward

    Blame the shooter not just the weapon.

    Crims will always find a way. Best deterrent is to cut their fingers off.

    I'll even donate my time and a rusty knife.

  17. Grunchy

    Big Deal

    I don't care about DDOS, what do they overwhelm, the FBI web site? Department of Justice?

    Like, I never surf to those sites. I just have no reason to!

    Krebs internet security website gets shut down, well big whoop. It's nothing I read anyway.

    The mechanism of this DDOS seems straightforward, some URL becomes targeted for some form of traffic overload. The structure of the internet should be able to identify this situation & resolve it. I'm positive a very straightforward algorithm could be invented to detect the overload & shut it off, automatically.

    I don't see what the problem is here.

    1. Anonymous Coward
      Anonymous Coward

      If it were as easy to do that as you suggest, it would have been done 15 years ago.

    2. Phil O'Sophical Silver badge

      I don't see what the problem is here.

      That would seem to be obvious.

      Where precisely would you suggest running this algorithm on such a distributed, self-healing, network as the Internet, so that it could detect the small number of packets coming from millions of devices across the world, and recognise that they are all targeting one site? Where would the "shut off" happen? Your "solution" is like the ones that politicians offer for porn sites, insisting that "someone" should "block porn", and it shows an equal lack of understanding about how the internet works. There is no central management framework or off switch.

  18. Hans 1 Silver badge
    Joke

    Krebs? Wirklich ?

    HK: "[...] Im Osten steht der Feind an der Linie Lichtenberg, Mahlsdorf, Karlshorst."

    AH: "Mit dem Angriff Steiners wird alles in Ordnung kommen."

  19. This post has been deleted by its author

  20. NonSSL-Login

    The OVH founder was claiming a DDoS double the size of the Krebs one last week. https://twitter.com/olesovhcom/status/778019962036314112

    I am surprised Akamai could not handle that size attack considering all the load balancing and local CDN work they do for customers such as Microsoft. DDoS mitigation providers keep their bandwidth limits secret so as not to give attackers a known target to reach. Dropping Krebs at 620gbits (after all these years im still not sure about the capitals m and g's on speeds) gives some sort of insight for future attacks.

    Love Krebs blog!

    1. Wensleydale Cheese

      "I am surprised Akamai could not handle that size attack considering all the load balancing and local CDN work they do for customers such as Microsoft."

      I don't know whether Apple still use Akamai to distribute software updates, but they certainly used to.

      macOS Sierra was released last week, plus a new version of Xcode, both several GB in size.

      Just an educated guess, but it's entirely possible that Akamai had prior commitments to Apple.

  21. tr1ck5t3r
    Trollface

    For a little test, block the google.com domain and see how data from google.com comes into your firewall from google.se in a matter of milliseconds.

    If you can keep Google out, you have a secure system.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019