Sign in / up

back to article Jeff Bezos feels a tap on the shoulder. Ahem, Mr Amazon, care to explain how Capital One's AWS S3 buckets got hacked?

After last week's revelations that a hacker stole the personal details of 106 million Capital One credit card applicants from its Amazon-hosted cloud storage, a US Senator has demanded Amazon CEO Jeff Bezos explain what exactly what went wrong. The sensitive information was siphoned from Capital One's Amazon Web Services S3 …

COMMENTS

Post your comment
House rules Send corrections
Add to 'My topics' unstar *
  1. Donn Bly

    Whose Fault

    If a customer takes an enterprise class firewall and configures it in such a way to be insecure, is it the customer at fault or the manufacturer who "allows" its products to be used insecurely?

    What ever happened to taking responsibility for your own stuff? We aren't talking about a $40 home router bought at a box store and plugged into a cable modem, we are talking about enterprise-class device configurations that don't route ANYTHING unless being told to do so. Proper security is HARD and mistakes can happen, but this senator sounds sounds clueless. I read his request as a demand for Amazon to deliver a list in 6 days of "all other companies that in the last 2 years have had their data pillaged while stored on AWS using both known and unknown vulnerabilities as well as misconfiguration". The scope of that question is enormous, and to expect an accurate answer in days?

    I get the points about asking if there are known security-related bugs in their services or whether she used inside knowledge and access from her brief stint at Amazon to conduct the raid, but at this point I haven't even seen enough on the Capitol One fiasco to know whether the misconfigured firewall even sat in the AWS cloud or was a on-premise firewall that had a tunnel to AWS. All I have seen is that Capitol One (not Amazon) reconfigured the firewall to close the hole, and that she used the same VPN to hit the buckets as her GitHub account so it was trivilly easy to trace her back. Neither of those makes it sound like an inside job, nor would I expect AWS to be the one heading up the forensic examination.

    I'm going to wait on the dust to settle so that an after-action report supported by documentation is written and published before I draw any other conclusions, and the US Senate should too. My guess is that the investigation by the FBI is "ongoing" and it would be improper for Amazon to even answer some of those questions.

    21 18 Reply
    1. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: Whose Fault

      I've read about a lot of "misconfigured" buckets and I have also wondered why it seems so hard for all these companies to protect my data on Bezo's cloud.

      I am thankful that senator Wyden is asking the same question(s) as I would if I were in his position.

      37 5 Reply
      1. Anonymous Coward
        Reply Icon
        Anonymous Coward

        Re: Whose Fault

        "I have also wondered why it seems so hard for all these companies to protect my data on Bezo's cloud."

        In the past the following configuration was used:

        Internet -> Firewall -> DMZ -> Firewall -> Data

        These days people put data in some vague bucket on someone else's computer and pat themselves on the shoulder about being so secure. It's no more than logic that at some point problems start to occur if you ask me, the whole model is fundamentally flawed on every level.

        10 4 Reply
        1. Anonymous Coward
          Reply Icon
          Anonymous Coward

          Re: Whose Fault

          You can have the same setup in Amazon.

          Internet -> Firewall -> Public facing subnet -> Firewall -> Private Subnet -> Data

          So many people on this board have no idea what they are talking about...

          9 0 Reply
          1. Anonymous Coward
            Reply Icon
            Anonymous Coward

            Re: Whose Fault

            "You can have the same setup in Amazon."

            And then still a group of unknown Amazon employees has access to your data. Also, the infrastructure console interface is directly accessable from the internet.

            Furthermore, the article was about S3 storage. Does that run behind the two firewalls? If all it takes is a leaked key or bug to access the data, then I wouldn't feel good about it.

            1 2 Reply
            1. Peter-Waterman1
              Reply Icon

              Re: Whose Fault

              "Also, the infrastructure console interface is directly accessible from the internet."

              Correct, so is your HSBC logon page. Hardware-based 2FA should secure that for you, just the same as any secure website on the internet / VPN connection into your on-prem network.

              "Furthermore, the article was about S3 storage. Does that run behind the two firewalls?"

              Yes - You can have S3 Buckets behind two firewalls using S3 VPC Endpoint within your private subnet without exposing onto the internet.

              4 1 Reply
              1. disk iops
                Reply Icon

                Re: Whose Fault

                > Yes - You can have S3 Buckets behind two firewalls using S3 VPC Endpoint within your private subnet without exposing onto the internet.

                utter rubbish!

                A VPC service endpoint simply means your request traffic that originates from within the VPC doesn't need a NAT or Internet Gateway to hit the S3 web tier servers that sit in publicly reachable IP space. The traffic stays "internal" to AWS datacenter routing CIDR.

                The information disclosure has nothing to do with someone sniffing the request/response traffic. Rather the security permissions on EACH object stored in the S3 bucket. If you mark an object as public or have a bucket policy that makes some/all of the paths public, the object WILL be served to you no matter how you requested the object.

                I can sit here in my own VPC with S3 endpoint and grab every public bucket object I want and it's totally legit. The bucket/object owner is exclusively the idiot who allowed people to enumerate and GET the objects.

                The reason there are so many cases of public objects which weren't intended to be so 2 fold:

                1) early on the defaults made it quite easy to pick 'public' without clear warnings as to what that meant

                2) Bucket policies are 'hard' to write and most people are simply too STUPID or lazy to tackle the topic and out of frustration just click the 'public' button in order to shut up the developer who also can't be bothered to use an IAM role bleating about his delivery is being impacted because he can't see the data.

                IT is not for stupid people but there are a many millions who are involved because they are cheap.

                1 0 Reply
        2. hmv
          Reply Icon

          Re: Whose Fault

          "so secure"

          I believe you may very well be far too optimistic there. I'm not sure the word "security" ever passes unobstructed through these people's minds.

          0 0 Reply
    2. Snake Silver badge
      Reply Icon
      IT Angle

      Re: Please re-read the article, between three lines

      "We aren't talking about a $40 home router bought at a box store and plugged into a cable modem, we are talking about enterprise-class device configurations that don't route ANYTHING unless being told to do so"

      With that statement, you have apparently missed the point of the senator's enquiry: Are you SURE about that?

      Amazon may *claim* that their configurations are correct yet 106 million data points, taken from a company that most certainly has an in-house tech department and therefore tried its very best to assure data safety compliance, seem to prove otherwise.

      You are assuming automatic secure profile configuration. Remember the union rule about assume, "You make an a...."

      31 3 Reply
      1. 0laf
        Reply Icon
        Meh

        Re: Please re-read the article, between three lines

        The senator may be clueless but unless he's been working on cloud technologies in the last few days following a significant amount of training and previous IT work experience then tbh he's going to struggle to get his head round cloud full stop.

        Cloud security is feckin hard. The shared responsibility model seems to work very well for the big vendors, "yeah we're really really secure, unless we're not; then it's your fault".

        I know of more than a few significant hacks that have happened because of a single mistake in a single checkbox buried in the depth of a cloud admin console. This stuff might just be too complex and too motile to keep on top of.

        16 0 Reply
        1. Anonymous Coward
          Reply Icon
          Anonymous Coward

          Re: Please re-read the article, between three lines

          If you can't handle cloud security, wtf are you doing selling cloud space - don't dry to me about how hard it is to secure something that you decided would be a lucrative business venture. Do it right or don't do it at all.

          4 4 Reply
          1. Kiwi
            Reply Icon
            Facepalm

            Re: Please re-read the article, between three lines

            don't dry to me about how hard it is to secure something...

            Do it right or don't do it at all.

            And thus, you learn one of the fundamental lessons about computer security... (well, two actually). A single simple typo can screw up a month's worth of security work. Or a year. Or a day. And it can be easily missed even by the best of us.

            (The other one is... Don't say things like "Do it right or don't do it at all" unless you proof and re-proof your post :) )

            12 0 Reply
    3. macjules
      Reply Icon

      Re: Whose Fault

      I am inclined to agree with what you say with a major caveat: I do not think that anyone can possibly factor in the criminal intent of a member of staff to undermine or thwart your security above or beyond what secure measures you take to harden your Security Group firewall rules or IAMs role permissions.

      11 1 Reply
      1. Anonymous Coward
        Reply Icon
        Anonymous Coward

        Re: Whose Fault

        re; macjules

        Yes, you can but its more than twice the cost. In a previous job, because of the astronomical cost of outages, $500k/hour, the other administrator and I never made any core network changes solo. One of us would type and the other would look over a shoulder....two brains are better than one.

        4 0 Reply
        1. Kiwi
          Reply Icon
          Pint

          Re: Whose Fault

          One of us would type and the other would look over a shoulder....two brains are better than one.

          Used to do that even for much smaller things.

          Especially when cloning a customer's disk. If you realise you got your OF and IF around the wrong way and cancel DD 1/110th of a second in, it's too late (well, recovery tools can help but they can be messy...)

          0 0 Reply
    4. WmK
      Reply Icon

      Re: Whose Fault

      'a demand for Amazon to deliver a list in 6 days of "all other companies that in the last 2 years have had their data pillaged while stored on AWS using both known and unknown vulnerabilities as well as misconfiguration".'

      Never mind being an enormous task, it's also an impossible one. Why would AWS know about all client screwups that led to client data losses?

      6 1 Reply
    5. steviebuk Silver badge
      Reply Icon

      Re: Whose Fault

      Senator Wyden is far from clueless. Look at his response to Trumps shills on encryption.

      12 1 Reply
  2. fredesmite
    Mushroom

    Remember - Cloud computing

    Is nothing more than putting your crap on someone else's computer that other people are using ,

    and expecting the owners to care more about it than you do..

    No wonder nothing works anymore. You can't find anyone responsible for anything.

    A simple SQL query is all you need if you have credentials of the DB:

    connectstring="ip=10.10.10.10, user=admin/password"

    db = cx_capone.connect(connectstring)

    query = "select *from system.capOne. where rownum< 1000000"

    cursor = db.cursor()

    cursor.execute(query)

    results = cursor.fetchall()

    pprint.pprint(results)

    13 11 Reply
    1. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: Remember - Cloud computing

      ...is nothing more than putting your crap on someone else's computer that other people are using, and expecting the owners to care more about it than you do..

      ^^^^

      This!!

      27 4 Reply
      1. Anonymous Coward
        Reply Icon
        Anonymous Coward

        Re: Remember - Cloud computing

        An alternative view: why wouldn’t you expect a company who’s entire business model depends on having customers trust them to hold their data, devote *far* more resources to platform security than organisations where IT is an enabler for their core functions, and is often being pushed to reduce costs?

        22 4 Reply
        1. Doctor Syntax Silver badge
          Reply Icon

          Re: Remember - Cloud computing

          "organisations where IT is an enabler for their core functions, and is often being pushed to reduce costs"

          Where IT is so deeply embedded in those core functions they are IT businesses whether they like it or not and whether they care or not and attempting to reduce costs is not a good idea. They are businesses whose entire business model depends on having customers trust them to hold their data. That's not a trust they can weasel out of by pushing it onto a third party, nor can either of them get out of it by finger pointing.

          8 0 Reply
          1. BoldMan
            Reply Icon

            Re: Remember - Cloud computing

            Try explaining that to the Board Members who'd rather be playing golf than actually doing some work to understand the business they are in charge of.

            7 0 Reply
            1. Commswonk
              Reply Icon

              Re: Remember - Cloud computing

              @ BoldMan: Try explaining that to the Board Members who'd rather be playing golf than actually doing some work to understand the business they are in charge of that is paying them.

              FTFY...

              6 0 Reply
        2. John Brown (no body) Silver badge
          Reply Icon
          Windows

          Re: Remember - Cloud computing

          "why wouldn’t you expect a company who’s entire business model depends on having customers trust them to hold their data, devote *far* more resources to platform security"

          I absolutely agree with this statement because "the security of your data is our number one priority".

          The head of our PR ------------------------------->

          4 0 Reply
        3. Anonymous Coward
          Reply Icon
          Anonymous Coward

          'why wouldn’t you expect a company who’s entire business model'

          Because as long as companies think they only have to 'maximize shareholders value', they will always try to compress costs and wages to increase profits until they go a step too far and customers pay the price. And that's regardless of their business model.

          Moreover when you're just a customer among many others, and not a critical one, anything that happens to you is just part of the risks of trying to maximize that shareholder value.

          A customer-focused company would be another thing, but I can't see any around. Once you're a monopoly or a duopoly, customers becomes irrelevant.

          5 0 Reply
        4. M.V. Lipvig Silver badge
          Reply Icon

          Re: Remember - Cloud computing

          BT's number one business is routing telephone calls. How many of their customers are dissatisfied? And yet they are still in business.

          BA's number one business is flying passengers. How many of their customers are dissatisfied? And yet they are still in business.

          What you describe is a perfect world company. What fredesmite describes is the real world. If they lose your data and you go out of business as a result, they will stop caring once your final check clears and the next day, they won't even remember your name even though they just gave you a good hard focking.

          1 0 Reply
    2. phuzz Silver badge
      Reply Icon

      Re: Remember - Cloud computing

      So the same as any hosting then basically.

      The cloud is just rented VMs, it's not some new and special case. It's good for some uses, rubbish for others.

      4 0 Reply
  3. chivo243 Silver badge
    Big Brother

    Didn't the slurper work for Amazon in some capacity?

    I thought I read that the woman taken by a dozen or so officers in camo was an Amazon employee? What encryption is this that governments are beefing about?

    Sorry crossed the streams here...

    4 2 Reply
    1. disk iops
      Reply Icon

      Re: Didn't the slurper work for Amazon in some capacity?

      certain employees have out-of-band access to objects because they can walk the metadata and hit objects "in the raw" and bypass the ACL mechanism. At some level this criminal had 'privileged' knowledge like which VPC or IP was defined in the bucket policy ACL and had abused/retained sufficient access to scrape the data.

      0 0 Reply
  4. Mark 85

    Interesting Timing...

    I'm not a fan of Amazon or Bezos but I do have to wonder about the timing of this. What's scary is that they are in the running for that fat, exclusive Defense Department contract (JEDI). If they won't help with Netflix's situation, I would wonder what they won't do for the Defense Department.

    14 2 Reply
  5. hammarbtyp

    Sorry I thought allowing backdoors into secure data was now US government policy

    https://www.theregister.co.uk/2019/07/23/us_encryption_backdoor/

    I wish they would make there mind up. Do they want it secure or not?

    24 2 Reply
  6. Anonymous Coward
    Anonymous Coward

    Apparently, it's easy to screw up AWS security

    I spent several months working at a managed services company helping to ride herd over a slew of web sites running on AWS. First day there we had a quick meeting to dole out who would handle what clients' servers in a case of "Who Let The Keys Out?". As a result of that screw-up, a bunch of miscreants were able to spin up a slew of the expensive flavor of AWS instances in several clients' environments. After that mess was cleaned up, it happened again a few months later. (I was rather glad my time there was short.)

    3 2 Reply
    1. Anonymous South African Coward Bronze badge
      Reply Icon

      Re: Apparently, it's easy to screw up AWS security

      First day there we had a quick meeting to dole out who would handle what clients' servers in a case of "Who Let The Keys Out?".

      You can also find out who's responsible for such things, and bribe/threaten/blackmail that person.

      1 0 Reply
  7. Anonymous South African Coward Bronze badge

    So this is the Dark Side of The Cloud.

    Now I'm waiting all the more in anticipation for the winner of the JEDI contract... now THAT should be a real hoot if something goes wrong...

    If you put it into a PUBLIC place, lots of people may see it, and you have little to no control over who's able to access the data. After all, it's on an Internet-facing server with an unknown security configuration.

    If you host it yourself, it's more PRIVATE and you have more control. You can opt to put sensitive data on a server that have no internet-facing, or is even air-gapped from the internet. Much better, plus YOU have full control over the security configuration, and should something go wrong, you can pull the plug quickly, so to speak.

    15 0 Reply
    1. Tree
      Reply Icon
      FAIL

      Give Mr Bozos a bath with a bucket of slop

      He named his company after bull-dyke female warriors, and this person, Paige, was a man who thinks he is a woman. Too humorous! Any outfit that trusts their private stuff to Wireless or Cloud organizations deserves to be sued for malfeasance. Use private landlines and sneakernet only. Iran's nuclear Calutrons were only accessed using USB keys to load Stuxnet malware. The Capital One machines had better not have USB ports or they are asking for IT.

      0 2 Reply
      1. Kiwi
        Reply Icon
        WTF?

        Re: Give Mr Bozos a bath with a bucket of slop

        Any outfit that trusts their private stuff to Wireless or Cloud organizations deserves to be sued for malfeasance.

        So... What of those who wish to do international trade, or provide other services that people want today?

        What of businesses that are growing but cannot afford new tin, or have times of massive spikes that'd require increasing their hardware by 2,000% for a week out of the year, but otherwise could run on a single P90 with a 20MB HDD for the rest of the year?

        With reasonable steps to make things secure, cloud computing is the best option for a number of options. It's also a hell of a lot better for the environment.

        0 0 Reply
  8. Anonymous Coward
    Anonymous Coward

    Stupid politician, if it's connected to the Internet then it's hackable. When will these muppets learn there is no such thing as computer security, only levels of insecurity.

    From the Shawshank Redemption.....

    Geology is the study of pressure and time. That's all it takes really... pressure... and time.

    1 16 Reply
  9. Claptrap314 Silver badge

    Bean counters only count beans

    As part of a programming exercise, I created an account on AWS & spun up a service. (Well, mostly.) I used a common provisioning tool, and following examples promptly ran into a bug in the tool. By "bug", I mean, "The product to follow the documentation within reasonable parameters." Digging around, I found that this was not a rare error--it had to do with the default region leaking into my configuration. Of course, I had a virgin system, which is why I was hitting the bug. It had been almost ten years since playing with AWS, and the rule is STILL, "First, sit down with your application and their API, and write (and test) your scripts lest you go insane".

    As I have repeatedly stated, for a lot of small to medium companies, cloud provisioning makes a LOT of sense in that you don't have to worry about spinning up a dozen servers across multiple time zones in an attempt to deliver four nines of uptime to your first one hundred customers.

    That does NOT mean you can do it without a highly skilled cloudops engineer plus a couple of less senior guys. There is a bare minimum of human effort required.

    But middle management read about "zero administration" deployments, and decided that means that at most you need an intern.

    20 0 Reply
    1. Glen 1
      Reply Icon

      Re: Bean counters only count beans

      "That does NOT mean you can do it without a highly skilled cloudops engineer plus a couple of less senior guys. There is a bare minimum of human effort required.

      But middle management read about "zero administration" deployments, and decided that means that at most you need an intern."

      There is a whole spectrum in between, the end result of which might not be distinguishable until something goes wrong.

      The first hit for a devops engineer job in my area pays 55K. That would be massive overkill for, say, a webshop punting *almost* custom wordpress sites.

      While IT is thought of as a cost centre, manglement will mangle.

      6 1 Reply
  10. Anonymous Coward
    Terminator

    Tech insecure by design?

    The problem isn't that the tech is insecure by design, the problem is the people writing the code don't address security issues and assume security is someone elses problem and the people implimenting the tech are inept and under-trained and assume security is someone elses problem.

    7 2 Reply
    1. Anonymous Coward
      Reply Icon
      Anonymous Coward

      Re: Tech insecure by design?

      >The problem isn't that the tech is insecure by design,

      Yes it is, did Intel flaws teach you nothing ?

      11 2 Reply
      1. Peter X
        Reply Icon

        Re: Tech insecure by design?

        Conversely, it's worth noting that if Netflix discovered an issue, reported it and didn't get a response, who's problem is it?

        And thus, Capital One engineers apparently failed to spot something that Netflix engineers did spot. To me it looks like the Capital One engineers are (1). not as good as the Netflix engineers, and (2). considering the sensitivity of the data they're dealing with, maybe Capital One need to improve their hiring/training?

        That's not to say I think Amazon are blameless, but if people (Netflix engineers at least) can spot the problem, then so can anyone, and since they (Netflix/Capital One) have a choice in which cloud provider they want to use, I can't see how Amazon can be at fault for this.

        13 0 Reply
      2. JassMan
        Reply Icon

        Re: Tech insecure by design?

        Not only that, but I bet a lot of manglent types see S3 and assume it means level 3 security (whatever that may be) especially when they see that buckets are accessed with a key.

        Unfortunately, S3 just means Simple Storage System and the key is just a unique identifier because like the internet is bigger than a desk and c: to z: just doesn't hack it. Nothing about AWS implies security. It is up to the client to provide their own. So yes it insecure by design. FFS its not even posix compliant.

        4 2 Reply
      3. Anonymous Coward
        Reply Icon
        Anonymous Coward

        Re: Tech insecure by design?

        >The problem isn't that the tech is insecure by design

        “Yes it is, did Intel flaws teach you nothing ?”, Anonymous

        Intel skipped testing the L3 cache for side-channel attacks as management wanted to rush the processor out the door to catch up with AMD. As such it was management interference with production.

        3 0 Reply
      4. Claptrap314 Silver badge
        Reply Icon

        Re: Tech insecure by design?

        You've not read anything here posted by those of us with knowledge at that level, have you?

        Anonymous coward indeed.

        2 0 Reply
    2. diodesign (Written by Reg staff) Silver badge
      Reply Icon

      Re: Tech insecure by design?

      Don't take sub-headlines too literally - they are short on space and high on flippancy. It means tech in the context of AWS: whether Amazon's cloud is insecure by design because it may be that it's too easy for customers to lose control of their data. It's just a thought, not a solid accusation.

      C.

      7 0 Reply
  11. Doogie Howser MD

    Another grandstanding politician

    I've not much time for Bezos but even less for politicians trying to make headlines. For starters, if S3 buckets are the attack vector then firewalls have sod all to do with it, S3 buckets are internet facing by design. Secondly, the instructions are clear on good practice - private by default (which new ones are now anyway) and don't embed secret keys in human readable code.

    The other aspect to this story is that from what I can gather, it was an inside job anyway. So regardless of whose computer it was, if you have the keys to the kingdom, you can do what you like.

    3 14 Reply
  12. Anonymous Coward Silver badge
    Big Brother

    "Lest you think Wyden is a knee-jerk Luddite picking on President Trump's least favorite tech leader, he's actually one of the most technologically literate Congresscritters out there."

    The one does not preclude the other.

    The most technologically literate critter can still be a knee-jerk luddite (and it was nice of you to include the 'knee' and 'luddite' there)

    4 0 Reply
  13. EnviableOne

    Props to Wyden (Again...)

    Now i wish some of the idiots in whithall had his grasp of the technology, we might get an end to this whole safe backdoors thing

    The Honourable member from Oregon may be a lot of things, but a luddite is not one of them, and he has been behind some of the best legislation sitting on Mitch McConnell's desk and even a few pieces that made it onto the floor

    10 0 Reply
  14. Kiwi
    Boffin

    "a US Senator has demanded Amazon CEO Jeff Bezos explain what exactly what went wrong."

    'Well you see sir, we're aware that there will soon be a law that says we must have back doors in our encryption. We were running a test of such a back door.

    As the government has previously been told, the existence of such a back door will allow anyone free access as it would not remain a secret door for long. Our test showed that while we were in full compliance with the proposed back-door laws, we are unable to comply with any data protection laws. The two are mutually exclusive."

    [Ok, I should probably try that when I've got a clearer head, but you get the gist :) ]

    1 0 Reply

POST COMMENT House rules

Not a member of The Register? Create a new account here.

Forgotten password ?

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like