back to article Crypto-busters reverse nearly 320 million hashed passwords

The anonymous CynoSure Prime “cracktivists” who two years ago reversed the hashes of 11 million leaked Ashley Madison passwords have done it again, this time untangling a stunning 320 million hashes dumped by Australian researcher Troy Hunt. CynoSure Prime's previous work pales compared to what's in last week's post. Hunt, of …

  1. Lee D Silver badge

    I keep saying:

    We need a website for developers that basically says:

    Hashes

    ======

    MD5 - NO.

    SHA-1 - No.

    (ALWAYS salt hashes)

    Encryption

    ========

    DES - NO.

    3DES - NO.

    AES - With 256+ bit keys only.

    Protocol

    ======

    WEP - NO

    WPA - NO

    WPA2 - Only with AES, not TKIP.

    etc.etc.etc.

    Maybe then we can use it as a definitive reference and just say "Look... isourencryptionuseless.com says that's it obsolete, stop using it.". Fling in an API and you can have dev tools start throwing up deprecation warnings. As it is, when something like MD5 is okay, and then changes, it takes DECADES for people to catch up.

    1. MJB7
      FAIL

      Re: Hashes

      There are two entirely different usages for Hashes:

      1. Signatures/HMAC/etc. For this usage, SHA1 is not *awful*, in fact, in practise it will be fine (although I would recommend SHA256 for new code).

      2. Password storage. For this, salted SHA3 is completely atrocious. You need something *expensive* (like PBKDF2, bcrypt or scrypt).

      1. This post has been deleted by its author

        1. Frumious Bandersnatch

          Re: Hashes

          A handy mnemonic for remembering -ice vs -ise is that if you're on the fence about spelling, remember that "fence" is a noun. Works for almost all tricky things like "defence", "advi[cs]e", "practi[cs]e", "licen[cs]e" and so on.

        2. Ken Hagan Gold badge

          Re: Hashes

          "p.s. It's "in practice", practise is a verb. Cf. "in fact" and "advise vs. advice"."

          Isn't that "fact" somewhat locale-dependent?

          1. Anonymous Coward
            Anonymous Coward

            Re: Hashes

            > Isn't that "fact" somewhat locale-dependent?

            Yes. Everywhere (UK, Aus, Can, NZ) but the US we distinguish between the verb and the noun, in US English they use 'practice' for both.

    2. Anonymous Coward
      Anonymous Coward

      Don't use any hash

      Use Argon2, the winning key derivation function of the Password Hashing Competition.

      It's highly optimized for the x86, and very GPU/ASIC hostile.

      1. Anonymous Coward
        Anonymous Coward

        Re: Don't use any hash

        Hashing functions and KDFs are used for two quite different things.

        You wouldn't use a KDF when creating a digital signature, storing a password or when creating a consistency checksum.

        And in any case, a KDF is a hashing function: the clue was in the name of the competition.

    3. This post has been deleted by its author

      1. PNGuinn

        You've got a list?

        You've got a blacklist of used website passwords?

        Is it available on the interwebz yet?

      2. Adam 1

        @def

        I've definitely seen worse. The per user hash and thousands of rounds do prevent precomputed attacks and would certainly up the cost of any attack on your site. The hash iterations are there primarily to multiply the effort per guess (75K times in your case). That is obviously important but it is based on an assumed CPU time per iteration. You are somewhat limited in your iterations by the capacity of your own server (eg, it probably couldn't be a million iterations or your own server would be too unresponsive). And you are limited to the performance characteristics of a general purpose CPU. The attackers may rather use a GPU cluster or even an ASIC and be able to compensate for the additional rounds.

        Other approaches try to max out some other resource (eg RAM). If a given guess expanded out to say 100MB then the idea of tens of thousands of parallel guesses isn't practical.

        At the end of the day, use a framework. What you're doing is terrific for learning but it is so easy to stuff up (eg how random is your salt actually). I like the common password idea. We've done something similar. And minimum length is pretty much the most important metric. Mixed case/symbols/digits all simply serve to make your password hard to remember and given people use common substitutions (a/@ etc) they tend to add only limited additional security in practice)

        Checkout scrypt, bcrypt or argon2 to handle password storage though rather than something bespoke. It will otherwise end in tears

        Disclaimer <- I am not a security guy either

      3. Dom De Vitto

        Oh dear.

        Firstly, max of any number of characters is silly - you're stream hashing, so length limits are dumb.

        Secondly, known dumb password check is good, but increased complexity is better e.g. mandating symbols.

        Thirdly, re-hashing a hash isn't necessarily better - or the writers of SHA512 would have done it. e.g. collisions are going to be increased, and that means *less* security. Every rehash is another chance your super-secure password will collide with 'Password1'.

        Fourthly, user specific salting is great....if it includes internal values (account number etc.) - just adding the username to the front isn't going to help any. This also needs to be longer than the hash (512 bits+)....

        Fifthly - global salt values, hope that's another longer-than-hash value...

        1. This post has been deleted by its author

          1. Anonymous Coward
            Anonymous Coward

            Yes, just yes.

            > No. Just no.

            Yes. Just yes.

            > https://xkcd.com/936/

            Randall is showing you how to create memorable complex passwords.

            But in that he's also wrong: use a Password Manager to both remember and store randomly generated and unique passwords.

            > http://www.telegraph.co.uk/technology/2017/08/08/man-wrote-password-bible-admits-advice-completely-wrong/

            If you look at the examples in the article none are actually complex. Simple number-letter substitution doesn't make them complex.

            The actual problem is having to remember many 'complex' passwords across multiple sites that they have to change every 30 days.

            The result is users work around it by using the same password across multiple sites and simply incrementing a number on their essentially fixed password.

            This means that if one site is compromised all the sites you use are: a breach late last year at Tesco was a result of attackers using usernames and passwords they stole from a completely different site to access customers' accounts at Tesco.

            The correct answer is to use unique, which means randomly generated, and complex passwords *and to use a Password Manager*.

        2. EnviableOne

          @Dom

          Firstly no one put upper limits on the number of charachters, just lower ones

          Secondly length trumps complexitly significantly:

          8 all ascii printable (95^8)

          10 upper and lowercase letters (52^10) is stronger cyryptographically than

          if its just leters with no case pref all you need is 12 (26^12)

          if you want to get silly, and digits only you only need 16 to hit the same complexity

          Thirdly, re hashing a hash, is exactly what hashes do. but collisions wont happen more frequently as you need to do the same work to the plaintext to get the cryptext

          Fourthly and fifthly, ideally salt should be the full key space combined, so the combination of user specific and global should be that size. Using any pre known value is risky as the attacker may find this, the recommended method is random salts, (but you need to erify your entropy source)

          1. Anonymous Coward
            Anonymous Coward

            @EnviableOne

            > Firstly no one put upper limits on the number of charachters, just lower ones

            Yeah actually they do. Several sites I use, that really should know better, absolutely do put upper limits. Which is really stupid.

            > Secondly length trumps complexitly significantly:

            Length is one aspect of complexity, so that statement makes little sense. And in case it will be hashed.

            fwq1sAs is FAR better than Password123, despite the latter being longer.

            Complex and unique is what you want.

            > 10 upper and lowercase letters (52^10) is stronger cyryptographically than

            I don't think you understand what 'cryptographically' means. All that using upper and lowercase means is that there are more possible passwords.

            Unfortunately password dictionaries are far more sophisticated than you seem to think they are and 56^10 is only double 26^10 which is insignificant.

            > Thirdly, re hashing a hash, is exactly what hashes do. but collisions wont happen more frequently as you need to do the same work to the plaintext to get the cryptext

            As he said, if that really had any value the authors of the SHA-2 hash family would have already done it.

            > Fourthly and fifthly, ideally salt should be the full key space combined, so the combination of user specific and global should be that size. Using any pre known value is risky as the attacker may find this, the recommended method is random salts, (but you need to erify your entropy source)

            That's just wrong.

            A salt isn't an encryption key. It doesn't need to be secret, just unique. It's there solely to defeat *pre-computed* rainbow tables.

            1. This post has been deleted by its author

        3. Anonymous Coward
          Anonymous Coward

          > Secondly, known dumb password check is good, but increased complexity is better e.g. mandating symbols.

          Unique is even better. But that's trickier to enforce. It can be done -system generates the password- but most don't.

          > Thirdly, re-hashing a hash isn't necessarily better - or the writers of SHA512 would have done it.

          That was exactly what I thought, although I'd take it further: at best you're just wasting time. And is a classic example of why you don't design your own crypto functions. You can actually inadvertently make it weaker. Consider a stupidly trivial example: XOR. Repeating it gets you back to your original clear text.

          > Fourthly, user specific salting is great....if it includes internal values (account number etc.) - just adding the username to the front isn't going to help any.

          Err hmm. Salt should be unique but they don't need to be secret: they're not encryption keys. Their sole purpose is to defeat precomputed rainbow tables.

    4. Anonymous Coward
      Anonymous Coward

      I keep saying:

      And what you keep saying isn't very helpful.

      The problem is that most people, including you based on your post, don't understand how difficult it is to do crypto properly.

      This isn't your fault, but more the fault of crypto library providers, whether in .Net or Java. They provide a seemingly random collection of functions that the developer has to string together in order to get what they actually needed. And the decisions on how to string them together, and even what to string together, takes a lot of knowledge and is prone to error.

      For example use System.random rather than System.Security.Cryptography.RandomNumberGenerator, perhaps because you don't understand the difference or just took Visual Studios' suggestion, and it doesn't matter what else you do, your crypto is borked.

  2. Andrew Oakley

    Plausible deniability

    You call it a security weakness. I call it plausible deniability.

    1. Anonymous Coward
      Anonymous Coward

      Re: Plausible deniability

      Just checked and "badpassword" is in the list, darn it - I need to go and update all my passwords ...

      Maybe I'll use "verybadpassword" - that seems to be OK.

  3. Adam 52 Silver badge

    It's getting very hard to tell the good guys from the bad guys these days.

    I can't see who on the "good" side this helps. If you have a stash of unsalted sha1 passwords then no amount of scanning against a list is going to change the fact that you have a problem, it's only the baddies that are helped here.

    1. scrubber

      Black vs white vs grey

      The list is useful when signing up new users (or existing users changing passwords) as you now have a list of compromised passwords that you can deny them the use of in real time. Having the list of hashes locally means you aren't sending data to an untrusted third party to check.

      OTOH given SHA1 collisions it is possible that your hash might get flagged even though the password is unique and not compromised. Highly unlikely, but possible.

      1. Robert Carnegie Silver badge

        Re: Black vs white vs grey

        I don't see a reason to prevent me from using password "tliuwsusuoeucp" because someone used it in 1997 on a web chat site that was subsequently hacked.

        Currently I use Randomuniqueconsonantsthendig145 which works in most places except where some genius thinks I need to use a non-European punctuation symbol as well. Which isn't on my keyboard or doesn't work through the web interface. And which happens on my work account. 1diot0!

        And whoever bans re-used letters in passwords needs a keeck in the butt, and fiqbly. (Fiqbly is blocked as password because it is a noun followed by a surname, says some champion Scrabble player with no reason to exist.)

        1. Charlie Clark Silver badge

          Re: Black vs white vs grey

          I don't see a reason to prevent me from using password "tliuwsusuoeucp"

          The passwords that people choose is really a separate issue. The problem under discussion here is whether businesses are taking the issue seriously and storing passwords securely.

          Passwords are acknowledged to be an unsatisfactory approach to security: to be effective they need to be both sufficiently random and complex. But to be useful they need to be easy to remember. Using tliuwsusuoeucp for one password is fine; using it all for all your passwords isn't. Many of us will have at least 100 passwords and rely on reuse and key chains to manage them: both of which have their flaws.

        2. Version 1.0 Silver badge
          Pint

          Re: Black vs white vs grey

          I keep checking and "mywifesbigtits" is still secure ... righty ho, off to the pub, no need to worry about any of this.

          1. Anonymous Coward
            Anonymous Coward

            Re: Black vs white vs grey

            Looks like "myhusbandslittledick" is good too.

    2. Charlie Clark Silver badge

      The benefit is making this information public. The "bad guys" will include the various secret services who will have access to at this amount of data and much better hardware.

      If we can classify various approaches as unsafe then we can use the definitions when determining liability. Only if liability is established are businesses likely to take the problem seriously: I'm looking at you Visa with the limit on the password size.

    3. Anonymous Coward
      Anonymous Coward

      That was my first thought.

      However if the good guys can decrypt the passwords then so can the bad guys therefore it's better to release the passwords forcing users to change their passwords than leaving it so the bad guys have the passwords without their knowledge.

      1. Adam 52 Silver badge

        Why release the passwords? The hashes are already out there and that's all you need for real-time filtering.

        If you just want an individual user to change theirs then (a) they won't be reading a tech site and (b) you only need to tell them the site that was compromised, as in "change your Ashley Madison password and anywhere else you used the same one."

        1. Alan Brown Silver badge

          "Why release the passwords?"

          1: Because the hashes change with salts AND with hashing methods.

          2: Because unless you actually tell a user "Your password is 'frobnutz', pick a better one", they won't.

          3: Because the bad guys already have this list.

        2. Brewster's Angle Grinder Silver badge

          "The hashes are already out there and that's all you need for real-time filtering."

          All the hashes "out there" are using a busted hash algorithm you wouldn't want to use in a production system. (And a different algorithm will produce a different hash.)

          You could, I suppose, hash it with the broken algorithms at the same time as computing the real hash, and then filter the list. But it's easier to just check it against the plain text before hashing.

    4. a_yank_lurker

      @Adam 52, the lists are already readily available so it not as if the really bad guys could not get their hands on them on a lazy Saturday afternoon. The two goals were to make the website operators more aware of how their bad practices contribute to the problem and make users aware that short, reused passwords are recipe for personal anguish.

      1. Adam 52 Silver badge

        "the lists are already readily available so it not as if the really bad guys could not get their hands on them on a lazy Saturday afternoon"

        I dispute this. First off this rainbow table didn't exist until these people created it, some might have had it but not everyone. Secondly the lists might have been available in theory but in practice it's a bit tricky. Tricky enough to stop your casual script kiddie finding them. What Troy's done is make them available to everyone. A bit like the difference between my mother's maiden name in theory being available to anyone who cares to look at the records office and being available to anyone to download easily on a Saturday afternoon.

        So whilst there may have been a hundred or so potential users of these leaked lists there are now millions.

        It's not good to rely on obscurity but having it helps, as anyone who does more than parrot back mantras knows.

        1. Charlie Clark Silver badge
          Facepalm

          It's not good to rely on obscurity but having it helps, as anyone who does more than parrot back mantras knows.

          Stop touting your naive bullshit. In security you must always assume that you're not the first to discover an exploit. The security services and organised crime have long had access to the necessary resources for this sort of thing and, for them, creating the rainbow tables comes with a handsome pay-off-.

  4. Anonymous Coward
    Anonymous Coward

    That's lovely, but

    Can it help me get my MySpace password back so I can close the account?

    The mechanism on the s(h)ite doesn't actually work, even if you have the correct email address and username.

    1. herman

      Re: That's lovely, but

      There is a simple, though somewhat messy solution. Your MySpace account problem will go away when you die, which can be arranged.

      1. Charles 9

        Re: That's lovely, but

        Oh, what's to stop someone else stealing your identity after you die? Remember ghost votes?

        1. Khaptain Silver badge

          Re: That's lovely, but

          After your dead, identity theft is no longer your problem ;-)

          1. This post has been deleted by its author

          2. Anonymous Coward
            Anonymous Coward

            Re: That's lovely, but

            True, but I want to close it now so that it's not embarrassing me any more (And no, you're not seeing my username!)

    2. phuzz Silver badge
      Thumb Up

      Re: That's lovely, but

      Report it for having copyright infringing content and Myspace will close it straight away.

      1. Anonymous Coward
        Joke

        Re: That's lovely, but

        Past me has used my likeness without my permission!

      2. Ken Hagan Gold badge

        Re: That's lovely, but

        Since the OP has the correct email and password, they can go one better and actually post the violating content. Bonus points if you can get the whole site closed down.

  5. David Webb

    Not really correct

    the hash gets stored in the database, and it's supposed to be impossible to get the password from the hash.

    It's never been impossible to get the password from the hash, it's always been impractical. You could always brute force the hash to get the password, as long as you had several thousand years to spare.

    Is the password a - no

    Is the password b - no

    ..... (several thousand years later)

    is the password U^7cw==1jjJWvna@~!("jlskInsertSaltHereM:;m089lk,mjL:KJH-0u9 - yes

    1. This post has been deleted by its author

      1. GrumpenKraut

        Re: Not really correct

        > So, even if you find something that produces the hash, how do you know you found the 'right' cleartext?

        Should not matter. Any cleartext that produces the hash would be accepted as password, right?

        1. This post has been deleted by its author

          1. hmv

            Re: Not really correct

            @Symon: I think you might be grasping at indefeasibly small invisible theoretical straws there.

            Yes there's a tiny chance that a password hash cracker will get the wrong cleartext, but it is also likely to come up with the smallest possible cleartext which is also the most likely password a user will have chosen.

      2. hmv

        Re: Not really correct

        Obtaining the 'right' cleartext isn't necessary; as long as you have a text that produces the right hash you have the password equivalent.

        The 'thousands of years' for brute forcing password hashes is a low-end estimation in the example given, but is deceptive - it is possible to brute force a collection of hashes and you will usually get some useful results.

        1. DJ Smiley

          Re: Not really correct

          Except it is, if every site used a different salt and encryption, but they don't because (they are dumb and don't add salts) and we constantly tell them to use encryption X because it's the best right now.

          1. Anonymous Coward
            Anonymous Coward

            Re: Not really correct

            Its not correct ... as brute force attack is not acquiring the clear text from the hash - this is simply making a great deal of guesses. Therefore the hash successfully stands as not reverse engineered - almost any password scheme can be broken by brute force eventually as the encryption process is irrelevant, that's why we (should) have long complex passwords that make it impractical.

        2. Alan Brown Silver badge

          Re: Not really correct

          "it is possible to brute force a collection of hashes and you will usually get some useful results."

          And this is where things go pearshaped.

          The first time we ran "crack" against our password file, a large number of passwords such as '{username}', 'password' and 'letmein' fell out within seconds.

          Yes, users really are that stupid, as well as believing that numeric swaps such as 0/o, 4/a, 3/e, 1/i make them more secure.

          1. ArrZarr Silver badge

            Re: Not really correct

            @ Alan

            Numeric swaps are useful for when a password system demands a number in the password. For me, they aren't used to make the password more secure; just to aid remembering the password.

      3. David Webb

        Re: Not really correct

        From my long term storage memory, I seem to recall it works like this (using MD5, yes, really long term storage....)

        Website asks "what do you want your password to be?"

        You input "password"

        Server gets "password", adds some salt, then hashes it into MD5. It then stores the MD5 and salt in a database (except where the salt is hard coded), so now your password is b305cadbb3bce54f3aa59c64fec00dea (passwordsalt).

        You visit the website the next day, it asks "what is your password?"

        You input "password", the server adds the salt and hashes the password then checks the result.

        If the result is b305cadbb3bce54f3aa59c64fec00dea then you enter, if the result is not b305cadbb3bce54f3aa59c64fec00dea then "password incorrect", this way the website never knows your password.

        Now, imagine that saltpassword produced the same hash as passwordsalt (it doesn't, but imagine), if you go to the website and put in saltpassword, the server doesn't know it's the wrong password, all it knows is the result of the hash, the result causes a collision and the password is accepted.

        Also, yes, 1,000 years is an underestimate, the possible number of combinations for a 15 letter Upper/lower/numeric password is 62^15 =768,909,704,948,766,668,552,634,368. In 2012 a rig could brute force 348 billion passwords per second so to guess that password would take 70,063,123 years, roughly.

  6. petethebloke

    Not really correct, but close

    Most of the sites that have leaked passwords are not banks. If you get a "password" from a hash collision it is still unlikely to work in other sites.

    @symon It does matter for Git because open-source relies on hashes not colliding to confirm that the code is right and proper.

    1. Donn Bly
      Boffin

      @petethebloke Re: Not really correct, but close

      Close but no cigar -

      First, because of password reuse and poor password storage methods - the entire POINT of this exercise - it is most DEFINITELY LIKELY that a password extracted from a hash collision is still likely to work on other sites, because an unsalted hash one one site is going to match the same unsalted hash on another.

      Second, when you take a 4K block of code (or 1K, or even 256 bytes) and reduce it into a hash, you are ALWAYS going to have the possibility of collision between other blocks of unrelated code. In fact it is actually such a high probability that the hash CANNOT be used for authentication in an of itself. If you can't rely on the hash on a string of thousands of characters for collision-free authentication, then a string of 8 to 10 is even less reliable.

      The hash can, however, be used with a high level of certainty to detect changes in the source code, taking a hash of a block of code and comparing it to a hash previous block to see if it changed. taking two hashes using different algorithms is even better. In this use case, the hash algorithm doesn't really matter - it can be sha1, sha256, or even a CRC32 or something else -- and that is why it doesn't matter for Git.

      1. Anonymous Coward
        Anonymous Coward

        Re: @petethebloke Not really correct, but close

        "First, because of password reuse and poor password storage methods - the entire POINT of this exercise - it is most DEFINITELY LIKELY that a password extracted from a hash collision is still likely to work on other sites, because an unsalted hash one one site is going to match the same unsalted hash on another."

        Well then, humans need to evolve a better memory if they want to deal with the problem of hundreds of sites that NEED (for security and/or liability reasons) their own login credentials each and every time and may not have access to some kind of store because they don't own a device they can really trust.

        "The hash can, however, be used with a high level of certainty to detect changes in the source code, taking a hash of a block of code and comparing it to a hash previous block to see if it changed. taking two hashes using different algorithms is even better. In this use case, the hash algorithm doesn't really matter - it can be sha1, sha256, or even a CRC32 or something else -- and that is why it doesn't matter for Git."

        Unless they can make a change and STILL come up with the same hash. It's called a Second Preimage Attack. The easier it is to make a collision, the easier it is to engineer a Preimage Attack.

  7. Anonymous Coward
    Anonymous Coward

    Mmmmmm.....hash

    All this talk of Hash and Salt : I'm picking up a tin of Corned Beef on the way home tonight.

  8. Anonymous Coward
    Linux

    Uh, oh!

    I see "ILoveLinux" is in the pwned password repository, so very many El Reg readers are at risk!

    1. TheVogon

      Re: Uh, oh!

      As is for instance youcunt, motherfucker, suckmeoff, ifuckpigs, and most other abusive terms i can think of!

      theregister is also there!

  9. Anonymous Coward
    Anonymous Coward

    Pine cone

    Robert

    Cortina

    1966

  10. Anonymous Coward
    Anonymous Coward

    scott

    tiger

  11. Version 1.0 Silver badge

    It's not just the users at fault

    Accounts, with all their details, live forever - most websites make it impossible to actually delete an account when you are finished with whatever service you signed up for. You want to try out dynamic DNS? You gotta sign up for an account and if you don't like the service or simply never use it, your account details and password stay on forever in their database.

    Open your browser and go look at those saved passwords, you'll have accounts in there that you haven't used in years but they are still sitting in the site database somewhere waiting for a little cross-site scripting.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon