back to article LibreSSL RNG bug fix: What's all the forking fuss about, ask devs

A bug found and fixed in LibreSSL, the OpenSSL fork maintained by OpenBSD developers, is “catastrophic" or "overblown", depending on whom you talk to. Just days after the release of a portable version of the crypto library, a flaw was reported in LibreSSL's pseudo-random number generator – its PRNG, a vital component in strong …

  1. bazza Silver badge

    Well, at least we seem to have a code base where bugs can be found, located and fixed fairly quickly. So much better than OpenSSL; the 'fixing' part was not achievable.

    The most important feature that anything secure must have is the ability to rapidly update and deploy in the face of bugs. Kudos to the LibreSSL guys for bringing that back. Now, if only Google can learn that lesson too...

    1. Daniel Palmer

      >Now, if only Google can learn that lesson too..

      Not sure what you're really referring too but I had to make a guess I would think this is about Android and vendors not shipping updates? OS level updates can't be pushed out via Play. The Android system partition is read only for a good reason. I guess what Google needs to do is either get more vendors shipping vanilla builds that Google will manage the over the air updates for or split the system partition up a bit so that vendors can add their junk in there but google can offer partial OTA updates for vital security updates. Kernel updates will be tricky as usually SoC vendors are very lazy. They'll get some old crap version of Linux working, release that as a BSP and forget about it. So if fixes for major issues are pushed to the Linux mainline it may take forever for those fixes to actually appear in the kernels for all of the devices out there.

      The steps between "git push origin master" and a patch being deployed on millions of devices are more complex than fixing the issue itself in most cases.

      1. bazza Silver badge

        "Not sure what you're really referring too but I had to make a guess I would think this is about Android and vendors not shipping updates?"

        Yep, that's the one.

        "OS level updates can't be pushed out via Play."

        And that doesn't look like a very clever idea now, does it. Come to think of it, Google must have looked at Linux, OS X and Windows (to name but a few) with their auto updating mechanisms and decided that pushable updates were a bad idea. What on earth made them think they would never have to do the same.

        "The Android system partition is read only for a good reason."

        And what is that reason? Judging by the amount of malware in the Android ecosystem it's certainly nothing to do with stopping bad things running on a handset.

        "I guess what Google needs to do is either get more vendors shipping vanilla builds that Google will manage the over the air updates for or split the system partition up a bit so that vendors can add their junk in there but google can offer partial OTA updates for vital security updates. Kernel updates will be tricky as usually SoC vendors are very lazy. They'll get some old crap version of Linux working, release that as a BSP and forget about it. So if fixes for major issues are pushed to the Linux mainline it may take forever for those fixes to actually appear in the kernels for all of the devices out there."

        Or you could do it properly, which is what Microsoft have tried (and mostly succeeded) to do. Which is, define a hardware architecture to which manufacturers must comply, and then MS can push out updates as and when necessary. Just like they do on PCs.

        The Internet of Things is going to suffer quite badly too unless some major players take control and set up a reasonable hardware standard to which everyone can comply.

        1. Daniel Palmer

          >And that doesn't look like a very clever idea now, does it. Come to think of it,

          So you want apps to be able to mess around in the /system partition and make your device unbootable?

          >Google must have looked at Linux, OS X and Windows (to name but a few) with

          > their auto updating mechanisms and decided that pushable updates were a bad idea.

          Google has moved more of the userland into the playstore so that it can be updated. OTA updates are supported for the OS itself. If vendors don't want to ship OTA updates then I'm not sure what exactly Google is meant to do other than take back control of the OS builds vendors ship (via their Nexus etc devices). When they do that they are accused of being control freaks. They really can't win.

          >Or you could do it properly, which is what Microsoft have tried (and mostly succeeded) to do.

          I'm not sure Windows update is the pinnacle of OS updating schemes. If you said they should have looked at apt or yum you might have a point.

          >Which is, define a hardware architecture to which manufacturers must comply,

          >and then MS can push out updates as and when necessary.

          Never going to be possible for phones unless you only ship a very restricted set of devices. It's working for Apple but not for MS. Apple use the same scheme as Android devices should for OS fixes btw.

      2. toejam13

        I guess what Google needs to do is either get more vendors shipping vanilla builds that Google will manage the over the air updates for or split the system partition up a bit so that vendors can add their junk in there but google can offer partial OTA updates for vital security updates.

        Another option would be for Google to keep a legacy kernel interface and driver model in newer versions of Android. One commonly cited reason for so many handsets being left on the 2.3 Gingerbread is because so many things changed in the kernel in 4.0 Ice Cream.

        I've heavily utilized the legacy driver support in Windows over the decades, and it has saved me from throwing out an older machine or peripheral card more than once. I have an older laptop running W7 with shimmed XP video drivers (XDDR) in the kids' room right now. That's a great thing for Microsoft to have left in.

        1. Daniel Palmer

          @toejam13

          >Another option would be for Google to keep a legacy kernel interface and

          >driver model in newer versions of Android. One commonly cited reason

          >for so many handsets being left on the 2.3 Gingerbread is because so

          > many things changed in the kernel in 4.0 Ice Cream.

          Linux has backports for things like WiFi drivers etc and options in the kernel to make it compatible with older userland utils etc. That won't stop chip vendors being lazy.

  2. A Non e-mouse Silver badge

    5 seconds of fame

    From the release notes of the very first portable version of LibreSSL:

    "This is intended as an initial release to allow the community to start using and providing feedback"

    I think this corner case bug is more feedback on a .0 release of a piece of software rather than a massive security scare.

    At least the LibreSSL people took the bug report and fixed it within a reasonable time frame.

    Remember people: Writing security & cryptographic code is hard. Very hard.

    1. Charlie Clark Silver badge

      Re: 5 seconds of fame

      Yep, the release was exactly for this purpose. Whether the exploit is esoteric or not is by-the-by, it was something the developers hadn't thought of. It's been fixed and there is now a test for it.

      Next!

  3. Glen Turner 666

    What is cruft, what is security, and can the LibreSSL programmers tell the difference?

    @bazza This issue had been fixed in the original OpenSSL code. I think it is reasonable for people to look at this bug and to have a concern that in "decrufting" the code people may be removing features which are actually essential and are not cruft at all.

    1. Bronek Kozicki Silver badge

      Re: What is cruft, what is security, and can the LibreSSL programmers tell the difference?

      It's been "fixed", by seeding RNG with your private keys. Thank you for such "fix", I'd rather not see something like this in a program I am using. Especially since OpenSSL is unable to provide a guarantee or verify that RNG subsystem is indeed trustworthy one and won't steal your private keys.

      Remember, no library can fix security flaws of the underlying operating systems - at best it can work around them. If you want an illusion of security by deploying ever more elaborate workarounds that's your problem, but I'd rather leave it to people who know about security more than you do (or OpenSSL team does).

    2. future research

      Re: What is cruft, what is security, and can the LibreSSL programmers tell the difference?

      The LibreSSL guys knew that throwing away tons of cruft was going to introduce bugs and problems. They didn't really want to start this task, but they felt they had no choice. They are now going to be able to fix them in a sane secure manner now.

      1. Charlie Clark Silver badge

        Re: What is cruft, what is security, and can the LibreSSL programmers tell the difference?

        The LibreSSL guys knew that throwing away tons of cruft was going to introduce bugs and problems…

        Less code should mean fewer bugs. This looks like a test was missing: "untested code is broken code".

        1. Dan 55 Silver badge

          Re: What is cruft, what is security, and can the LibreSSL programmers tell the difference?

          OpenBSD's approach to porting is use plain OS functions and if the functions are broken on the target OS then it's the target OS which should be fixed, in contrast to OpenSSL's approach to porting which is to reinvent the wheel several hundred times. OpenBSD also said that they released the library to get testing and feedback.

          In this case there is a highly-commented Linux work-around for obtaining entropy showing exactly where the target OS has problems, what the work around is, and asking for it to be fixed. If it is finally fixed then the work around will be removed. Until that time the work around will be improved on.

          So what this guy did was read the Linux work-around comments and code and found the Linux work-around was not perfect. He could do this because most of the code is common to all platforms, apart from a few work arounds for each platform.

          In short, this was what was supposed to happen.

          1. elip

            Re: What is cruft, what is security, and can the LibreSSL programmers tell the difference?

            Thank you Dan 55, you get the cookie! This is the first correct assessment of the issue. Andrew himself states that the root cause of the problem lays with Linux in his original blog post the article mentions. Thankfully, we're all running a much more sane OS than Linux so we won't be hitting this issue either way, right guys? Enjoy your faux entroopy.

  4. Daniel Palmer

    The drama level in this article makes it initially seem the PRNG itself is broked.. which it isn't.

    The logic to tell the PRNG that it's in a new process and needs to flush it's state being broken is a lot less "oh noes the sky is falling in! won't somebody think of the security!" I guess.

    The process has worked: There was an issue, someone spotted it, the LibreSSL guys stated that it wasn't a big deal but they got on and fixed it. If things continue this way we might actually have a decent opensource TLS library eventually. Slightly off-topic; eglibc (a fork of glibc) has recently been dropped in favour of going back to glibc in Debian. Why? Because forking glibc and fixing a ton of crap that was wrong with it worked. glibc is now open to development and stupid historical mess due to one guy having a strangle hold on the project is getting cleaned up. LibreSSL could wipe out OpenSSL or OpenSSL could consume LibreSSL. Either way we should end up with something less crap and more maintainable.

  5. Anonymous Coward
    Anonymous Coward

    Portability is the problem

    It needs to be emphasized that it's the portable version, i.e. not the one the core developers are working on. After reading a couple of articles about random numbers in OpenBSD vs non-OpenBSD, this is going to be far from the last random number bug. Expect lots of programs to have flawed implementations due to dependency on EGD etc until someone (such as Linus and his counterparts in other kernel teams) gets angry and lets userspace get good-strength randomness directly from kernel via (non-deprecated) syscalls. Probably by making an ad-hoc standard that hopefully would be incorporated into posix.

    I expect this is going to be a very interesting year in cryptolibs, and loads of critical bugs found in both LibreSSL and OpenSSL. I cannot stress enough how happy it is to get a team focused on security, and not on implementing specs, working on THE core library for a connected world.

    1. A Non e-mouse Silver badge

      Re: Portability is the problem

      The Insane Coding blog recently had an article on the difficulties of dealing with randomness in *nix.

  6. Anonymous Coward
    Anonymous Coward

    Just a couple of thoughts from a complete non-expert in this area

    1. Though not in this case, being random the same number could occur twice or even more (randomly) often. Random does not mean unrepeatable.

    2. If a PID is recycled, there could be more dramatic and obvious side-effects. I did see and resolve this once on a Solaris system. The Oracle DB process was not at all happy when its PID was "hijacked". Mercifully, I forgot the details..

    1. Michael H.F. Wilkinson Silver badge

      Re: Just a couple of thoughts from a complete non-expert in this area

      The same number could well occur in a random sequence, but not the same sequence of numbers. In the old (and often still (ab)used) linear congruential PRNGs finding the same number would mean the exact same sequence would be generated again, because the entire state of the linear congruential generator is the last number generated. Modern PRNGs have states with much larger bit counts than the pseudo-random numbers they generate.

  7. Pet Peeve

    Openssl is infamous for making their own versions of common calls. If all they were doing was implementing missing functionality on a given platform, and only using the homegrown version on that platform (using a function like openssl_printf()) they would be fine, but openssl uses the homegrown version in ALL platforms, and they sneak extra functionality into them too.

    So, my guess is that the libressl folks yanked out the special version of fork in openssl, thinking there is no platform they care about where fork isn't in libc. However, openssl's fork was actually just a wrapper that called the normal fork, and then seeded the rng on the child process before exec-ing.

    Was this good practice on openssl's part? Hell no, they should have made the extra behavior explicit. But it illustrates the trouble that libressl is going to have getting rid of all those overridden functions - some of them do stuff you still need to do, or have extra semantics that the code itself will need to implement.

    1. Brewster's Angle Grinder Silver badge

      It's mark and sweep garbage collection on an entire codebase...

      1. Pet Peeve

        haha, no.

        Maybe if openssl was designed properly to extract platform support from the code, you could easily filter out out the dross. But that's the problem - openssl is an unholy mess in a lot of ways, and it is very likely that libressl will run into these problems over and over trying to figure out what's really junk (like support for platforms noboby's tried to deploy on in a decade and probably don't work if they did) and what isn't.

        The scary part is that this was an obvious problem. How long is it going to take to find all the broken edge cases that you don't know are broken until they happen? My understanding is that a lot of the openssl versions of libc calls had additional functionality, or returned not QUITE the same things the libc call did, or handled errors differently. All of those are going to bite libressl in the ass, eventually.

        I heard someone say the other day that if you see any linux distro with libressl in their stable branch, that is a distro to stay away from, for years at least, and possibly forever. Security is never proven, only demonstrated over time, and libressl's clock starts from zero. All of that said, the best of luck to them!

        1. Anonymous Dutch Coward

          @Pet Peeve

          @Pet Peeve

          Staying away from libressl for years... ok, so you'd suggest relying on OpenSSL's bloated unmaintainable code base (by their own admission). Looks like a dilemma...

          1. Pet Peeve

            Re: @Pet Peeve

            If the alternative is treating an experimental fork as stable? Yes! Read what I said before being a dick about it.

            Frankly, I would have preferred the libressl people started from scratch. They are a very knowledgable team, and I have more faith that they'd do a rewrite correctly than I do of them figuring out and removing all the dumb stuff in openssl. PV

    2. Dan 55 Silver badge

      OpenBSD's fork() re-seeds the child's PRNG, some other platforms' fork() don't. It did make sense to get rid of OpenSSL's fork() for OpenBSD, but it seems that the portable version overlooked an OpenBSD-equivalent version of fork() for those platforms that needed it.

      1. Pet Peeve

        Really?

        @Dan55: How would openBSD know WHICH prng to re-seed on fork?

        It would have to take a callback to the prng's library for that to happen, and I find it hard to believe that openbsd would add that to a core posix function like fork().

        The correct practice would be to have a function that does this work, which calls your prng's reseed function when a fork call returns 0 (meaning you're the child process).

        Edit: Ha, you don't mean it reseeds RAND, do you? That would be spectacularly pointless for crypto.

        1. Dan 55 Silver badge

          Re: Really?

          OpenBSD seem to have decided that they wouldn't want the chance of one thread following another thread's random numbers, and AFAIK that's been the behaviour for years.

          1. Pet Peeve

            Re: Really?

            I get that, but reseeding rand isn't going to help crypto, since no crypto package worth a dime would come within a lightyear of a linear congruential prng, especially one with only 32 bits of state.

            Even if openBSD happens to reseed rand (with what? The right answer would be 32 bits of /dev/random), this wouldn't do a bit of good for libressl.

            1. Pet Peeve

              Some additional info

              if openBSD does reseed a prng when fork() executes, it's undocumented behavior:

              http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/fork.2?&manpath=OpenBSD-current&arch=i386&query=fork

              The fork man page is pretty explicit about what it resets. You would think it would include this if it actually happened.

              1. Dan 55 Silver badge

                Re: Some additional info

                "The subsystem is re-seeded from the kernel random number subsystem using sysctl(3) on a regular basis, and also upon fork(2)."

                http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man3/arc4random.3?&manpath=OpenBSD-current&arch=i386

                1. Pet Peeve

                  Re: Some additional info

                  Interesting - does libressl change the default prng engine from ssleay (which openssl has carried around forever) to arc4random if no engine is chosen in the config? RC4 is fine for cyptographic randomness, the way openBSD manages it. Pretty neat, actually.

                  That said, if they did such an override in the openBSD version, it definitely would explain the behavior change in the portable version, or if some randomizer other than RC4 was selected in configuration.

                  1. Dan 55 Silver badge

                    Re: Some additional info

                    It's not configurable any more, LibreSSL only uses arc4random (incidentally that now uses ChaCha20 instead of RC4) as it's better than OpenSSL's generator and the other choice was FIPS which they're not bothered about.

                    OpenSSL had the same problem on forking, their fix for this was to allow the child to call a function afterwards which reseeds. That's if the programmer knew that the problem existed which didn't really sit well with OpenBSD's "it should just work" mentality and it doesn't protect against someone up to no good either either by not calling it or calling it too many times.

                    OpenBSD's arc4random takes care of the fork() problem automatically but the original workaround for LibreSSL portable was to check the PID which could fail if it wrapped back round and became the grandfather's PID, this new workaround traps fork().

                    Not that I know anything about cryptography but I'm following opensslrampage as it's a great way of learning about programming pitfalls.

  8. Anonymous Coward
    Facepalm

    As has been commented elsewhere on this issue, the furore kicked up by this event is rather dumbfounding.

    LibreSSL as I understand it, is OpenSSL currently stripped down to the bare essentials required for OpenBSD.

    They are fairly happy with what they have now *for OpenBSD* and have opened it up for re-porting by others. The fact that a Linux-specific issue has been found is principally due to the fact that LibreSSL is not currently ported properly to Linux. That there will be issues in this process is pretty much guaranteed, even if there were some issues that were originally "fixed" in OpenSSL.

    1. Pet Peeve

      Actually, using a pid change to indicate a reseed was necessary is a bad practice regardless of OS. Changing it to trap fork in a function was the right move.

      Also note - the libressl developers do NOT consider themselves ready for prime time in any OS. They would be the first to tell you that.

      1. Anonymous Coward
        Anonymous Coward

        > Also note - the libressl developers do NOT consider themselves ready for prime time in any OS. They would be the first to tell you that.

        Agreed. I guess my main point, echoed by you, is that the devs have made no claim about the readiness of LibreSSL for general use and this is doubly true for Linux.

        My understanding however, is that this PID test on fork is not required or used on OpenBSD anyway. I could be wrong on this point.

        1. Pet Peeve

          There is no way that the generic posix fork() is going to reseed a crypto package's prng. It could (and probably does) reseed the system random number function, but no crypto developer, sane or otherwise, uses that. See some of my messages above.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019