back to article Great, you've moved your website or app to HTTPS. How do you test it? Here's a tool to make local TLS certs painless

A Google cryptoboffin is close to releasing a tool that will hopefully make all of us more secure online. Now that most web traffic travels over HTTPS and browser features increasingly expect security, developers really should be creating and testing apps in an HTTPS environment. Doing so requires installing a TLS/SSL …

  1. Anonymous Coward
    Anonymous Coward

    mkcert is a simple by design tool that hides all the arcane knowledge required to generate valid TLS certificates

    Great, so a few years down the line, nobody will have any idea how to troubleshoot issues with SSL certificates, as nobody will be taught "the arcane knowledge" anymore, just like nobody seems to be taught the basics of memory management nowadays.

    1. pavel.petrman Bronze badge

      It used to be like that - every developer indeed had to understand memory management, because it was involved in every development effort. Today it is not anymore so - even in languages and environments with explicit memory management (for example c++), there is rarely need to venture to the memory management territory for most developers (there are tools for that, and increasingly, advanced language or standard library tooling as well). And I for one find it a good thing - more useful programs can be done in less time because people can concentrate on productivity and understanding the usage, instead of being busy with the hardware idiosyncrasies, or worse (and very common in the olden days) burdening the user with them.

      My pet theory, albeit not underlined with substantial rigorous sources, is that the number of people who need and indeed do memory management programming is constant in time and the understanding doesn't go away, as is so often feared. Only now these people don't do the GUI and the whole works, they do the memory management only and more often than not do it well. It is a bit overstretched, but I still like to compare this sigh to a grampa few thousand years ago saying "these youths have their house built, how are they going to live in it when they haven't built it themselves! In a generation or two, houses will start falling on people's heads, take my word for it."

      1. big_D Silver badge

        Optimization, whether it is for the processor, memory or the database is still very important. And so is making certs - and making your own certs on Linux is a doddle, it is hardly arcane knowledge, any half decent admin or developer should be able to do it in their sleep; at my last job, a security company, we had to set up our own personal, email and internal website certs all the time, the process was well documented and took a couple of minutes.

        There are many examples of why memory management and optimization is important.

        When Mozilla tried to release their first mobile browser, they complained that the memory on the devices was constrained and the OS developers and the device manufacturers should retro fit more memory and allow more memory per app on those devices, so they could get Firefox working! Not, lets look at how we make Firefox efficient, no, the OS and the hardware is at fault, because we can't have as much memory as we want... :-S

        On one project, an online shop, every time the PayPal newsletter came out and the shop was mentioned, the whole thing collapsed and the DB Admin would have to restart the database service every couple of minutes.

        The problem turned out to be that the devs knew how to code, making it look pretty and human readable, but they didn't know sweet Fanny Adams about optimization. I looked at the PHP code and the queries, re-ordered the query to work from least to most, instead of what was more readable for the developer. I also optimized the PHP code to be more logical, testing positives and testing the most common things first.

        The result was that when the next PayPal newsletter came out, the database didn't cease up (before it was taking over 60 seconds per visitor / page impression to retrieve the menu under load, with the optimization it was down to around 300 milliseconds) and the database server plus 4 front end servers went from collapsing with 200 simulatenous visitors across the 4 servers to handling over 250 per server and still not being stressed.

        After that, I had to give a workshop for the dev teams (the company ran webshops and site for many clients) to teach them about optimization techniques and thinking in terms of data.

        The code can be functional and elegant, but that doesn't stop it being a pile of steaming doggie do-dos and using far too many resources. Knowing how to program is a lot different to being able to stick elegant code together.

        1. ds6 Bronze badge
          Windows

          I think the real problem isn't that we're making things easier for ourselves—the current generation that knows how these things work, it's that we're making things easier for the next generation of programmers that hasn't learned what we know.

          Sure, it would be neat to have a one-liner to generate a cert for everything, as opposed to fuddling around with openssh parameters, but it's not a big deal that we—those that already know and understand how OpenSSH works—make it simpler for ourselves, because we understand the implications of what we are doing, and know how much control we are giving up by using a higher-level solution. This of course applies to everything from memory management to database queries and more.

          But here comes along Billy Big School that just got their 2-year degree in Really Quite Basic Programming that only taught them high level languages in a Windows environment with Visual Studio, and now they think they're cool and go apply for an entry-level, underqualified web dev position because "I write JScript in my free time PLUS I went to school for it" they say, and now they suddenly need to know how to generate certs from the command line to get the test environment started, something they know nothing about and have never done.

          In enters mkcert, giving them an effortless way to make certs! How does it work? What is even a cert? Who cares, I can just run it and put a file here and suddenly the test environment works in Chrome! Hurray!

          When things get easy enough that even a trained monkey can do it, it allows for the untrained human to slough their way through without really understanding the implications of what they are doing. That's the real problem we're facing.

          To apply this logic to the memory management/DB examples we've seen, it's obvious those PHP developers were learned in how PHP and databases worked, but did not understand how the commands affected the underlying memory model or how much IO they clogged the database with.

          No, I don't have a solution for this. One can't just say "have better schooling!" or "enforce testing of all employees for basic knowledge!" because both are terribly unrealistic in the education and corporate worlds respectively. Even simply "leave things complicated enough to require at least intermediate knowledge on the subject!" doesn't work anymore, because one can just pop on over to SO and copypaste a sexy snippet and be on their merry way.

          Teaching the fundimentals as building blocks is very important, because it leads to a deeper understanding of the actual functionality of the technology we use daily, high-level or otherwise. Don't do that and you get a whole host of people that will eventually learn on the job that they have no real idea what they or their code is doing from a lower-level perspective.

    2. Michael Wojcik Silver badge

      There are already plenty of toolkits and tutorials for creating test CAs and issuing certificates. That hasn't measurably changed the number of people with some understanding of the ghastly, shambling horror that is the X.509v3 PKIX (i.e., few); or the number of people who understand it well (a handful).

      Valsorda is a good researcher, but frankly I don't think there's much to get excited about here. Anyone who's likely to use mkcert could probably have gotten by just fine with one of the hundreds of online tutorials using "openssl ca". True masochists could fire up Windows Server Cryptography Services instead.

      People who don't test their TLS apps with certificate chains skip it because of the cognitive load, not because there aren't relatively easy tools. (Yeah, openssl ca isn't user-friendly, but it's a walk in the park compared to actually understanding the X.509 PKIX in any depth. Look at the problems caused by OpenSSL's enforcement of RFC 5280 timestamp rules - those were due to other TLS implementations generating non-conforming certificates.)

  2. Zilla

    Lol arcane knowledge of memory management

    Nobody ever taught memory management to anyone.

    As the tools get better, we can spend less time faffing around and more time getting things working.

    1. Blane Bramble

      Re: Lol arcane knowledge of memory management

      @Zilla "Nobody ever taught memory management to anyone."

      Maybe not to your generation.

      1. big_D Silver badge

        Re: Lol arcane knowledge of memory management

        And it is my generation that then has to go along and dig them out of their hole, once their apps have swamped all available resources, but, hey, the code looks elegant!

        1. ds6 Bronze badge
          Joke

          Re: Lol arcane knowledge of memory management

          If it looks pretty, it goes faster. Duh.

          It's just like racing stripes. Don't you know how racing stripes work?

    2. HieronymusBloggs

      Re: Lol arcane knowledge of memory management

      "Nobody ever taught memory management to anyone."

      You appear to be suffering from faulty memory management.

    3. HieronymusBloggs

      Re: Lol arcane knowledge of memory management

      "As the tools get better"

      Those tools don't write themselves.

      1. Phil Endecott Silver badge

        Re: Lol arcane knowledge of memory management

        "As the tools get better"

        I read that as “As the trolls get better”.

    4. overunder Bronze badge

      Re: Lol arcane knowledge of memory management

      WTF? If you attended a school that didn't teach memory management get your money back! If you still don't understand it, go read something on object pools or stack/heap in general.

      I know the University of Javascript or whatever pushes idiocy and "Make it new, rename it." * , but you still want to know the established necessities.

      * Browse random npm's for node.js, it's laughable... but I still like V8 :-p.

      1. doublelayer Silver badge

        Re: Lol arcane knowledge of memory management

        Memory management still needs to be known by most software devs. However, I think that will also decrease in the coming years without much harm. I know quite a bit about it; I taught it for some time. Let's consider other things I taught in the same course. One of them was caching locality. Interesting thing, that. However, when they implemented a system that was supposed to provide increased speed through keeping data close together for optimized caching, it actually ran slower. This had worked a few years before, but it is different now because processors became better at caching things, and the better solution became the one with the least overhead. You could view this as a problem, but the result was that less effort had to be spent on getting the full benefits of the cache. Some people still have to know about the architecture of the cache, either to really intensify performance or to build a better cache, but your standard dev could spend time on the functionality of the software without making things less organized and more fragile to get better performance.

        1. overunder Bronze badge

          Re: Lol arcane knowledge of memory management

          But is the lesson actually about the hardware memory or just memory in general? That is why I specifically suggest object pools. But here's the strange thing, apparently that is becoming too "difficult" to learn or teach. So you now need a M.I.T. degree to learn such practices? Bullshit. Once you skip over memory management, you're completely dependent on some garbage collector. And once you're dependent on that, the cuffs are off and your wasting all kinds of things you don't even know you're wasting.

          It may come down to laziness or lack of mathematical skills, but 1 thing is for sure, the current teaching practices are becoming lazy and non-mathema. You might want to get back to teaching in some form you're now comfortable with.

    5. Ilsa Loving

      Re: Lol arcane knowledge of memory management

      And you know what? It really shows. Developers today have absolutely zero concept or concern about just how much memory they're wasting, which is why you end up with chat apps that require a gigabyte of memory to run.

  3. LDS Silver badge

    Late to the party?

    Windows SDK had a makecert utility for ages - and if you setup a Windows CA generating certificates through the web interface or the certificate management snap-in is even easier - and you can test using CRLs and OSCP as well.

    Anyway I've been using XCA for years (https://sourceforge.net/projects/xca/, it's built upon OpenSSL) which makes very simple to generate a whole PKI (including intermediate CAs and CRLs), manage and export/import certificates in different formats.

    Then if you're in Windows you should use the proper OS certificate store - not always the system one - in *nixes you have to learn where each application stores certificates... had a some fun lately with pfSense (FreeBSD) which lets you upload your certificates to its own store, but then for SMTP and TLS uses one of the system stores....

    1. Privatelyjeff

      Re: Late to the party?

      That’s what I’ve been using too. It’s fairly easy to use and you don’t have to renew stuff like you do with LE.

      1. LDS Silver badge

        Re: Late to the party?

        LE is designed for production sites which don't want or cannot afford commercial certificates. Given the way it verifies site owners, it's OK its certs expire often. In a test environment you may want longer life, plus being able to test with invalid, expired and revoked ones - which requires a far broader control on generated certificates. You may also want to use more complex certificate chains and specific certificate types, plus better control on certificate data - i.e. how subject alternative names and key usages are set up may matter a lot if you're serious about using certificates, and not just accepting everything thrown at you (as some HTTPS libraries did until not long ago...)

        Certificates are more than a simple private/public key part - creating and using them correctly is important, or for compatibility reasons applications will end to have to accept almost everything, and that will downgrade security.

        Here I believe a decent GUI application may also act as a "tutorial" showing what options are available and encouraging to understand what they are used for. Simple command line utilities will make most users just copy&past from StackOverflow, without understanding what they're doing and why, and if it's what they really need.

        1. Radelix

          Re: Late to the party?

          I regret that I have but one upvote to give you. As someone whom has to "do it all" so to speak, sometimes I do need a bit of handholding to accomplish a task correctly.

  4. suburbazine

    That feature photo

    Is an awesome conglomeration of incompatible products. Also, very in keeping with certificate setup.

    1. ds6 Bronze badge
      Unhappy

      Re: That feature photo

      First thing I thought was "they're going to ruin that CPU".

  5. Steve Graham

    Funny, I was just trying this the other day on my home server (no external access) -- the old-fashioned way. I got it all working too, except I couldn't find how to get Vivaldi, my default browser, to accept a self-signed certificate. I think it's actually missing that functionality. I lost interest at that point and put it all back the way it was. I think a CA might be the fix, so I might give this tool a go.

    1. LDS Silver badge

      Did you import your certificate into Vivaldi, or whatever store it uses? A self-signed certificate is the CA of itself (actually, all root CAs are self-signed), so the browser can't use any other CA to verify it. You have to install it both on the server and the browser. If you use a real CA, and then certificates signed by that CA, you'll need to import only the CA, and all (valid) certificates will be trusted.

      1. ds6 Bronze badge

        Vivaldi is a Chrome fork so it has all the same features. They're likely using an OS where the certificates are not handled by the browser, as it usually is.

  6. GnuTzu Silver badge

    minica

    I found minica to be a little too minimal. It was a good starting place, but there are just things about OpenSSL configurations that are just too weakly documented, which of course is why minica and other projects exist.

  7. Martin
    Unhappy

    But why is it so complicated?

    But this should be simple, and not require arcane knowledge, surely?

    The problem I have found is that there seems to be a bit of an assumption that you know roughly what a certificate is, understand how it works, and how to get one before you can even start. But, frankly, I barely understand even that. I just know they work, somehow, and make everything secure. That's all I want or care about.

    Case in point - I have the unify software running to look after three local ubiquiti unifi wireless access points in the house. It works with an https browser interface.

    I only want to get onto the broswer interface from my local network, not from anywhere else. So I'm just ignoring the warnings and going through to it. It's unsatisfactory, but there doesn't seem to be a bog-standard, simple way of getting a self-signed certificate so that I don't have to get the warning.

    So I'll be looking at this, to see if I can make it out.

    .

    1. LDS Silver badge

      Re: But why is it so complicated?

      The issue arises when you are developing and testing an application that will have to work in a much more complex and riskier environment. You have to check not only that it encrypts traffic somehow, but it has to be able to validate certificates, spot issue. and communicate errors properly so they can be fixed without long troubleshooting. The scenario is not as simple as some home APs.

      For example an application we worked on a year ago had several clients communicating with servers over TLS (one channel were message queues, not only HTTPS), and a web interface accessible with HTTPS for management as well. Clients used certificates for authentication also. We had a CA, two intermediate CAs (one for the servers, the other for the clients), and all the required certificates with the proper settings - each certificate type was different.

      Certificate revocation checks happened on both sides, and client certificates renewals had to be managed also. We couldn't ignore any warning or other issue.

      We had to setup a whole test PKI, and been able to test what happened when something went wrong.

      You need something to help you create and manage all that stuff.

    2. HieronymusBloggs

      Re: But why is it so complicated?

      "I just know they work, somehow, and make everything secure."

      They don't make everything secure. They just make things a bit more secure than not having them. If you're going through a company proxy or public access point equipped with MITM software (eg. Bluecoat) the security is broken. This seems to get swept under the carpet in the current "HTTPS everywhere" craze. HTTPS is good, but don't be lulled into a false sense of security.

      1. doublelayer Silver badge

        Re: But why is it so complicated?

        HTTPS can be circumvented by some MITM systems, but not by others. A company proxy likely has that built into it, but that's because the company can control which devices are connected to it and configure them to trust the proxy and to allow http traffic to be left unencrypted until the proxy. A MITM system on a standard network can't necessarily do that to you, because the browser will inform you that the traffic is now in insecure HTTP, that is if it doesn't complain more vociferously which some will do. So HTTPS is still useful against many attacks.

      2. LDS Silver badge

        Re: But why is it so complicated?

        Usually MITM HTTPS requires a specific CA to be installed on your system so it can fake certificates. In a company environment that quite easy because the CA(s) may be deployed automatically. Otherwise they'll have to make you install their certificate.

        You will still spot the issue if you look at the certificate.

    3. ds6 Bronze badge
      IT Angle

      Re: But why is it so complicated?

      But, frankly, I barely understand even that. I just know they work, somehow[...]. That's all I want or care about.

      This kind of attitude is terrible and hurts not only yourself, but others. If you ever have to set up an Internet-facing service—even if you don't know it is—your lack of knowledge will now open you up to being attacked, spied upon, information stolen, the device in question put on a botnet... And this only encourages those with malintent to keep doing what they're doing because people keep on making the same basic mistakes that allows them (users and miscreants) to continue operating.

      "I don't care as long as it works" is why grannies everywhere buy trash Chinese routers with 0% security features enabled so that her grankids can get on the dang ol' Internet, and suddenly it comes as a big surprise when granny gets busted for downloading pirated movie rips and child pornography when it was really a distant neighbor, or some miscreant using her Mirai-vulnerable device as a proxy.

      "I don't understand it but it works" is why we have shoddily-coded websites, databases, and other pieces of critical infastructure being breached seemingly on the daily and billions of people's information being sold on back-alley channels. As another example, let's say you think you know how something works and try to give advice to your friend—now you're both spreading FUD about something you truthfully know little to nothing about.

      Please. Educate yourself. For everyone's sake.

      1. Martin
        Happy

        Re: But why is it so complicated?

        This kind of attitude is terrible and hurts not only yourself, but others.

        ...

        Please. Educate yourself. For everyone's sake.

        Fair comment. I accept the rebuke.

        But I'm now retired. This is no longer my day job, and it feels to me as if it's a lot of faff for something that is merely an irritation to me - I don't need and will never need to be able to connect to my network support software from outside my network.

        I do the best I can in the time I want to set aside to keep things reasonably secure in my home network. But if I need to take a few days to educate myself about certificates, I sort of feel I'd rather be playing my guitar, or going on a trip to see my grandchildren, or doing all the other things that I now have time to do.

        1. ds6 Bronze badge
          Boffin

          Re: But why is it so complicated?

          Alright mister "I can pinch my grandkid's cheeks while simultaneously playing guitar on vacation in the Alps", have at you. Welcome to Certs 101—or How I Learned to Stop Worrying and Love OpenSSH

          Certificates are cryptographically secure chains (terminology: the chain of trust) that are able to verify eachother through a web of cryptographic keys and signatures. Just like you don't need to know polynomials to know algebra exists and works, you don't need to know the ins and outs of cryptography to use certificates, and it really is quite straightforward.

          Firstly, it is necessary to mention that there are usually 3 separate parts to a valid certificate:

          1. The private key;

          2. The public key;

          3. The signature.

          There are other varyingly relevant data points (including expiry date) but this is all you need to know to fathom the idea of certificates. What people call a "certificate" or "cert" is usually a file containing the public key and the signature, as well as various related (and cryptographically secure) metadata. On Windows, they appear as *.crt files, and you might have seen them as .pem on *nix. The private key is kept separate, for reasons outlined below.

          A cert's signature is generated using a second cert's private key. The private key should never be shared, ensuring the person that originally created the second cert can sign the first. The public key is used to verify signatures created with its sister private key, but cannot be used to sign new signatures. This process makes it easy to verify the validity and trust of a signature, while making it highly improbable to fake one.

          Since we need a private key to sign a certificate in the first place, we can create self-signed certs, also known as root certificates. These root certs can be used to sign other certs down the line, creating the chain of trust. But if the certs in the chain all come back to this root certificate, how can we trust the root, thus giving credulence to the chain?

          A CA, or Certificate Authority, is an entity that is responsible for widely-trusted root certs. They create root certs and provide tools to have other people's certs signed by them. We choose to trust CAs and their root certs because they put their reputation as a company and policies behind the certs they provide. If you look through a certificate viewer eg. in your web browser, you'll see all certificates on your computer trickle back up to root certs provided by Microsoft, Google, Verisign, GoDaddy, Symantec, etc. Every operating system and/or web browser has its own list of root certs that it trusts, and these certs are usually provided along with the software.

          That's it, really. Not too complicated right? Now we can get to more practical uses of certificates.

          The reason why your browser is giving errors when you try to connect over HTTPS to your control panel is likely because of any of these reasons:

          1. The certificate is expired;

          2. The certificate is self-signed, and your computer/web browser does not have the cert in its list of trusted root certs;

          3. There's no certificate at all.

          A solution that would work fine for you is to:

          1. Create your own cert;

          2. Install it in the server;

          2. Install it in your browser/OS's trusted cert store.

          You can easily accomplish this with many methods, and I will provide one for OpenSSH (works on any platform). Contrary to popular belief you do not need to pay a CA to get a certificate and can easily self-sign your own. A self-signed cert of your own making is not any less safe than a CA-signed cert, and depending on your viewpoint may be more safe since you probably trust yourself more than some faceless company :)

          For the creation of certificates, aside from the private key, there is also the certificate signing request which is a simple file outlining some of the extra metadata that is used to generate an X.509 cert, which is the standardized format for the web. Ever see the company name next to the green padlock in your web browser? That's some of the metadata in the cert that the CSR is used to generate.

          You can easily create a private key, CSR, and certificate in order using the following oneliner:

          openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 1096

          key.pem is the private key that was used to sign the cert. cert.pem is the public key, signature, and metadata, in one file. You will be prompted for various pieces of information to fill out the CSR after running the command. The CSR is consumed in the process here; it's optional to save it since you likely won't need it in this instance.

          The only thing you need to worry about in the CSR is the Common Name, which should be the address you are accessing the web service through. The Common Name is how your web browser knows it's using the right cert for the right site. If you access it via IP, since you are only using it locally, it is safe (though not recommended) to use local IPs for the CN, eg. 192.168.50.1.

          As for installing the cert in the system/OS's trusted store, if you visit the site with the new cert installed in the web server, on a modern browser it should ask you if you want to allow the cert, and if you want to store the changes permanently. This method should work on all major operating systems and browsers and should be all you need to do. Alternatively, if you don't want to have to repeat this process for all the browsers you may use, while mildly outdated here's the easiest way to install a certificate into Windows' trusted store: https://blogs.technet.microsoft.com/sbs/2008/05/08/installing-a-self-signed-certificate-as-a-trusted-root-ca-in-windows-vista/

          I don't know how your software works so I won't comment on how to get the cert in there.

          (Most of this came from memory so I apologize if I was wrong or misleading somewhere.)

          Finally, yes, it doesn't really matter if you have a valid cert on your local network or not, but denying the information just because it might not be immediately relevant is not a very good practice to keep. Even if you don't go through with setting things up properly, let this at least be a lesson into certs and how really not confusing they are. Things can get crazy with OCSP stapling and other additive technologies, but those are outside of your use-case and you likely will never need to care. The basics, however, are still important. Even knowing the basics can give a lot of insight into what might otherwise be considered ungrokable geekspeak, and with how widespread HTTPS is, that makes it even more inportant to know.

          Certs 201 is next week and I expect to see you there young man.

    4. Anonymous Coward
      Anonymous Coward

      Re: But why is it so complicated?

      I think that this may be answered by a simple Youtube video around the 16 minute mark:

      https://www.youtube.com/watch?v=fwcl17Q0bpk

    5. Michael Wojcik Silver badge

      Re: But why is it so complicated?

      But this should be simple, and not require arcane knowledge, surely?

      No. It is not, and does.

      And that's just one collection of problems with the X.509 PKIX. There are others, and since that one was written, we've introduced a whole bunch of additional things to worry about, like algorithm deprecation, OCSP Stapling, and Certificate Transparency.

  8. Borg.King

    Scripted

    I've had this whole shebang scripted on Windows Server for several months now. I even have an nginx proxy that's properly certified too. This is hardly a difficult thing to do, and once you've learnt the underpinnings (about three days with trial and error), it's good knowledge to have.

    Just make sure you create a certificate with a subject alternate name defined to please Chrome.

  9. Frozit

    2 years ago it became apparent that my software company didn't understand certs. There was even an internal meme for it.

    So, set up an internal CA, ran a couple courses, forced everybody to request certs for their test servers, etc.

    It worked, certs are no longer an issue.

    Eventually ran into the SAN cert issue with Windows Server 2012 R2 not requesting SAN by default, so ended up making a wild card get out of jail free cert. Which kinda defeated the whole purpose.

    However, certs are no longer a meme, or a support issue, so win!

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019