Certbot and HTTP Public Key Pinning (HPKP)

When I wanted to enable HTTP Public Key Pinning for this site, I couldn't find a complete and still easy to understand guide for doing so in combination with Certbot (formerly known as Let's Encrypt). While it's perfectly possibly to do HPKP which Certbot, it does require some manual preparation and maintenance. Here's what I did.

The following steps were tested on a recent Gentoo Linux box with Certbot version 0.9.3 and should work on other platforms as well.

Installing Certbot

With Certbot you can easily add free SSL/TLS certificates to your website. Start by installing Certbot (you may have to add a couple of keywords to /etc/portage/package.keywords in order to get the most recent version of Certbot):

Raw code
emerge certbot

It is possible to use HPKP with Certbot / Let's Encrypt certificates as long as you accept the following restrictions:

  • You must use a fairly recent version of the Certbot client (I'm using version 0.9.3 for this documentation).
  • You cannot use Certbot's renew command line argument to create and update your certificates.
  • Instead, you will have to use certonly to renew your certificates.
  • The setup requires manual preparation and maintenance.

Create a first certificate for your domain

First, create a Certbot configuration file for your domain(s). I usually put them under /etc/letsencrypt/vhosts.d, but that's completely arbitrary. The config file should look like this (of course, please replace the domain name and email address):

Raw code
domains = tollwerk.de
webroot-path = /www/vhtdocs/tollwerk
 
rsa-key-size = 4096
email = info@tollwerk.de
text = True
authenticator = webroot
renew-by-default = true
agree-tos = true

As you see, I'm using the webroot plugin for acquiring the certificate. I'm not sure if this is relevant for the process, but I never tried any of the other plugins.

Don't forget to add additional domains and subdomains (comma separated, after the main domain) in case you need them and don't redirect them to your main domain. Also, make sure the webroot-path points to your publicly accessible web root.

Now issue your certificate by running:

Raw code
certbot -c /path/to/your/config/file certonly

Certbot should've created the following files (simplified for the sake of brevity):

Raw code
/etc/letsencrypt
|-- archive
|   `-- tollwerk.de
|       |-- cert1.pem
|       |-- chain1.pem
|       |-- fullchain1.pem
|       `-- privkey1.pem
`-- live
    `-- tollwerk.de
        |-- cert.pem -> ../../archive/tollwerk.de/cert1.pem
        |-- chain.pem -> ../../archive/tollwerk.de/chain1.pem
        |-- fullchain.pem -> ../../archive/tollwerk.de/fullchain1.pem
        `-- privkey.pem -> ../../archive/tollwerk.de/privkey1.pem

For security reasons, I advise you to make a backup copy of the /etc/letsencrypt/archive/tollwerk.de directory.

Get the fingerprint of your certificate

Use the commands given in Scott Helme's article to extract the fingerprint of your newly created certificate:

Raw code
openssl x509 -pubkey < /etc/letsencrypt/archive/tollwerk.de/cert1.pem | \
    openssl pkey -pubin -outform der | \
    openssl dgst -sha256 -binary | base64

The result should look similar to this:

Raw code
1XH16wMcwsBfOsMEb/ufcsR0NAtbK028XEWial43Kxc=

This is going to be the first part of of your HPKP header later on.

Optional: Locate and backup the original key and Certificat Signing Request

For whatever reasons, I usually make a backup copy of the key and Certificate Signing Request (CSR) Certbot creates while creating the first certificate. You should be able to find the CSR in the directory /etc/letsencrypt/csr. Just search for a file with the same timestamp as your certificate file /etc/letsencrypt/archive/tollwerk.de/cert1.pem — this should be your CSR. In my case, the CSR is named 0000_csr-certbot.pem but this could vary for you.

I keep all my CSRs in a central location, so I copy over (and rename) the original key and CSR:

Raw code
mkdir -p /etc/ssl/hpkp/tollwerk.de
cp /etc/letsencrypt/archive/tollwerk.de/privkey1.pem /etc/ssl/hpkp/tollwerk.de/tollwerk_de.first.key
cp /etc/letsencrypt/csr/0000_csr-certbot.pem /etc/ssl/hpkp/tollwerk.de/tollwerk_de.first.csr

Create backup keys & CSRs

Just to be sure, I create two backup keys, CSRs and their respective fingerprints following Scott Helme's guide:

Raw code
cd /etc/ssl/hpkp/tollwerk.de
 
openssl genrsa -out tollwerk_de.second.key 4096
openssl req -new -key tollwerk_de.second.key -sha256 -out tollwerk_de.second.csr
openssl req -pubkey < tollwerk_de.second.csr | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | base64
 
# IktlynCPgsYgaFU2bGDbmQQt+xB/e3pqiIWM08Wo6fE=
 
openssl genrsa -out tollwerk_de.third.key 4096
openssl req -new -key tollwerk_de.third.key -sha256 -out tollwerk_de.third.csr
openssl req -pubkey < tollwerk_de.third.csr | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | base64
 
# GAEdJjO4u7V/FzqxY0uN7SVfMppmk9vCWCwFQYQDXJo=

Configure Certbot for future certificate requests

The single important thing for Certbot is to use the original CSR for requesting follow-up certificates. You can configure it do so with the help of the relatively new csr option. Unfortunately, as soon as you use this option, Certbot also assumes that you want to take care of the certificate storage yourself (ahem ...). As a result, you'll have to give it some more paths — modify your Certbot configuration file like this:

Raw code
domains = tollwerk.de
webroot-path = /www/vhtdocs/tollwerk
 
csr = /etc/ssl/hpkp/tollwerk.de/tollwerk_de.first.csr
cert-path = /etc/letsencrypt/live/tollwerk.de/cert.pem
fullchain-path = /etc/letsencrypt/live/tollwerk.de/fullchain.pem
chain-path = /etc/letsencrypt/live/tollwerk.de/chain.pem
 
rsa-key-size = 4096
email = info@tollwerk.de
text = True
authenticator = webroot
renew-by-default = true
agree-tos = true

Also, Certbot will mess up the symlinks from the live to the archive directory in csr mode, so I recommend to

  • only use a symlink for the private key (which is not updated with every certificate request) and point it directly to your central key / CSR directory:
Raw code
cd /etc/letsencrypt/live/tollwerk.de/
rm privkey.pem
ln -s /etc/ssl/hpkp/tollwerk.de/tollwerk_de.first.key privkey.pem
  • manually remove the current certificates prior to each renewal process (see below).

Add the HPKP header to your website

Finally, add the HKPK header to your website. I'm using Apache here, so e.g. adding these lines to a .htaccess will do the trick:

Raw code
<IfModule mod_headers.c>
        Header append Public-Key-Pins 'pin-sha256="1XH16wMcwsBfOsMEb/ufcsR0NAtbK028XEWial43Kxc="; pin-sha256="IktlynCPgsYgaFU2bGDbmQQt+xB/e3pqiIWM08Wo6fE="; pin-sha256="GAEdJjO4u7V/FzqxY0uN7SVfMppmk9vCWCwFQYQDXJo="; max-age=300; includeSubdomains'
</IfModule>

Note that I start testing with a low max-age=300 value which should be increased to at least one week (604800) when all tests succeed. That's it! You should now be able to successfully test the validity of your pins using this testing tool.

Renewing your certificate

As mentioned earlier, you cannot use Certbot's handy renew switch when using a custom CSR. Instead, you have to request new certificates manually and shouldn't forget to delete the old certificates prior to that.

Raw code
cd /etc/letsencrypt/live/tollwerk.de/
rm cert.pem chain.pem fullchain.pem
certbot -c /path/to/your/config/file certonly

It should be fairly easy to automate this with a bash script in case you have more certificates to renew on a regular basis.

Please let me know if this works for you as well or if there's something I should improve. Thanks for reading!