Vault Certificate Resolver Guide¶

Traefik Enterprise 2.3 and later supports using Vault with the PKI secrets engine enabled as a certificate resolver for automatic TLS certificate management.

This guide will walk you through the process of enabling Vault PKI for use with Traefik Enterprise on a Kubernetes cluster. It will also demonstrate how a certificate is automatically generated for an HTTPS IngressRoute with the certificate resolver enabled.

If you'd prefer a brief overview of how to configure Vault PKI in Traefik Enterprise, consult the reference guide.

Prerequisites¶

To complete this tutorial, you will need a few things:

A running Kubernetes cluster
The kubectl binary installed and configured to connect to the cluster
Traefik Enterprise installed on the cluster
Vault installed on the cluster

For purposes of this tutorial, you should make sure Vault is started in dev mode. Using the Helm chart:

helm install vault hashicorp/vault --set "server.dev.enabled=true"

Static Configuration¶

First, create the Vault PKI certificate resolver by adding the following to Traefik Enterprise's static configuration:

File (YAML)

certificatesResolvers:
  my-vault-pki:
    vault:
      url: "http://vault.default.svc.cluster.local:8200"
      auth:
        token: "root"
      role: "test-role"

File (TOML)

[certificatesResolvers.my-vault-pki.vault]
  url = "http://vault.default.svc.cluster.local:8200"
  role = "test-role"
  [certificatesResolvers.my-vault-pki.vault.auth]
    token = "root"

There are a few things to observe about this sample configuration:

It defines a certificate resolver called my-vault-pki.
The URL shown is the default URL and port where Vault is listening when it is installed on Kubernetes using the Helm chart (the recommended method). If your Vault server is listening on another URL, you should adjust accordingly.
The token root implies the Vault server is running in dev mode. If you wish to use a production server, replace this with your server's token.
The role test-role will be needed in later steps.

Configure Vault¶

Next, enable the Vault PKI feature on your server by entering the following from the CLI:

kubectl exec -it vault-0 -n default -- vault secrets enable pki
kubectl exec -it vault-0 -n default -- vault write pki/root/generate/internal common_name="VAULT PKI CERT" ttl=1000h
kubectl exec -it vault-0 -n default -- vault write pki/roles/test-role \
        allowed_domains=tls.example.com \
        allow_subdomains=true \
        allow_bare_domains=true \
        max_ttl=100h

The second command generates a root certificate with the common name VAULT PKI CERT and sets its time-to-live (TTL) to 1000 hours.

The third command configures the test-role role we specified earlier. First, it specifies our cluster's DNS address, tls.example.com, as an allowed domain. (Be sure to replace this with your own cluster's actual DNS address.) This parameter can also be a comma-separated list of addresses.

Support for subdomains and bare domains are enabled. We'll use subdomains to demonstrate the Valut PKI certificate resolver in action.

Finally, the command sets the maximum time-to-live (max_ttl) for the role at 100 hours. Note that at least one of ttl or max_ttl must be explicitly set here.

Time-to-Live (TTL) Management in Vault

In the example above, the role test-role will no longer be able to issue certificates once 900 hours have elapsed since the time the VAULT PKI CERT certificate was created. Although VAULT PKI CERT is specified as valid for 1000 hours, test-role has its max_ttl parameter set to 100 hours, which means Vault will stop issuing certificates using this role 100 hours before VAULT PKI CERT expires.

For more information on configuring Vault, consult its documentation.

Deploy a Demo App¶

Next, fire up an instance of our trusty, all-purpose demo app, whoami. The following code creates a whoami Namespace, a Deployment, a Service, and an IngressRoute to expose it on Traefik Enterprise:

apiVersion: v1
kind: Namespace
metadata:
  name: whoami
  namespace: whoami
---
# Pods
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: whoami
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: traefik/whoami
---
# Services
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
  labels:
    app: whoami
spec:
  type: ClusterIP
  ports:
    - port: 80
      name: whoami
  selector:
    app: whoami
---
## Expose the service on Traefik Enterprise
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: simpleingressroute
  namespace: whoami
spec:
  entryPoints:
    - https
  routes:
    - match: Host(`a.tls.example.com`)
      kind: Rule
      services:
        - name: whoami
          namespace: whoami
          port: 80
  tls: {}

The IngressRoute section is the interesting part. Notice that it creates a route to whoami from the a.tls.example.com subdomain and it enables TLS, but it does not specify a certificate resolver. When you access the service, you'll see the disappointing results. (Remember to replace the tls.example.com DNS name with your own.)

Command

curl -kv https://a.tls.example.com

Expected Output (Partial)

* Server certificate:
*  subject: CN=TRAEFIK DEFAULT CERT
*  start date: Oct 14 20:30:06 2020 GMT
*  expire date: Oct 14 20:30:06 2021 GMT
*  issuer: CN=TRAEFIK DEFAULT CERT
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fcff800d800)
> GET / HTTP/2
> Host: x.vault.containous.tech
> User-Agent: curl/7.64.1
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< content-type: text/plain; charset=utf-8
< date: Wed, 14 Oct 2020 20:30:12 GMT
< content-length: 400
< 
Hostname: whoami-57bcbf7487-7tn8b
IP: 127.0.0.1
IP: 10.244.0.31
RemoteAddr: 10.244.0.21:44406
GET / HTTP/1.1
Host: a.tls.example.com
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.240.0.6
X-Forwarded-Host: a.tls.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: default-proxy-66859ddc49-b5zpl
X-Real-Ip: 10.240.0.6

* Connection #0 to host a.tls.example.com left intact
* Closing connection 0

While our whoami service has responded to the request, there's a problem with TLS. The certificate is listed with TRAEFIK DEFAULT CERT as its issuer. That's because it's the standard, self-signed certificate that Traefik Enterprise issues whenever no other certificate is available.

Vault PKI in Action¶

Now set the Vault PKI certificate resolver to the task. Remembering to replace the DNS name with your own, the following configration creates a new route to the same whoami service – this time from b.tls.example.com – only now it specifies the my-vault-pki certificate resolver you defined earlier:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutetls
  namespace: whoami
spec:
  entryPoints:
    - https
  routes:
    - match: Host(`b.tls.example.com`)
      kind: Rule
      services:
        - name: whoami
          namespace: whoami
          port: 80
  tls:
    certResolver: my-vault-pki

Accessing this service shows a certificate with example.com as its issuer, indicating that Vault PKI has automatically generated a certificate for the b.tls.example.com subdomain. Had we supplied Vault with a proper, signed root certificate, the HTTPS connection would be fully secure.

Command

curl -kv https://b.tls.example.com

Expected Output (Partial)

* Server certificate:
*  subject: CN=b.tls.example.com
*  start date: Oct 14 20:31:39 2020 GMT
*  expire date: Oct 15 06:32:08 2020 GMT
*  issuer: CN=VAULT PKI CERT
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fa85d80d600)
> GET / HTTP/2
> Host: b.tls.example.com
> User-Agent: curl/7.64.1
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< content-type: text/plain; charset=utf-8
< date: Wed, 14 Oct 2020 20:32:28 GMT
< content-length: 400
< 
Hostname: whoami-57bcbf7487-7tn8b
IP: 127.0.0.1
IP: 10.244.0.31
RemoteAddr: 10.244.2.16:57290
GET / HTTP/1.1
Host: b.tls.example.com
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.240.0.5
X-Forwarded-Host: b.tls.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: default-proxy-66859ddc49-7rnx8
X-Real-Ip: 10.240.0.5

* Connection #0 to host b.tls.example.com left intact
* Closing connection 0

On the other hand, if you execute curl -kv https://a.tls.example.com again, you'll see that it's still serving the Traefik Enterprise default certificate. That's because the domain a.tls.example.com is not attached to the certificate resolver.

Generate a wildcard certificate

You can generate wildcard certificates using the Vault certificate resolver if you specify the domains option in your router. In the example below, a wildcard certificate will be generated for all tls.example.com subdomains.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingressroutetls
  namespace: whoami
spec:
  entryPoints:
    - https
  routes:
    - match: Host(`b.tls.example.com`)
      kind: Rule
      services:
        - name: whoami
          namespace: whoami
          port: 80
  tls:
    certResolver: my-vault-pki
    domains:
        - main: "*.tls.example.com"

Congratulations! You should now have a grasp of the basics of using Vault PKI as a certificate resolver for Traefik Enterprise. There are many other ways to configure Kubernetes IngressRoutes to take advantage of TLS and Vault PKI. Feel free to get creative.