Certificates

This page describes the two ways provided by the h8lio platform to manage the TLS certificates to secure your ingress routes (https scheme) using:

• Traefik Proxy - quickest setup, no extra resource to manage
• Cert-Manager - recommended for production domains, full control and reliable issuance

Which one should I use?

For a quick test or a handful of routes, the Traefik default resolver works out of the box. For production custom domains (especially if you script the creation of many routes), use the platform cluster issuers: issuance is faster, observable, and immune to the rate-limit pitfalls described below.

Traefik

Traefik integrates the TLS certificates management and we are providing a default certificate resolver using LetsEncrypt TLS-ALPN-01 Challenge which can be used in your routes.

🟢 Pros:

• Zero configuration - transparently managed by the Traefik Proxy, no extra resource to create or manage

🔴 Cons:

• Little control - certificates are internally managed by Traefik, with no observable status or events
• Unpredictable issuance - the ingress layer runs multiple instances, each with its own ACME state: a new certificate may take several attempts before being served consistently
• Rate limits exposure - LetsEncrypt limits (5 certificates / 7 days for the same exact set of domains) can be reached quickly when creating many routes at once or when a misconfigured route makes the challenge fail repeatedly (see Best Practices)
• No wildcard certificates

Usage

Here is an example how to use the default certificate resolver within a Traefik IngressRoute. This example also shows you how to redirect the HTTP trafic to the HTTPS entry point using a Middleware:

# http: scheme route redirected to https: scheme.
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: example
spec:
  # http entry point
  entryPoints:
  - http
  # route rules definition
  routes:
  # list of rules
  - kind: Rule
    match: Host(`test.example.com`)
    # redirect middleware
    middlewares:
    - name: https-redirect
      namespace: traefik
    # backend Kubernetes service
    services:
    - name: my-service
      port: 8080
---
# secured route (https scheme) with "default" certificate resolver
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: example-tls
spec:
  # https entry point
  entryPoints:
  - https
  routes:
  - kind: Rule
    match: Host(`test.example.com`)
    # backend Kubernetes service
    services:
      - name: my-service
        port: 8080
  tls:
    # TLS certificate resolver
    certResolver: default

That’s it! Traefik is doing all the certificate generation and renewal processes internally.

Best practices

To facilitate the TLS challenge and avoid the LetsEncrypt rate limits you should follow the following recommendations:

• (mandatory) If you are using your own domain, make sure the routes are redirected to edge.h8l.io (CNAME record to edge.h8l.io or A record to the load balanced IP of edge.h8l.io) and the DNS propagation has been done before deploying the secured route (LetsEncrypt could perform the challenge from the US)
• If your route rules are complex, you can help Traefik to resolve the hosts to challenge by specifying the tls domains
• You can group your routes in a single IngressRoute and/or in a single match: with Host(`a.b.com`,...,`c.b.com`) to help Traefik resolve the hosts and limit the number of challenges.
• If a domain hits the rate limit (5 certificates / 7 days for the same exact set of domains), the only options are to wait for the sliding window to free a slot, or to switch the route to a cert-manager certificate so the next available slot is used efficiently.

Cert-Manager

The h8lio integrates cert-manager to manage your certificates. It allows to get more control of your certificates and share them with multiple Kubernetes resources.

Note

The platform cluster issuers are the recommended way to secure production custom domains. Every feedback is welcome at hello@h8l.io.

🟢 Pros:

• Ready-to-use issuers - the platform cluster issuers require no configuration and issue certificates in seconds
• Full control - finely manage your certificates, or provide your own issuers and self-signed certificates
• Observable - Ready status, events, and the cert-manager command line tool
• Wildcard certificates - using the LetsEncrypt DNS-01 challenge with your own issuer, which also avoids the LetsEncrypt rate limits when creating a lot of secured routes (in case of migration for example)
• Easy integration - a single tls.secretName reference in your Traefik IngressRoutes

🔴 Cons:

• One extra resource - a Certificate to manage per domain, compared to the integrated Traefik certificate resolver
• You manage your certificates - lifecycle ownership moves to you
• DNS-01 provider management - only when you bring your own issuer for wildcard certificates

Platform cluster issuers (Let’s Encrypt HTTP-01)

The platform provides two ready-to-use cluster-wide issuers. They can be referenced from any of your clusters, without any configuration or credential on your side:

ClusterIssuer	Purpose
letsencrypt-http01	Production certificates, trusted by browsers
letsencrypt-http01-staging	Pipeline testing only: certificates are not trusted by browsers and do not consume the LetsEncrypt production rate limits

Prerequisites:

• the domain must resolve to the platform (CNAME record to edge.h8l.io or A record to its load balanced IP) and the DNS propagation must be done before creating the Certificate (the validation is performed over HTTP on your domain)
• no wildcard certificates (HTTP-01 limitation): for wildcards, bring your own DNS-01 issuer

1. Create a Certificate for your domain:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-com                     # convention: the domain slug
spec:
  dnsNames:
    - example.com                       # apex domain
    - www.example.com                   # additional names if needed
  issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    name: letsencrypt-http01
  secretName: example-com-cert          # the secret which will hold the certificate

2. Follow the issuance (usually under a minute):

# READY=True when the certificate has been issued
kubectl get certificate example-com
# events detail in case of trouble (rate limit, DNS not propagated...)
kubectl describe certificate example-com
# pending validations
kubectl get challenges,orders

3. Reference the secret in your IngressRoute (instead of tls.certResolver):

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: example-tls
spec:
  entryPoints:
  - https
  routes:
  - kind: Rule
    match: Host(`example.com`) || Host(`www.example.com`)
    services:
      - name: my-service
        port: 8080
  tls:
    # TLS certificate secret reference (replaces certResolver: default)
    secretName: example-com-cert

Note

Renewal is fully automatic (about 30 days before expiry). The http (redirect) routes are not impacted: only the tls: block of the secured route changes.

Migrating an existing route

To migrate an existing route from certResolver: default: create the Certificate first, wait for READY=True, then replace the tls: block. The current certificate keeps being served until the switch, so there is no interruption.

Bring your own issuer (DNS-01, wildcard certificates)

Note

For now, we deployed only the OVH Challenge Provider, if you need another supported provider or add your own provider let us know by opening a ticket or by emailed us at hello@h8l.io.

The following example shows how to create and use a wildcard certificate using cert-manager and LetsEncrypt DNS-01 challenge on a OVH managed domain.

1. Create a secret holding the OVH API credentials in the cluster where you want to use the certificate:

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: ovh-<domain>-api
data:
  applicationKey: <base64 encoded application key>
  applicationSecret: <base64 encoded application secret>
  consumerKey: <base64 encoded consumer key>

replace the <...> by the value of your choice

Tip

you can use sealed-secret if you want to securely store the secret outside of the h8lio Kubernetes Cluster (example, in a Git repository)

2. Create your cert-manager Issuer using LetsEncrypt and the OVH challenge provider:

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: ovh-<domain>
spec:
  acme:
    # LetsEncrypt production server
    server: "https://acme-v02.api.letsencrypt.org/directory"
    # email or mailing list used for the LetsEncrypt account
    email: <my@email.com>
    # reference to the secret holding your LetsEncrypt account key
    # it will be created by cert-manager if it doesn't exist
    privateKeySecretRef:
      name: "ovh-<domain>-account-key"
    solvers:
      # DNS-01 challenge configuration
      - dns01:
          cnameStrategy: "None"
          # OVH DNS-01 challenge webhook configuration
          webhook:
            solverName: ovh                    # do not change this value
            groupName: acme.h8l.io             # do not change this value
            config:
              # OVH zone
              endpoint: ovh-eu
              # OVH Authentication method (possible values: application or oauth2)
              authenticationMethod: application
              # OVH credentials from the secret
              # adapts the name: and key: based of your configuration
              applicationKeyRef:
                name: ovh-<domain>-api
                key: "applicationKey"
              applicationSecretRef:
                name: ovh-<domain>-api
                key: "applicationSecret"
              applicationConsumerKeyRef:
                name: ovh-<domain>-api
                key: "consumerKey"

see webhook documentation configuration for more details

Once the Issuer manifest has been applied to your cluster. You can check the events and status: of the cert-manager.io/v1 Issuer CustomResourceDefinition to validate its configuration and its readiness. It should look like:

status:
  acme:
    lastPrivateKeyHash: yTty...
    lastRegisteredEmail: my@email.com
    uri: https://acme-v02.api.letsencrypt.org/acme/acct/12345...
  conditions:
    - lastTransitionTime: '2023-08-05T13:58:40Z'
      message: The ACME account was registered with the ACME server
      observedGeneration: 2
      reason: ACMEAccountRegistered
      status: 'True'
      type: Ready

3. Create the cert-manager Certificate’s secret using the previous Issuer which will be used in the IngressRoutes

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: ovh-<domain>
spec:
  # replace with your domain
  # it could be a second level subdomain (ie. "*.sub.domain.com")
  dnsNames:
    - "*.domain.com"
  issuerRef:
    group: cert-manager.io
    kind: Issuer
    name: ovh-<domain>                # the issuer name
  secretName: ovh-<domain>-cert       # the name of the secret which will hold the certificate

Once the Certificate manifest has been applied to your cluster. You can check the events and status: of the cert-manager.io/v1 Certificate CustomResourceDefinition to validate its configuration and its readiness.

This operation could take few seconds while cert-manager is internally managing the creation of all the resources involved in the DNS-01 challenge process to create the secret containing the new Certificate. It should look like:

status:
  conditions:
    - lastTransitionTime: '2023-08-07T07:44:14Z'
      message: Certificate is up to date and has not expired
      observedGeneration: 1
      reason: Ready
      status: 'True'
      type: Ready
  notAfter: '2023-11-05T06:44:12Z'
  notBefore: '2023-08-07T06:44:13Z'
  renewalTime: '2023-10-06T06:44:12Z'
  revision: 1

If the certificate generation process is successful, you should see your ovh-<domain>-cert secret with the two tls file entries:

kind: Secret
apiVersion: v1
metadata:
  name: ovh-<domain>-cert
  annotations:
    # cert-manager annotations
    cert-manager.io/alt-names: ...
    cert-manager.io/certificate-name: ...
    cert-manager.io/common-name: ...
    cert-manager.io/ip-sans: ...
    cert-manager.io/issuer-group: ...
    cert-manager.io/issuer-kind: ...
    cert-manager.io/issuer-name: ...
    cert-manager.io/uri-sans: ...
data:
  tls.crt: >-
    LS0t...
  tls.key: >-
    LS0t...
type: kubernetes.io/tls

4. Use your certificate secret in your Traefik IngressRoutes (as many times you need):

# secured route (https scheme) with your certificate from a cert-manager secret
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: example-tls
spec:
  # https entry point
  entryPoints:
  - https
  routes:
  - kind: Rule
    # rule's match using subdomains of the generated wildcard certificate
    match: Host(`test.domain.com`,`demo.domain.com`)
    # backend Kubernetes service
    services:
      - name: my-service
        port: 8080
  tls:
    # TLS certificate secret reference
    secretName: ovh-<domain>-cert

Note

No LetsEncrypt challenge is done by Traefik when you deploy this secured route as it has already been done and managed by the cert-manager (avoiding also the LetsEncrypt Rate Limits).

*.namespace.h8l.io certificates

If not already present, you may want to create an h8lio-tls secret holding the certificate for the routes *.[domain-cluster].h8l.io:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: h8lio
  namespace: <domain-cluster>
spec:
  dnsNames:
    # certificate common name and to avoid the TXT challenge issue:
    # ⚠️ DNS TXT record challenge may break the wildcard domain config (OVH)
    # see https://github.com/cert-manager/cert-manager/issues/806
    - "<domain-cluster>.h8l.io"
    # wildcard certificate (one per sub domain level)
    - "*.<domain-cluster>.h8l.io"
  issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    # cluster wide h8l.io issuer using LetsEncrypt
    # ⚠️ works only for the h8l.io subdomains
    name: h8lio-issuer
  # target secret name holding the
  secretName: h8lio-tls

Replace <domain-cluster> with your namespace and apply this manifest. After few seconds (checks the Certificate events/status), the h8lio-tls secret should be generated and can be used in you ingress routes tls: configuration:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
...
spec:
  entryPoints:
  - https
  routes:
  - ...
  tls:
    secretName: h8lio-tls

Caution

Remember to change your pipelines and/or helm chart to uses the new TLS configuration

Caution

If you are using the secret name, don’t mix your domain’s routes rules with the h8l.io ones. The generated certificate will only work for the specified h8l.io dnsNames