TLS Certificates in Kubernetes

Overview

• cert-manager-spiffe uses Kubernetes serviceAccounts, clusterDomain, roles, and rolebindings to build the SVID, e.g., spiffe://forge.local/forge-system/carbide-api
• Certificates are available in pods at /run/secrets/spiffe.io/{tls.crt,tls.key,ca.crt}
• To retrieve a certificate, you must first create a serviceAccount, role, and roleBinding (example below)
• Don’t forget to update the namespace to the correct value
• Helm upgrade/install generates the Labels you see in the example below; you can omit those.
• The role associated with the serviceAccount grants enough permissions to request a certificate from cert-manager-csi-driver-spiffe

Cert-Manager

The CertificateRequest (which includes the CSR) references a ClusterIssuer set up during the initial bootstrap of the site.

The ClusterIssuer sends CSRs to Vault for signing using the forgeCA PKI. Before a CertificateRequest can be signed, it must be approved.

cert-manager-csi-driver-spiffe-approver runs as a deployment and is responsible for verifying the CertificateRequest meets specific criteria

If all criteria are met, the CertificateRequest is approved, and cert-manager sends the CSR portion of the CertificateRequest to Vault for signing.

SPIFFE

SPIFFE is a means of identifying software systems. The identity of the software is cryptographically verifiable and exists within a “trust domain” The trust domain could be a user, organization, or anything representable in a URI.

With SPIFFE formatted Certificates, the only field populated is the SAN (Subject Alternative Name). The SAN must conform to the SPIFFE ID format.

The validation of the SPIFFE ID format and submission of CertificateRequest gets handled by cert-manager-csi-driver-spiffe-approver and cert-manager-csi-driver-spiffe, respectively.

cert-manager-csi-driver-spiffe runs as a DaemonSet. It is responsible for generating the TLS key, CSR and submitting the CSR for approval (By way of CertificateRequest).

NOTE

The TLS key generated in every pod never leaves the host which it was generated on. If a migration event occurs, the CSR/key are regenerated, submitted to CertManager, and then signed again.

How to obtain a SPIFFE formatted cert

1
apiVersion: v1
2
kind: ServiceAccount
3
metadata:
4
name: carbide-api
5
namespace: "default"
6
labels:
7
app.kubernetes.io/name: carbide-api
8
helm.sh/chart: carbideApi-0.0.1
9
app.kubernetes.io/instance: release-name
10
app.kubernetes.io/managed-by: Helm
11
app.kubernetes.io/component: carbide-api
12
automountServiceAccountToken: true
13
14
---
15
16
kind: Role
17
apiVersion: rbac.authorization.k8s.io/v1
18
metadata:
19
name: carbide-api
20
namespace: "default"
21
labels:
22
app.kubernetes.io/name: carbide-api
23
helm.sh/chart: carbideApi-0.0.1
24
app.kubernetes.io/instance: release-name
25
app.kubernetes.io/managed-by: Helm
26
app.kubernetes.io/component: carbide-api
27
rules:
28
29
- apiGroups: ["cert-manager.io"]
30
  resources: ["certificaterequests"]
31
  verbs: ["create"]
32
33
---
34
35
kind: RoleBinding
36
apiVersion: rbac.authorization.k8s.io/v1
37
metadata:
38
name: carbide-api
39
namespace: default
40
labels:
41
app.kubernetes.io/name: carbide-api
42
helm.sh/chart: carbideApi-0.0.1
43
app.kubernetes.io/instance: release-name
44
app.kubernetes.io/managed-by: Helm
45
app.kubernetes.io/component: carbide-api
46
roleRef:
47
apiGroup: rbac.authorization.k8s.io
48
kind: Role
49
name: carbide-api
50
subjects:
51
52
- kind: ServiceAccount
53
  name: carbide-api
54
  namespace: "default"

After creating the serviceAccount, role, and rolebinding, modify your deployment/pod spec to request a Certificate

1
spec:
2
  serviceAccountName: carbide-api
3
...
4
      volumeMounts:
5
        - name: spiffe
6
          mountPath: "/var/run/secrets/spiffe.io"
7
...
8
    volumes:
9
    - name: spiffe
10
    csi:
11
      driver: spiffe.csi.cert-manager.io
12
      readOnly: true

NON-SPIFFE

Some components in Kubernetes cannot use SPIFFE formatted certs ValidatingWebhooks and MutatingWebhooks can not use SPIFFE formatted CertificateRequests

For those resources, there is a separate ClusterIssuer that signs CertificateRequests which are not SPIFFE formatted.

There is a CertificateRequestPolicy that enforces specific criteria for non-SPIFFE CertificateRequests. The policy only allows signing requests for Service based TLS certs.