Enable KMS encryption

You can enable external Key Management Service (KMS) encryption for etcd data to centralize key management and meet compliance requirements.

Prerequisites
  • You have access to the cluster as a user with the cluster-admin role.

  • You have enabled the TechPreviewNoUpgrade feature set to enable the KMSEncryption feature gate.

  • You have a HashiCorp Vault Enterprise instance accessible from your control plane nodes. Vault Community Edition is not available.

  • Your control plane nodes have network access to the Vault server.

  • You have configured your Vault instance with:

    • Transit secrets engine enabled at the default transit/ mount path

    • Encryption key created with type aes256-gcm96 (recommended for FIPS 140-3 compliance)

    • Vault policy allowing the Kubernetes KMS v2 plugin to encrypt, decrypt, and read key information

    • Authentication method (token, AppRole, or userpass) configured with the policy attached

The Kubernetes KMS v2 plugin requires a Vault policy with the following capabilities:

path "transit/encrypt/kms-key" {
  capabilities = ["update"]
}

path "transit/decrypt/kms-key" {
  capabilities = ["update"]
}

path "transit/keys/kms-key" {
  capabilities = ["read"]
}

path "auth/token/lookup-self" {
  capabilities = ["read"]
}

Replace kms-key with your Vault Transit key name if using a different name.

Important

Create an etcd backup before enabling KMS encryption.

Procedure
  1. Deploy the KMS plugin on each control plane host as a static pod:

    • Configure the plugin to listen at unix:///var/run/kmsplugin/kms.sock

    • For your static pod, mount /var/run/kmsplugin as hostPath

    • Configure KMS provider connection details and authentication credentials

    The following steps show how to deploy the HashiCorp Vault KMS plugin as a static pod.

  2. Create a static pod manifest file:

    $ cat > /tmp/vault-kms-plugin.yaml <<'EOF'
    apiVersion: v1
    kind: Pod
    metadata:
      name: vault-kms-plugin
      namespace: kube-system
      labels:
        app: vault-kms-plugin
        tier: control-plane
    spec:
      priorityClassName: system-node-critical
      hostNetwork: true
      containers:
      - name: vault-kms-plugin
        image: quay.io/redhat-isv-containers/698df066f8d1ddf179c15ef9:<version>
        command:
        - /vault-kubernetes-kms
        - -vault-address=<https://vault.example.com:8200>
        - -<vault_namespace>=admin
        - -auth-method=userpass
        - -userpass-username=<vault_username>
        - -userpass-password=<vault_password>
        - -transit-mount=transit
        - -transit-key=kms-key
        - -socket=unix:///var/run/kmsplugin/kms.sock
        volumeMounts:
        - name: kmsplugin
          mountPath: /var/run/kmsplugin
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 512Mi
        securityContext:
          privileged: true
      volumes:
      - name: kmsplugin
        hostPath:
          path: /var/run/kmsplugin
          type: DirectoryOrCreate
    EOF

    Replace the following values to match your environment:

    <version>

    The Vault KMS plugin image version tag. Use 0.1.0-beta-ubi.

    <vault_username>

    Your Vault username for authentication.

    <vault_password>

    Your Vault password for authentication.

    https://vault.example.com:8200

    The Vault address. Update this field to match your Vault server URL.

    <vault_namespace>

    Optional field.

    The manifest uses the Red Hat certified container image from Quay.io (quay.io/redhat-isv-containers/698df066f8d1ddf179c15ef9), which is the recommended image for Red Hat OpenShift Container Platform.

    Note

    Alternatively, you can use the HashiCorp image from Docker Hub by replacing the image field with docker.io/hashicorp/vault-kube-kms:0.1.0-beta-ubi.

  3. Deploy the static pod manifest to each control plane node by running the following commands:

    $ MANIFEST=$(cat /tmp/vault-kms-plugin.yaml | base64)
    $ for node in $(oc get nodes --selector=node-role.kubernetes.io/master -o name | cut -d/ -f2); do
        echo "Deploying to $node..."
        oc debug node/$node -- chroot /host bash -c \
          "echo '$MANIFEST' | base64 -d > /etc/kubernetes/manifests/vault-kms-plugin.yaml"
      done

    The kubelet automatically detects and starts static pods from /etc/kubernetes/manifests/.

  4. Verify the static pods are running by entering the following command:

    $ oc get pods -n kube-system -o wide | grep vault-kms
    Example output
    vault-kms-plugin-ip-10-0-16-7.compute.internal    1/1  Running  0  2m  10.0.16.7
    vault-kms-plugin-ip-10-0-32-93.compute.internal   1/1  Running  0  2m  10.0.32.93
    vault-kms-plugin-ip-10-0-69-106.compute.internal  1/1  Running  0  2m  10.0.69.106

    Static pod names include the node name as a suffix. You should see one pod per control plane node.

  5. Verify the socket exists on a control plane node by entering the following command:

    $ oc debug node/<node_name> -- chroot /host ls -la /var/run/kmsplugin/kms.sock
    Example output
    srwxr-xr-x. 1 root root 0 <timestamp> /var/run/kmsplugin/kms.sock
    Note

    Static pods are managed by kubelet on each node and cannot be deleted with oc delete pod. To remove a static pod, delete the manifest file from /etc/kubernetes/manifests/ on each control plane node.

  6. Edit the APIServer custom resource by entering the following command:

    $ oc edit apiserver cluster
  7. Add the KMS configuration to the spec.encryption section:

    apiVersion: config.openshift.io/v1
    kind: APIServer
    metadata:
      name: cluster
    spec:
      encryption:
        type: KMS
  8. Save and exit.

    Migration begins automatically. The kube-apiserver, openshift-apiserver and oauth-apiserver Operators will restart and roll out new revisions.

    Note

    The openshift-apiserver and authentication operators typically complete migration in 5-10 minutes. The kube-apiserver operator uses a conservative rollout strategy, updating one control plane node at a time and waiting for health checks before proceeding to the next node. This process can take 30 minutes or longer depending on cluster load.

Verification
  1. Verify the encryption type by entering the following command:

    $ oc get apiserver cluster -o jsonpath='{.spec.encryption.type}'

    Output should show KMS.

  2. Monitor the kube-apiserver rollout progress by entering the following command:

    $ oc get kubeapiserver cluster -o jsonpath='{.status.nodeStatuses}' | jq -r '.[] | "\(.nodeName | split(".")[0]): current=\(.currentRevision) target=\(.targetRevision)"'
    Example output during rollout
    ip-10-0-16-166: current=10 target=0
    ip-10-0-32-93: current=9 target=10
    ip-10-0-69-106: current=9 target=0

    The operator rolls out one node at a time. When all nodes show the same current revision and target is 0, the rollout is complete.

  3. Check the encryption migration status by entering the following command:

    $ oc get kubeapiserver cluster -o jsonpath='{.status.conditions[?(@.type=="Encrypted")]}' | jq .
    Example output when complete
    {
      "lastTransitionTime": "2026-05-15T17:39:02Z",
      "message": "All resources encrypted: secrets, configmaps",
      "reason": "EncryptionCompleted",
      "status": "True",
      "type": "Encrypted"
    }

    Wait for reason to show EncryptionCompleted before proceeding to verify encryption in etcd.

  4. Verify secrets are encrypted in etcd:

    Warning

    Wait for the kube-apiserver rollout to complete on all control plane nodes before verifying encryption. During the rollout, API requests are distributed across nodes, and secrets created while some nodes are still on an earlier revision will not be encrypted with Kubernetes KMS v2.

    1. Create a test secret by entering the following command:

      $ oc create secret generic test-secret --from-literal=key=value -n default
    2. Get an etcd pod name by entering the following command:

      $ oc get pods -n openshift-etcd -l app=etcd -o name | head -1
    3. Check the secret data in etcd by entering the following command:

      $ oc exec -n openshift-etcd <etcd_pod_name> -- etcdctl get /kubernetes.io/secrets/default/test-secret --print-value-only | hexdump -C | head -1

      Output should begin with k8s:enc:kms:v2: followed by encrypted binary data.

    4. Delete the test secret by entering the following command:

      $ oc delete secret test-secret -n default