Red Hat OpenShift Container Platform

Disable KMS encryption

You can disable external KMS encryption and migrate to local encryption to simplify operations or resolve KMS connectivity issues.

Prerequisites

You have access to the cluster as a user with the cluster-admin role.
You have created an etcd backup.

Re-encryption of all etcd data will occur.

Procedure

Edit the APIServer custom resource by entering the following command:
```
$ oc edit apiserver cluster
```

Change the encryption configuration:

apiVersion: config.openshift.io/v1
kind: APIServer
metadata:
  name: cluster
spec:
  encryption:
    type: <encryption>

Replace <encryption> with aescbc, aesgcm, or identity.

Save and exit.

Red Hat OpenShift Container Platform automatically migrates etcd data. Migration time depends on etcd size and secret count.

Monitor the migration progress by entering the following command:

$ oc get openshiftapiserver -o=jsonpath='{range .items[0].status.conditions[?(@.type=="Encrypted")]}{.reason}{"\n"}{.message}{"\n"}{end}'

Wait until the output shows EncryptionCompleted.

Verify kube-apiserver pods rolled out by entering the following command:

$ oc get pods -n openshift-kube-apiserver -l app=openshift-kube-apiserver

Remove the static pod manifest from each control plane node by running the following command:

$ for node in $(oc get nodes --selector=node-role.kubernetes.io/master -o name | cut -d/ -f2); do
    echo "Removing static pod from $node..."
    oc debug node/$node -- chroot /host \
      rm -f /etc/kubernetes/manifests/vault-kms-plugin.yaml
  done

The kubelet automatically stops static pods when their manifest is removed from /etc/kubernetes/manifests/.

Clean up the socket directory by running the following command:

$ for node in $(oc get nodes --selector=node-role.kubernetes.io/master -o name | cut -d/ -f2); do
    oc debug node/$node -- chroot /host rm -rf /var/run/kmsplugin
  done

After backup retention period passes, decommission the KMS key.

Verification

Verify the encryption type by entering the following command:

$ oc get apiserver cluster -o jsonpath='{.spec.encryption.type}'

Verify that a test secret uses the new encryption type:
1. Create a test secret by entering the following command:
  
  $ oc create secret generic encryption-test --from-literal=key=value -n default
2. Get an etcd pod name by entering the following command:
  
  $ oc get pods -n openshift-etcd -l app=etcd -o name | head -1
3. Check the encryption prefix by entering the following command:
  
  $ oc exec -n openshift-etcd <etcd_pod_name> -- etcdctl get /kubernetes.io/secrets/default/encryption-test --print-value-only | hexdump -C | head -1
  
  Output should begin with k8s:enc:aescbc:v1:, k8s:enc:aesgcm:v1:, or show readable JSON for identity.
4. Delete the test secret by entering the following command:
  
  $ oc delete secret encryption-test -n default

Do not delete the KMS key until migration completes successfully.