OpenShift Container Platform

Check node status during updates

During the update of a machine config pool (MCP), you can monitor the progress of all of the nodes in your cluster by using the oc get machineconfignodes and oc describe machineconfignodes commands. These commands provide information that can be helpful if issues arise during the update and you need to troubleshoot a node.

For more information on the meaning of these fields, see "About checking machine config node status."

Prerequisites

In order to see specific machine config node output, as described in "About checking machine config node status", you must enable the TechPreviewNoUpgrade feature set on the cluster. For more information, see "Enabling features using feature gates".

Enabling the TechPreviewNoUpgrade feature set cannot be undone and prevents minor version updates. These feature sets are not recommended on production clusters.

The custom layered image output is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Procedure

View the update status of all nodes in the cluster, including the current and desired machine configurations, by running the following command:

$ oc get machineconfignodes

Example output

NAME                                       POOLNAME   DESIREDCONFIG                                      CURRENTCONFIG                                      UPDATED   AGE
ci-ln-mdb23yt-72292-kzdsg-master-0         master     rendered-master-f21b093d20f68a7c06f922ed3ea5fbc8   rendered-master-1abc053eec29e6c945670f39d6dc8afa   False     27M
ci-ln-mdb23yt-72292-kzdsg-master-1         master     rendered-master-1abc053eec29e6c945670f39d6dc8afa   rendered-master-1abc053eec29e6c945670f39d6dc8afa   True      27M
ci-ln-mdb23yt-72292-kzdsg-master-2         master     rendered-master-1abc053eec29e6c945670f39d6dc8afa   rendered-master-1abc053eec29e6c945670f39d6dc8afa   True      27M
ci-ln-mdb23yt-72292-kzdsg-worker-a-gfqjr   worker     rendered-worker-d0130cd74e9e576d7ba78ce166272bfb   rendered-worker-8f61bf839898a4487c3b5263a430e94a   False     20M
ci-ln-mdb23yt-72292-kzdsg-worker-b-gknq4   worker     rendered-worker-8f61bf839898a4487c3b5263a430e94a   rendered-worker-8f61bf839898a4487c3b5263a430e94a   True      20M
ci-ln-mdb23yt-72292-kzdsg-worker-c-mffrx   worker     rendered-worker-8f61bf839898a4487c3b5263a430e94a   rendered-worker-8f61bf839898a4487c3b5263a430e94a   True      19M

View of all machine config node status fields for the nodes in your cluster by running the following command:

$ oc get machineconfignodes -o wide

Example output

NAME                                       POOLNAME   DESIREDCONFIG                                      CURRENTCONFIG                                      UPDATED   AGE   UPDATEPREPARED   UPDATEEXECUTED   UPDATEPOSTACTIONCOMPLETE   UPDATECOMPLETE   RESUMED   UPDATEDFILESANDOS   CORDONEDNODE   DRAINEDNODE   REBOOTEDNODE   UNCORDONEDNODE
ci-ln-g6dr34b-72292-g9btv-master-0         master     rendered-master-d4e122320b351cdbe1df59ddb63ddcfc   rendered-master-6f2064fcb36d2a914de5b0c660dc49ff   False     27M   True             Unknown          False                      False            False     Unknown             False          False         False          False
ci-ln-g6dr34b-72292-g9btv-master-1         master     rendered-master-6f2064fcb36d2a914de5b0c660dc49ff   rendered-master-6f2064fcb36d2a914de5b0c660dc49ff   True      27M   False            False            False                      False            False     False               False          False         False          False
ci-ln-g6dr34b-72292-g9btv-master-2         master     rendered-master-6f2064fcb36d2a914de5b0c660dc49ff   rendered-master-6f2064fcb36d2a914de5b0c660dc49ff   True      27M   False            False            False                      False            False     False               False          False         False          False
ci-ln-g6dr34b-72292-g9btv-worker-a-sjh5r   worker     rendered-worker-671b88c8c569fa3f60dc1a27cf9c91f2   rendered-worker-d5534cb730e5e108905fc285c2a42b6c   False     20M   True             Unknown          False                      False            False     Unknown             False          False         False          False
ci-ln-g6dr34b-72292-g9btv-worker-b-xthbz   worker     rendered-worker-d5534cb730e5e108905fc285c2a42b6c   rendered-worker-d5534cb730e5e108905fc285c2a42b6c   True      20M   False            False            False                      False            False     False               False          False         False          False
ci-ln-g6dr34b-72292-g9btv-worker-c-gnpd6   worker     rendered-worker-d5534cb730e5e108905fc285c2a42b6c   rendered-worker-d5534cb730e5e108905fc285c2a42b6c   True      19M   False            False            False                      False            False     False               False          False         False          False

Check the update status of nodes in a specific machine config pool by running the following command:

$ oc get machineconfignodes $(oc get machineconfignodes -o json | jq -r '.items[]|select(.spec.pool.name=="<pool_name>")|.metadata.name')

where:

<pool_name>

Specifies the name of the machine config pool.

Example output

NAME                                       POOLNAME   DESIREDCONFIG                                      CURRENTCONFIG                                      UPDATED   AGE
ci-ln-g6dr34b-72292-g9btv-worker-a-sjh5r   worker     rendered-worker-d5534cb730e5e108905fc285c2a42b6c   rendered-worker-d5534cb730e5e108905fc285c2a42b6c   True      20M
ci-ln-g6dr34b-72292-g9btv-worker-b-xthbz   worker     rendered-worker-d5534cb730e5e108905fc285c2a42b6c   rendered-worker-faf6b50218a8bbce21f1370866283de5   False     20M
ci-ln-g6dr34b-72292-g9btv-worker-c-gnpd6   worker     rendered-worker-faf6b50218a8bbce21f1370866283de5   rendered-worker-faf6b50218a8bbce21f1370866283de5   True      19M

Check the update status of an individual node by running the following command:

$ oc describe machineconfignode/<node_name>

Example output

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigNode
metadata:
  creationTimestamp: "2025-04-28T18:52:16Z"
  generation: 3
  name: ci-ln-921r7qk-72292-kxv95-worker-a-zmxrr
  ownerReferences:
  - apiVersion: v1
    kind: Node
    name: ci-ln-921r7qk-72292-kxv95-worker-a-zmxrr
    uid: e548a8d1-4f16-42cd-9234-87ac5aede6c1
  resourceVersion: "62331"
  uid: 11d96e07-582d-4569-a84a-9d8c5229a551
spec:
  configVersion:
    desired: rendered-worker-1930ca7433b7f0153286a3f04e4cb57b
  node:
    name: ci-ln-921r7qk-72292-kxv95-worker-a-zmxrr
  pool:
    name: worker
status:
  conditions:
# ...
    lastTransitionTime: 2025-04-23T14:55:31Z
    message: Update Compatible. Post Cfg Actions: [] Drain Required: true
    reason: UpdatePrepared
    status: True
    type: UpdatePrepared
# ...
    lastTransitionTime: 2025-04-23T14:55:31Z
    message: Draining node. The drain will not be complete until desired drainer drain-rendered-worker-1930ca7433b7f0153286a3f04e4cb57b
      matches current drainer uncordon-rendered-worker-a9673968884f1ea42c26edcd914af907
    reason: UpdateExecutedDrained
    status: True
    type: Drained
# ...
    lastTransitionTime: 2025-04-23T14:55:31Z
    message: Cordoned node. The node is reporting Unschedulable = true
    reason: UpdateExecutedCordoned
    status: True
    type: Cordoned
# ...
  - lastTransitionTime: "2025-04-28T18:52:16Z"
    message: This node has not yet entered the NodeDegraded phase
    reason: NotYetOccurred
    status: "False"
    type: NodeDegraded
# ...
  configversion:
    current: rendered-worker-8110974a5cea69dff5b263237b58abd8
    desired: rendered-worker-1930ca7433b7f0153286a3f04e4cb57b
  observedgeneration:  4
  pinnedImageSets:
  - desiredGeneration: 1
    name: worker-pinned-images
# ...