Red Hat OpenShift Container Platform

Cluster autoscaler resource definition

This ClusterAutoscaler resource definition shows the parameters and sample values for the cluster autoscaler.

When you change the configuration of an existing cluster autoscaler, it restarts.

apiVersion: "autoscaling.openshift.io/v1"
kind: "ClusterAutoscaler"
metadata:
  name: "default"
spec:
  podPriorityThreshold: -10
  resourceLimits:
    maxNodesTotal: 24
    cores:
      min: 8
      max: 128
    memory:
      min: 4
      max: 256
    gpus:
    - type: <gpu_type>
      min: 0
      max: 16
  logVerbosity: 4
  scaleDown:
    cordonNodeBeforeTerminating: Enabled
    enabled: true
    delayAfterAdd: 10m
    delayAfterDelete: 5m
    delayAfterFailure: 30s
    unneededTime: 5m
    utilizationThreshold: "0.4"
  scaleUp:
    newPodScaleUpDelay: "10s"
  expanders: ["Random"]

Table 81. Cluster autoscaler parameters
Parameter	Description
`podPriorityThreshold`	Specify the priority that a pod must exceed to cause the cluster autoscaler to deploy additional nodes. Enter a 32-bit integer value. The `podPriorityThreshold` value is compared to the value of the `PriorityClass` that you assign to each pod.
`maxNodesTotal`	Specify the maximum number of nodes to deploy. This value is the total number of machines that are deployed in your cluster, not just the ones that the autoscaler controls. Ensure that this value is large enough to account for all of your control plane and compute machines and the total number of replicas that you specify in your `MachineAutoscaler` resources.
`cores.min`	Specify the minimum number of cores to deploy in the cluster.
`cores.max`	Specify the maximum number of cores to deploy in the cluster.
`memory.min`	Specify the minimum amount of memory, in GiB, in the cluster.
`memory.max`	Specify the maximum amount of memory, in GiB, in the cluster.
`gpus.type`	Optional: To configure the cluster autoscaler to deploy GPU-enabled nodes, specify a `type` value. This value must match the value of the `spec.template.spec.metadata.labels[cluster-api/accelerator]` label in the machine set that manages the GPU-enabled nodes of that type. For example, this value might be `nvidia-t4` to represent Nvidia T4 GPUs, or `nvidia-a10g` for A10G GPUs. For more information, see "Labeling GPU machine sets for the cluster autoscaler".
`gpus.min`	Specify the minimum number of GPUs of the specified type to deploy in the cluster.
`gpus.max`	Specify the maximum number of GPUs of the specified type to deploy in the cluster.
`logVerbosity`	Specify the logging verbosity level between `0` and `10`. The following log level thresholds are provided for guidance: `1`: (Default) Basic information about changes. `4`: Debug-level verbosity for troubleshooting typical issues. `9`: Extensive, protocol-level debugging information. If you do not specify a value, the default value of `1` is used.
`scaleDown`	In this section, you can specify the period to wait for each action by using any valid ParseDuration interval, including `ns`, `us`, `ms`, `s`, `m`, and `h`.
`scaleDown.cordonNodeBeforeTerminating`	Optional: Specify whether the cluster autoscaler should cordon a node before removing that node by using one of the following values: `Enabled`: The cluster autoscaler cordons the node before draining any pods and removing that node. `Disabled`: The cluster autoscaler does not cordon the node before draining any pods and removing that node. This is the default.
`scaleDown.enabled`	Specify whether the cluster autoscaler can remove unnecessary nodes.
`scaleDown.delayAfterAdd`	Optional: Specify the period to wait before deleting a node after a node has recently been added. If you do not specify a value, the default value of `10m` is used.
`scaleDown.delayAfterDelete`	Optional: Specify the period to wait before deleting a node after a node has recently been deleted. If you do not specify a value, the default value of `0s` is used.
`scaleDown.delayAfterFailure`	Optional: Specify the period to wait before deleting a node after a scale down failure occurred. If you do not specify a value, the default value of `3m` is used.
`scaleDown.unneededTime`	Optional: Specify a period of time before an unnecessary node is eligible for deletion. If you do not specify a value, the default value of `10m` is used.
`scaleDown.utilizationThreshold`	Optional: Specify the node utilization level. Nodes below this utilization level are eligible for deletion. The node utilization level is the sum of the requested resources divided by the allocated resources for the node, and must be a value greater than `"0"` but less than `"1"`. If you do not specify a value, the cluster autoscaler uses a default value of `"0.5"`, which corresponds to 50% utilization. You must express this value as a string.
`scaleUp`	In this section, you can specify the period to wait before recognizing newly pending pods by using any valid ParseDuration interval, including `ns`, `us`, `ms`, `s`, `m`, and `h`.
`scaleUp.newPodScaleUpDelay`	Optional: Specify the period to ignore a new unschedulable pod before adding a new node. If you do not specify a value, the default value of `0s` is used.
`expanders`	Optional: Specify any expanders that you want the cluster autoscaler to use. The following values are valid: `LeastWaste`: Selects the machine set that minimizes the idle CPU after scaling. If multiple machine sets would yield the same amount of idle CPU, the selection minimizes unused memory. `Priority`: Selects the machine set with the highest user-assigned priority. To use this expander, you must create a config map that defines the priority of your machine sets. For more information, see "Configuring a priority expander for the cluster autoscaler." `Random`: (Default) Selects the machine set randomly. If you do not specify a value, the default value of `Random` is used. You can specify multiple expanders by using the `[LeastWaste, Priority]` format. The cluster autoscaler applies each expander according to the specified order. In the `[LeastWaste, Priority]` example, the cluster autoscaler first evaluates according to the `LeastWaste` criteria. If more than one machine set satisfies the `LeastWaste` criteria equally well, the cluster autoscaler then evaluates according to the `Priority` criteria. If more than one machine set satisfies all of the specified expanders equally well, the cluster autoscaler selects one to use at random.

When performing a scaling operation, the cluster autoscaler remains within the ranges set in the ClusterAutoscaler resource definition, such as the minimum and maximum number of cores to deploy or the amount of memory in the cluster. However, the cluster autoscaler does not correct the current values in your cluster to be within those ranges.

The minimum and maximum CPUs, memory, and GPU values are determined by calculating those resources on all nodes in the cluster, even if the cluster autoscaler does not manage the nodes. For example, the control plane nodes are considered in the total memory in the cluster, even though the cluster autoscaler does not manage the control plane nodes.