Red Hat build of MicroShift

Configure kubelet parameters and values in MicroShift

To enable low latency on a MicroShift node, add the required settings to the MicroShift config.yaml file.

Prerequisites

You installed the OpenShift CLI (oc).
You have root access to the node.
You made a copy of the provided config.yaml.default file in the /etc/microshift/ directory, and renamed it config.yaml.

Procedure

Add the kubelet configuration to the MicroShift config.yaml file:
Example passthrough kubelet configuration
```
apiServer:
# ...
kubelet:
  cpuManagerPolicy: static
  cpuManagerPolicyOptions:
    full-pcpus-only: "true"
  cpuManagerReconcilePeriod: 5s
  memoryManagerPolicy: Static
  topologyManagerPolicy: single-numa-node
  reservedSystemCPUs: 0-1
  reservedMemory:
  - limits:
      memory: 1100Mi
    numaNode: 0
  kubeReserved:
    memory: 500Mi
  systemReserved:
    memory: 500Mi
  evictionHard:
    imagefs.available: "15%"
    memory.available: "100Mi"
    nodefs.available: "10%"
    nodefs.inodesFree: "5%"
  evictionPressureTransitionPeriod: 5m
# ...
```
kubelet

If you change the CPU or memory managers in the kubelet configuration, you must remove files that cache the previous configuration. Restart the host to remove them automatically, or manually remove the /var/lib/kubelet/cpu_manager_state and /var/lib/kubelet/memory_manager_state files.

kubelet.cpuManagerPolicy

The name of the policy to use. Valid values are none and static. Requires the CPUManager feature gate to be enabled. Default value is none.

kubelet.cpuManagerPolicyOptions.full-pcpus-only

A set of key=value pairs for setting extra options that fine tune the behavior of the CPUManager policies. The default value is null. Requires both the CPUManager and CPUManagerPolicyOptions feature gates to be enabled.

kubelet.memoryManagerPolicy

The name of the policy used by Memory Manager. Case-sensitive. The default value is none. Requires the MemoryManager feature gate to be enabled.

kubelet.reservedSystemCPUs

Required. The reservedSystemCPUs value must be the inverse of the offlined CPUs because both values combined must account for all of the CPUs on the system. This parameter is essential to dividing the management and application workloads. Use this parameter to define a static CPU set for the host-level system and Kubernetes daemons, plus interrupts and timers. Then the rest of the CPUs on the system can be used exclusively for workloads.

kubelet.reservedMemory[0].limits.memory

The value in reservedMemory[0].limits.memory, 1100 Mi in this example, is equal to kubeReserved.memory + systemReserved.memory + evictionHard.memory.available.

kubelet.evictionHard

The evictionHard parameters define under which conditions the kubelet evicts pods. When you change the default value of only one parameter for the evictionHard stanza, the default values of other parameters are not inherited and are set to zero. Provide all the threshold values even when you want to change just one.

kubelet.evictionHard.imagefs.available

The imagefs is a filesystem that container runtimes use to store container images and container writable layers. In this example, the evictionHard.imagefs.available parameter means that the pod is evicted when the available space of the image filesystem is less than 15%.

kubelet.evictionHard.memory.available

In this example, the evictionHard.memory.available parameter means that the pods are evicted when the available memory of the node drops below 100MiB.

kubelet.evictionHard.nodefs.available

In this example, the evictionHard.nodefs.available parameter means that the pods are evicted when the main filesystem of the node has less than 10% available space.

kubelet.evictionHard.nodefs.inodesFree

In this example, the evictionHard.nodefs.inodesFree parameter means that the pods are evicted when more than 15% of the node’s main filesystem’s inodes are in use.

kubelet.evictionPressureTransitionPeriod

For container garbage collection: The duration to wait before transitioning out of an eviction pressure condition. Setting the evictionPressureTransitionPeriod parameter to 0 configures the default value of 5 minutes.

Verification

After you complete the next steps and restart the host, you can use a root-access account to check that your settings are in the config.yaml file in the /var/lib/microshift/resources/kubelet/config/ directory.

Next steps

Enable workload partitioning.
Tune your operating system. For example, configure and activate a TuneD profile.
Optional: Configure automatic enablement of your TuneD profile.
Optional: If you are using the x86_64 architecture, you can install Red Hat Enterprise Linux for Real Time (real-time kernel).
Prepare your MicroShift workloads for low latency.