Red Hat OpenShift Container Platform

About the UserDefinedNetwork CR

To create advanced network segmentation and isolation, users and administrators create UserDefinedNetwork (UDN) custom resource (CR)s. UDNs provide granular control over network traffic within specific namespaces.

The following diagram shows four cluster namespaces, where each namespace has a single assigned user-defined network (UDN), and each UDN has an assigned custom subnet for its pod IP allocations. The OVN-Kubernetes handles any overlapping UDN subnets. Without using the Kubernetes network policy, a pod attached to a UDN can communicate with other pods in that UDN. By default, these pods are isolated from communicating with pods that exist in other UDNs. For microsegmentation, you can apply network policy within a UDN. You can assign one or more UDNs to a namespace, with a limitation of only one primary UDN to a namespace, and one or more namespaces to a UDN.

The namespace isolation concept in a user-defined network (UDN)

Figure 20. Namespace isolation using a UserDefinedNetwork CR

Best practices for UserDefinedNetwork CRs

To deploy a successful instance of the UserDefinedNetwork (UDN) CR, you must follow masquerade IP address requirements, avoid default and openshift-* namespaces, set a proper namespace selector configuration, and ensure physical network parameter matching.

The following details provide a best practice for designing a UDN CR:

openshift-* namespaces should not be used to set up a UserDefinedNetwork CR.
UserDefinedNetwork CRs should not be created in the default namespace. This can result in no isolation and, as a result, could introduce security risks to the cluster.
For primary networks, the namespace used for the UserDefinedNetwork CR must include the k8s.ovn.org/primary-user-defined-network label. This label cannot be updated, and can only be added when the namespace is created. The following conditions apply with the k8s.ovn.org/primary-user-defined-network namespace label:
- If the namespace is missing the k8s.ovn.org/primary-user-defined-network label and a pod is created, the pod attaches itself to the default network.
- If the namespace is missing the k8s.ovn.org/primary-user-defined-network label and a primary UserDefinedNetwork CR is created that matches the namespace, a status error is reported and the network is not created.
- If the namespace is missing the k8s.ovn.org/primary-user-defined-network label and a primary UserDefinedNetwork CR already exists, a pod in the namespace is created and attached to the default network.
- If the namespace has the label, and a primary UserDefinedNetwork CR does not exist, a pod in the namespace is not created until the UserDefinedNetwork CR is created.

2 masquerade IP addresses are required for user defined networks. You must reconfigure your masquerade subnet to be large enough to hold the required number of networks.

For Red Hat OpenShift Container Platform 4.17 and later, clusters use 169.254.0.0/17 for IPv4 and fd69::/112 for IPv6 as the default masquerade subnet. These ranges should be avoided by users. For updated clusters, there is no change to the default masquerade subnet.
Changing the cluster’s masquerade subnet is unsupported after a user-defined network has been configured for a project. Attempting to modify the masquerade subnet after a UserDefinedNetwork CR has been set up can disrupt the network connectivity and cause configuration issues.

Ensure tenants are using the UserDefinedNetwork resource and not the NetworkAttachmentDefinition (NAD) CR. This can create security risks between tenants.
When creating network segmentation, you should only use the NetworkAttachmentDefinition CR if user-defined network segmentation cannot be completed using the UserDefinedNetwork CR.
The cluster subnet and services CIDR for a UserDefinedNetwork CR cannot overlap with the default cluster subnet CIDR. OVN-Kubernetes network plugin uses 100.64.0.0/16 as the default join subnet for the network. You must not use that value to configure a UserDefinedNetwork CR’s joinSubnets field. If the default address values are used anywhere in the network for the cluster you must override the default values by setting the joinSubnets field. For more information, see "Additional configuration details for user-defined networks".

Creating a UserDefinedNetwork CR by using the CLI

Create a UserDefinedNetwork CR by using the CLI to enable namespace-scoped network segmentation and isolation, allowing you to define custom Layer 2 or Layer 3 network topologies for pods within specific namespaces.

The following procedure creates a UserDefinedNetwork CR that is namespace scoped. Based upon your use case, create your request by using either the my-layer-two-udn.yaml example for a Layer2 topology type or the my-layer-three-udn.yaml example for a Layer3 topology type.

When deploying a UserDefinedNetwork custom resource (CR) on IBM Power® Virtual Server with installer-provisioned infrastructure (IPI), you must set the MTU size to 1300 or 1250.

Prerequisites

As a cluster administrator, you have created a namespace.
- During namespace creation, ensure you also applied the k8s.ovn.org/primary-user-defined-network label to the namespace.
- After you create the namespace, a user that has view and edit role-based access control (RBAC) permissions can create a UserDefinedNetwork CR in the namespace.

Procedure

Optional: For a UserDefinedNetwork CR that uses a primary network, create a namespace with the k8s.ovn.org/primary-user-defined-network label by entering the following command:
```
$ cat << EOF | oc apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: <udn_namespace_name>
  labels:
    k8s.ovn.org/primary-user-defined-network: ""
EOF
```
Create a user-defined network for either a Layer2 or Layer3 topology type:
1. Create a YAML file, such as my-layer-two-udn.yaml, to define your request for a Layer2 topology as in the following example:
  apiVersion: k8s.ovn.org/v1 kind: UserDefinedNetwork metadata: name: udn-1 namespace: <some_custom_namespace> spec: topology: Layer2 layer2: role: Primary subnets: - "10.0.0.0/24" - "2001:db8::/60"
  where:
  name
  
  Name of your UserDefinedNetwork resource. This should not be default or duplicate any global namespaces created by the Cluster Network Operator (CNO).
  
  topology
  
  Specifies the network configuration; accepted values are Layer2 and Layer3. Specifying a Layer2 topology type creates one logical switch that is shared by all nodes.
  
  role
  
  Specifies a Primary or Secondary role.
  
  subnets
  
  For Layer2 topology types the following specifies config details for the subnet field:
  
  The subnets field is optional.
  
  The subnets field is of type string and accepts standard CIDR formats for both IPv4 and IPv6.
  
  The subnets field accepts one or two items. For two items, they must be of a different family. For example, subnets values of 10.100.0.0/16 and 2001:db8::/64.
  
  Layer2 subnets can be omitted. If omitted, users must configure IP addresses for the pods. As a consequence, port security only prevents MAC spoofing.
  
  The Layer2 subnets field is mandatory when the ipamLifecycle field is specified.
2. Create a YAML file, such as my-layer-three-udn.yaml, to define your request for a Layer3 topology as in the following example:
  apiVersion: k8s.ovn.org/v1 kind: UserDefinedNetwork metadata: name: udn-2-primary namespace: <some_custom_namespace> spec: topology: Layer3 layer3: role: Primary subnets: - cidr: 10.150.0.0/16 hostSubnet: 24 - cidr: 2001:db8::/60 hostSubnet: 64 # ...
  where:
  name
  
  Name of your UserDefinedNetwork resource. This should not be default or duplicate any global namespaces created by the Cluster Network Operator (CNO).
  
  topology
  
  Specifies the network configuration; accepted values are Layer2 and Layer3. Specifying a Layer2 topology type creates one logical switch that is shared by all nodes.
  
  role
  
  Specifies a Primary or Secondary role.
  
  subnets
  
  For Layer3 topology types the following specifies config details for the subnet field:
  
  The subnets field is mandatory.
  
  The type for the subnets field is cidr and hostSubnet:
  
  cidr is equivalent to the clusterNetwork configuration settings of a cluster. The IP addresses in the CIDR are distributed to pods in the user defined network. This parameter accepts a string value.
  
  hostSubnet defines the per-node subnet prefix.
  
  For IPv6, only a /64 length is supported for hostSubnet.
Apply your request by running the following command:
```
$ oc apply -f <my_layer_two_udn>.yaml
```
Where <my_layer_two_udn>.yaml is the name of your Layer2 or Layer3 configuration file.

Verify that your request is successful by running the following command:

$ oc get userdefinednetworks udn-1 -n <some_custom_namespace> -o yaml

Where some_custom_namespace is the namespace you created for your user-defined network.

Example output

apiVersion: k8s.ovn.org/v1
kind: UserDefinedNetwork
metadata:
  creationTimestamp: "2024-08-28T17:18:47Z"
  finalizers:
  - k8s.ovn.org/user-defined-network-protection
  generation: 1
  name: udn-1
  namespace: some-custom-namespace
  resourceVersion: "53313"
  uid: f483626d-6846-48a1-b88e-6bbeb8bcde8c
spec:
  layer2:
    role: Primary
    subnets:
    - 10.0.0.0/24
    - 2001:db8::/60
  topology: Layer2
status:
  conditions:
  - lastTransitionTime: "2024-08-28T17:18:47Z"
    message: NetworkAttachmentDefinition has been created
    reason: NetworkAttachmentDefinitionReady
    status: "True"
    type: NetworkCreated

Creating a UserDefinedNetwork CR by using the web console

To implement isolated network segments with layer 2 connectivity in Red Hat OpenShift Container Platform, create a UserDefinedNetwork custom resource (CR) by using the web console. Defining this resource ensures that your cluster workloads can communicate directly at the data link layer.

Currently, creation of a UserDefinedNetwork CR with a Layer3 topology or a Secondary role are not supported when using the Red Hat OpenShift Container Platform web console.

Prerequisites

As a cluster administrator, you have created a namespace.
- During namespace creation, ensure you also applied the k8s.ovn.org/primary-user-defined-network label to the namespace.
- After you create the namespace, a user that has view and edit role-based access control (RBAC) permissions can create a UserDefinedNetwork CR in the namespace.

Procedure

From the Administrator perspective, click Networking → UserDefinedNetworks.
Click Create UserDefinedNetwork.
From the Project name list, select the namespace that you previously created.
Specify a value in the Subnet field.
Click Create. The user-defined network serves as the default primary network for pods that you create in this namespace.