Red Hat OpenShift Container Platform

Configuring metrics collection for the Operator by using a ServiceMonitor

The Zero Trust Workload Identity Manager exposes metrics by default on port 8443 at the /metrics service endpoint. You can configure metrics collection for the Operator by creating a ServiceMonitor custom resource (CR) that enables the Prometheus Operator to collect custom metrics. For more information, see "Configuring user workload monitoring".

The SPIRE Server operand exposes metrics by default on port 9402 at the /metrics endpoint. You can configure metrics collection for the SPIRE Server by creating a ServiceMonitor custom resource (CR) that enables the Prometheus Operator to collect custom metrics.

Prerequisites

You have access to the cluster as a user with the cluster-admin cluster role.
You have installed the Zero Trust Workload Identity Manager.
You have enabled the user workload monitoring.

Procedure

Configure the Operator to use HTTP or HTTPS protocols for the metrics server.

Update the subscription object for Zero Trust Workload Identity Manager to configure the HTTP protocol by running the following command:

$ oc -n zero-trust-workload-identity-manager patch subscription zero-trust-workload-identity-manager-subscription --type='merge' -p '{"spec":{"config":{"env":[{"name":"METRICS_BIND_ADDRESS","value":":8080"}, {"name": "METRICS_SECURE", "value": "false"}]}}}'

Verify the Zero Trust Workload Identity Manager pod is redeployed and that the configured values for METRICS_BIND_ADDRESS and METRICS_SECURE is updated by running the following command:

$ oc set env --list deployment/zero-trust-workload-identity-manager-controller-manager -n zero-trust-workload-identity-manager | grep -e METRICS_BIND_ADDRESS -e METRICS_SECURE -e container

Example output

deployments/zero-trust-workload-identity-manager-controller-manager, container manager
METRICS_BIND_ADDRESS=:8080
METRICS_SECURE=false

Create the Secret resource with kubernetes.io/service-account.name annotation to inject the token required for authenticating with the metrics server.

Create the secret-zero-trust-workload-identity-manager.yaml YAML file:

apiVersion: v1
kind: Secret
metadata:
 labels:
   name: zero-trust-workload-identity-manager
 name: zero-trust-workload-identity-manager-metrics-auth
 namespace: zero-trust-workload-identity-manager
 annotations:
   kubernetes.io/service-account.name: zero-trust-workload-identity-manager-controller-manager
type: kubernetes.io/service-account-token

Create the Secret resource by running the following command:

$ oc apply -f secret-zero-trust-workload-identity-manager.yaml

Create the ClusterRoleBinding resource required for granting permissions to access the metrics.

Create the clusterrolebinding-zero-trust-workload-identity-manager.yaml YAML file:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 labels:
   name: zero-trust-workload-identity-manager
 name: zero-trust-workload-identity-manager-allow-metrics-access
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: zero-trust-workload-identity-manager-metrics-reader
subjects:
- kind: ServiceAccount
  name: zero-trust-workload-identity-manager-controller-manager
  namespace: zero-trust-workload-identity-manager

Create the ClusterRoleBinding resource by running the following command:

$ oc apply -f clusterrolebinding-zero-trust-workload-identity-manager.yaml

Create the following ServiceMonitor CR if the metrics server is configured to use http.

Create the servicemonitor-zero-trust-workload-identity-manager-http.yaml YAML file:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    name: zero-trust-workload-identity-manager
  name: zero-trust-workload-identity-manager-metrics-monitor
  namespace: zero-trust-workload-identity-manager
spec:
  endpoints:
    - authorization:
        credentials:
          name: zero-trust-workload-identity-manager-metrics-auth
          key: token
        type: Bearer
      interval: 60s
      path: /metrics
      port: metrics-http
      scheme: http
      scrapeTimeout: 30s
  namespaceSelector:
    matchNames:
      - zero-trust-workload-identity-manager
  selector:
    matchLabels:
      name: zero-trust-workload-identity-manager

Create the ServiceMonitor CR by running the following command:

$ oc apply -f servicemonitor-zero-trust-workload-identity-manager-http.yaml

Create the following ServiceMonitor CR if the metrics server is configured to use https.

Create the servicemonitor-zero-trust-workload-identity-manager-https.yaml YAML file:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    name: zero-trust-workload-identity-manager
  name: zero-trust-workload-identity-manager-metrics-monitor
  namespace: zero-trust-workload-identity-manager
spec:
  endpoints:
    - authorization:
        credentials:
          name: zero-trust-workload-identity-manager-metrics-auth
          key: token
        type: Bearer
      interval: 60s
      path: /metrics
      port: metrics-https
      scheme: https
      scrapeTimeout: 30s
      tlsConfig:
        ca:
          configMap:
            name: openshift-service-ca.crt
            key: service-ca.crt
        serverName: zero-trust-workload-identity-manager-metrics-service.zero-trust-workload-identity-manager.svc.cluster.local
  namespaceSelector:
    matchNames:
      - zero-trust-workload-identity-manager
  selector:
    matchLabels:
      name: zero-trust-workload-identity-manager

Create the ServiceMonitor CR by running the following command:
```
$ oc apply -f servicemonitor-zero-trust-workload-identity-manager-https.yaml
```
After the ServiceMonitor CR is created, the user workload Prometheus instance begins metrics collection from the SPIRE Server. The collected metrics are labeled with job="zero-trust-workload-identity-manager-metrics-service".

Verification

In the Red Hat OpenShift Container Platform web console, navigate to Observe → Targets.
In the Label filter field, enter the following label to filter the metrics targets:
```
$ service=zero-trust-workload-identity-manager-metrics-service
```
Confirm that the Status column shows Up for the zero-trust-workload-identity-manager entry.