Red Hat build of MicroShift

Export model-server metrics by using Open Telemetry

You can export model-server metrics by using Open Telemetry if you installed the microshift-observability RPM for MicroShift.

You can alternatively get the Prometheus-format metrics of the model server by making a request on the /metrics endpoint. See "Getting the model-server metrics" for more information.

Prerequisites

You configured the ServingRuntimes CR.
You have root user access to your machine.
The OpenShift CLI (oc) is installed.
You installed the microshift-observability RPM.
Your MicroShift Open Telemetry configuration includes the Prometheus Receiver. For more information, see Prometheus Receiver.

Procedure

Add the following Open Telemetry annotation to your InferenceService custom resource:

Example InferenceService object with Open Telemetry

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: ovms-resnet50
#...
  annotations:
    prometheus.io/scrape: "true"
#...