Get the model-server metrics

After making a query, you can get the model server’s metrics to identify bottlenecks, optimize resource allocation, and ensure efficient infrastructure utilization.

Note

You can alternatively configure Open Telemetry for MicroShift to get model-server metrics. See "Adding Open Telemetry to an InferenceService custom resource" for more information.
Prerequisites

  • MicroShift is running.

  • There have been enough queries to provide the metrics data you want to see.

Procedure

  • Get the Prometheus-format metrics of the model server by making a request on the /metrics endpoint by running the following command:

    $ curl "${DOMAIN}/metrics" --connect-to "${DOMAIN}::${IP}:"
    Partial example output
    # HELP ovms_requests_success Number of successful requests to a model or a DAG.
# TYPE ovms_requests_success counter
ovms_requests_success{api="KServe",interface="REST",method="ModelReady",name="ovms-resnet50"} 4
ovms_requests_success{api="KServe",interface="REST",method="ModelMetadata",name="ovms-resnet50",version="1"} 1

↑ Up: Monitor AI model performance on MicroShift | ⌂ Home: Red Hat build of MicroShift | Next: Export model-server metrics by using Open Telemetry →