Red Hat build of MicroShift

Verify that the model-serving runtime is ready

You can use the OpenShift CLI (oc) to verify that your model-serving runtime is ready for use by checking that the downstream generation activities are complete.

Prerequisites

You configured the ServingRuntimes CR.
You created the InferenceService CR.
You have root user access to your machine.
The OpenShift CLI (oc) is installed.

Procedure

Check that the AI model is deployed in your custom namespace by running the following command:

$ oc get -n ai-demo deployment

Example output

NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
ovms-resnet50-predictor   1/1     1            1           72s

Confirm that your deployment is in progress by running the following command:

$ oc rollout status -n ai-demo deployment ovms-resnet50-predictor

Example output

deployment "ovms-resnet50-predictor" successfully rolled out

Check that the AI model workload pod is deployed in your custom namespace by running the following command:

$ oc get -n ai-demo pod

Example output

NAME                                       READY   STATUS    RESTARTS      AGE
ovms-resnet50-predictor-6fdb566b7f-bc9k5   2/2     Running   1 (72s ago)   74s

Check for the service that KServe created by running the following command:

$ oc get svc -n ai-demo

Example output

NAME                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
ovms-resnet50-predictor   ClusterIP   None         <none>        80/TCP    119s

Next steps

Create a Route object so that your applications can reach the MicroShift node.