Red Hat build of MicroShift

Get your AI model ready for inference

Before querying your AI model through the API, you can get the model ready to provide answers based on the training data.

The following examples continue with the OVMS model.

Prerequisites

MicroShift is running.
You have the xxd utility, which is part of the vim-common package.
You configured the model-serving runtime.
You uploaded your AI model to MicroShift.

Procedure

Download an image of a bee from the OpenVINO Model Server examples by running the following command:

$ curl -O https://raw.githubusercontent.com/openvinotoolkit/model_server/main/demos/common/static/images/bee.jpeg

Create the request data by running the following script:

IMAGE=./bee.jpeg
REQ=./request.json

# Add an inference header
echo -n '{"inputs" : [{"name": "0", "shape": [1], "datatype": "BYTES"}]}' > "${REQ}"
# Get the size of the inference header
HEADER_LEN="$(stat -c %s "${REQ}")"
# Add size of the data (image) in binary format (4 bytes, little endian)
printf "%08X" $(stat --format=%s "${IMAGE}") | sed 's/\(..\)/\1\n/g' | tac | tr -d '\n' | xxd -r -p >> "${REQ}"
# Add the data, that is, append the image to the request file
cat "${IMAGE}" >> "${REQ}"

The inference header size must be passed to OpenVINO Model Server later in the form of an HTTP header.
The OpenVINO Model Server requires 4 bytes in little endian byte order.