Red Hat build of MicroShift

Choose the artificial intelligence (AI) model that best aligns with your edge application and the decisions that need to be made at MicroShift deployment sites.

Develop, train, and test your model in your cloud or data center.

Plan for the system requirements and additional resources your AI model requires to run.

Configure your Red Hat Device Edge for the specific hardware your deployment runs on, including driver and device plugins.

To enable GPU or other hardware accelerators for MicroShift, follow the guidance specific for your edge device about what you need to install. For example, to use an NVIDIA GPU accelerator, begin by reading the following NVIDIA documentation: Running a GPU-Accelerated Workload on Red Hat Device Edge (NVIDIA documentation).

For troubleshooting, consult the device documentation or product support.

Using only a driver and device plugin instead of an Operator might be more resource-efficient.

Install the microshift-ai-model-serving RPM package.

Restart MicroShift if you are adding the RPM while MicroShift is running.

Package your AI model into an OCI image, otherwise known as the ModelCar format. If you already have S3-compatible storage or a persistent volume claim set up, you can skip this step, but only the ModelCar format is tested and supported for MicroShift.

Select a model-serving runtime, which acts as your model server. Configure the runtime with the serving runtime and inference service.

Optional: Create a Route object so that your model can connect outside the node.

Make requests against the model server. For example, another pod running in your MicroShift deployment that is attached to a camera can stream an image back to the model-serving runtime. The model-serving runtime prepares that image as data for model inferencing. If the model was trained in the binary identification of a bee, the AI model outputs the likelihood that the image data is a bee.