How Red Hat OpenShift AI works in MicroShift

Edge deployments are where data happens and decisions need to be made. You can use Red Hat OpenShift AI (Red Hat OpenShift AI Self-Managed) to integrate a fleet of MicroShift-driven edge devices into the artificial intelligence and machine learning (AI/ML) operations cycle.

MicroShift is compatible with a single-model serving platform based on the KServe component of Kubernetes. KServe is a platform that orchestrates model serving.

Red Hat OpenShift AI Self-Managed is a platform for data scientists and developers of AI/ML applications. First, use Red Hat OpenShift AI Self-Managed in the cloud or data center to develop, train, and test an AI model. Then, run your model in your edge deployments on MicroShift.

After you deploy your AI model, application data can be sent to the model, so that the model can make data-driven decisions without a human user. This is an ideal scenario for edge applications where interaction with an administrator is naturally limited.

Implemented with KServe

The KServe component includes model-serving runtimes that implement the loading of various types of model servers. These runtimes are configured with custom resources (CRs). KServe custom resource definitions (CRDs) also define the life cycle of the deployment object, storage access, and networking setup.

Specifics of using Red Hat OpenShift AI Self-Managed with MicroShift

As an edge-optimized Kubernetes deployment, MicroShift has the following limitations when using Red Hat OpenShift AI Self-Managed:

  • AI model serving on MicroShift is only available on the x86_64 architecture.

  • A subset of Red Hat OpenShift AI Self-Managed Operator components are supported on MicroShift.

  • As a single-node Kubernetes distribution, MicroShift does not support multi-model deployments. You must use the single-model serving platform.

  • You must develop the AI models you want to run on the MicroShift model-serving platform in the cloud or your data center. Using MicroShift as a development platform for AI models is not supported.

  • You must plan for any additional RAM, disk space, and storage configurations required to serve your AI model.

  • Not all model servers support the IPv6 networking protocol. Check each model server’s documentation to verify that your networking configuration is supported.

  • You must secure the exposed model server endpoint, for example, with OAUTH2.

  • ClusterServingRuntimes CRDs are not supported by Red Hat OpenShift AI Self-Managed, which means that you must copy the ServingRuntime CR shipped within the microshift-ai-model-serving RPM to your workload namespace.

  • To administer model serving on MicroShift, you must use the CLI. The Red Hat OpenShift AI Self-Managed dashboard is not supported.

↑ Up: AI | ⌂ Home: Red Hat build of MicroShift | Next: Workflow for using Red Hat OpenShift AI Self-Managed with MicroShift →