Skip to main content

Public preview: Kubernetes AI Toolchain Operator (KAITO) add-on for AKS

Published date: March 18, 2024

The Kubernetes AI Toolchain Operator (KAITO) add-on for AKS is now available in preview. You can now run specialized machine learning workloads like large language models (LLMs) on AKS more cost-effectively and with less manual configuration. The add-on is based on the open-source Kubernetes AI Toolchain Operator (KAITO).

Streamlined to a few steps, the AI toolchain operator add-on for AKS automates LLM deployment across available CPU and GPU resources by selecting optimally sized infrastructure for the model.

This add-on makes it possible to easily split inferencing across multiple lower-GPU count VMs, increasing the number of Azure regions where workloads can run, eliminating wait times for higher GPU-count VMs, and lowering overall cost.

You can also choose from preset models with images hosted by AKS, significantly reducing overall inference service setup time on your cluster.

Learn more.

  • Azure Kubernetes Service (AKS)
  • Features