Modelplane: The Open Source Control Plane for AI Inference Fleets¶
If you've spent any time managing AI inference workloads in production, you know the mess. One team's running vLLM on a GPU cluster. Another's using KServe on a different cluster. A third handed everything to a managed provider because the ops burden was too much. Requests route manually. Nobody's quite sure which model is running where. And when something slows down at 2am, nobody knows where to look first.
Every tool in this space — vLLM, SGLang, KServe, NVIDIA Dynamo — is excellent at what it does. But they all solve the within-cluster problem. The fleet problem — placing models across available capacity, failing over across regions, routing by cost and sovereignty, caching weights so you're not downloading 70GB every time a pod restarts — has always been something the labs and hyperscalers built privately and kept to themselves.
On June 23, Upbound shipped Modelplane: the open source version of that missing layer. It's Apache 2.0, built on Crossplane, and it's early — v0.1, developer release. But the architecture is exactly right for platform teams already running Crossplane for infrastructure, and the problem it solves is real.
What Modelplane gives you in one sentence
Declare your inference topology as Kubernetes resources. Modelplane provisions the GPU clusters, schedules replicas across them using DRA and CEL selectors, caches model weights once per cluster, and routes everything through a single OpenAI-compatible gateway.
Quick takeaways¶
- Modelplane launched June 23, 2026 as Apache 2.0 from Upbound — the team behind Crossplane and Rook, both CNCF Graduated
- It's built on Crossplane and requires it. If you're already running Crossplane for infrastructure, you're one install away
- The API splits cleanly between platform teams (fleet topology) and ML teams (model deployments) — same separation-of-concerns pattern you already apply to infra
- Inference clusters need DRA (Kubernetes v1.35+) to bind GPUs. That's the minimum requirement for the managed clusters
- Upbound plans to donate it to an open source foundation later in 2026 — this is genuinely community-first
The problem: inference is a fleet problem, and nobody had an open solution¶
Most inference tooling assumes one cluster. Your production reality is different.
Capacity is scattered across hardware types, regions, and providers. A compliance requirement pins workloads to a specific geography. Large clusters concentrate blast radius. So inference grows into a fleet — and suddenly you're managing model placement, failover, GPU utilisation, and weight distribution manually across multiple clusters with no unified control plane.
The specific failures you hit without fleet-level control:
GPU nodes idle while others are saturated. Without a scheduler that sees the whole fleet, you can't place replicas on the cluster with free capacity. You're over-provisioned in one region and throttling in another.
Weight downloads happening in parallel on every pod start. A 70B parameter model is 140GB. Without shared caching, every new replica downloads from scratch. On a cold cluster with 8 pods starting simultaneously, that's 1.1TB hitting your HuggingFace egress at once.
Manual fallback to managed providers. When your primary cluster hits capacity, someone has to update a config to route overflow to Azure OpenAI or Baseten. At 2am, that someone is you.
The silent cost
GPU nodes are billed by the hour whether or not they're serving requests. Without fleet-level scheduling, you're routinely paying for idle capacity on one cluster while another is rejecting requests. Modelplane's scheduler sees the whole fleet — it places replicas on clusters with free capacity before provisioning new nodes.
What Modelplane actually does: the API¶
Modelplane runs as a control plane on an ordinary Kubernetes cluster — your control cluster — with Crossplane installed and no GPUs of its own. The inference clusters it manages do the serving.
The API splits along the two teams that actually do this work.
Platform teams define the fleet topology: hardware shapes, clusters, and the unified gateway.
# platform.yaml — platform team owns these resources
apiVersion: modelplane.ai/v1alpha1
kind: InferenceClass # a hardware shape offered by the platform
metadata:
name: gke-l4-1x
spec:
provisioning:
provider: GKE
gke:
machineType: g2-standard-8
accelerator: { type: nvidia-l4, count: 1 }
devices:
- name: gpu
claim: DRA
driver: gpu.nvidia.com
count: 1
capacity:
memory: { value: "23034Mi" } # L4's usable VRAM
---
apiVersion: modelplane.ai/v1alpha1
kind: InferenceCluster # a cluster in the fleet
metadata:
name: starter
labels:
modelplane.ai/region: us-central
spec:
cluster:
source: GKE
gke: { project: my-gcp-project, region: us-central1 }
nodePools:
- name: gpu-l4
className: gke-l4-1x
minNodeCount: 1
maxNodeCount: 2
---
apiVersion: modelplane.ai/v1alpha1
kind: InferenceGateway # the unified, OpenAI-compatible entry point
metadata:
name: default
spec:
backend: Traefik
traefik:
version: "40.2.0"
ML teams deploy models. They describe what hardware a replica needs — Modelplane's scheduler finds the cluster and pool that fits.
# model.yaml — ML team owns these resources
apiVersion: modelplane.ai/v1alpha1
kind: ModelDeployment # the model, the engine, the replica count
metadata:
name: qwen-demo
namespace: ml-team
spec:
replicas: 1
engines:
- name: qwen
members:
- role: Standalone
nodeSelector:
devices:
- name: gpu
count: 1
selectors:
# CEL selector: any GPU with >= 20Gi VRAM
- cel: device.capacity["gpu.nvidia.com"].memory.compareTo(quantity("20Gi")) >= 0
template:
spec:
containers:
- name: engine
image: vllm/vllm-openai:v0.7.3
args: ["--model=Qwen/Qwen2.5-0.5B-Instruct"]
---
apiVersion: modelplane.ai/v1alpha1
kind: ModelService # one stable, OpenAI-compatible endpoint
metadata:
name: qwen
namespace: ml-team
spec:
endpoints:
- selector:
matchLabels:
modelplane.ai/deployment: qwen-demo
Developers never name a cluster. They describe the hardware a replica needs, and Modelplane places it. Switching engines — from vLLM to SGLang, for example — is a change to your ModelDeployment, not to Modelplane itself.
The CEL selector is the piece that ties Modelplane to DRA GA
Kubernetes v1.36 shipped Dynamic Resource Allocation to GA — which we covered here. Modelplane uses DRA + CEL expressions to match replicas to GPU pools by actual hardware attributes: memory capacity, architecture, driver version. This is exactly what DRA was designed for. If you're running K8s 1.35+, you're already halfway there.
Five things Modelplane runs continuously across the fleet¶
Once your resources exist, Modelplane reconciles five concerns across every cluster, continuously:
Provisioning. Create GPU node pools on hyperscalers and neoclouds, or bring your own cluster with source: Existing. Modelplane installs the serving stack on each cluster it manages.
Scheduling. A two-level scheduler filters first by cluster labels (region, tier, provider), then by pool hardware using DRA + CEL selectors. It accounts capacity at the node level and never overcommits. Your ML team describes what they need; Modelplane finds the fit.
Autoscaling. ModelDeployment exposes the standard Kubernetes scale subresource, so kubectl scale works, and so does a KEDA ScaledObject. There's no per-pod autoscaling inside a cluster — replicas are the only scaling axis, and each replica is a complete serving instance.
Routing. One ModelService load-balances across every replica of a deployment, wherever they run. You can include a ModelEndpoint pointing at an external provider — Baseten, Together, Azure OpenAI — and Modelplane routes overflow there automatically. Weighted traffic is built in for canary and A/B rollouts.
Caching. A ModelCache stages weights once per cluster on shared (ReadWriteMany) storage. Pods read locally on start — no parallel downloads, no race conditions on cold clusters. Optional for single-node, recommended for multi-node.
What this means for your platform if you're running Crossplane¶
This is where it gets concrete. Modelplane is built on Crossplane's function framework and uses its infrastructure providers directly. If you're already running Crossplane to manage VPCs, RDS instances, and node pools, Modelplane extends that same reconciliation pattern to the inference layer.
Your InferenceFleet manifests live in Git alongside your infrastructure Compositions. ArgoCD syncs them. The same GitOps drift detection pattern you apply to infra applies here — desired state in Git, Modelplane reconciles the fleet back to it.
# argocd-app-inference-fleet.yaml — sync inference declarations from Git
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: inference-fleet
namespace: argocd
spec:
project: platform
source:
repoURL: https://github.com/your-org/platform-config
path: inference/fleet
targetRevision: main
destination:
server: https://kubernetes.default.svc
namespace: modelplane-system
syncPolicy:
automated:
prune: true
selfHeal: true
The Internal Developer Platform angle is significant. Once a platform team owns the InferenceClass and InferenceGateway, ML teams get self-service model deployment through the ModelDeployment API — the same gated self-service pattern you'd apply to any other infrastructure capability. You control the hardware shapes on offer; they choose from what's available.
The policy-as-code hook
Because everything is Kubernetes resources, your existing Kyverno or OPA Gatekeeper policies apply. You can gate which model images are allowed, which namespaces can deploy ModelDeployments, or require specific labels before any InferenceCluster can be added to the fleet. No new policy layer required.
What's not production-ready yet¶
Modelplane is v0.1. Be clear-eyed about that. A few things to know before you evaluate it for production:
Exclusive cluster ownership. Modelplane assumes it owns every inference cluster it manages. Don't share those clusters with other workloads — it'll interfere with Modelplane's scheduling.
Provider support. Today it provisions on a handful of hyperscalers and neoclouds. Check Supported Providers before committing. The bring-your-own path (source: Existing) works on any Kubernetes now.
Control plane HA. At v0.1, treat the control cluster as a critical dependency. HA for the Modelplane control plane itself isn't documented yet — watch the GitHub issues for that milestone.
Weight caching from HuggingFace only. ModelCache hydrates from Hugging Face today. NVIDIA NGC staging is on the roadmap.
The right move right now: stand up a dev cluster, deploy the getting-started guide (it's 45 minutes from nothing to a live endpoint), and get familiar with the API before it hits GA. The project is moving fast and building in the open — this is the time to influence the direction.
Getting started
The getting-started guide is at docs.modelplane.ai/getting-started. The GitHub repo is github.com/modelplaneai/modelplane. Community Slack at slack.modelplane.ai.
FAQ¶
Does this replace vLLM or KServe?
No — it orchestrates them. Modelplane is the fleet layer above the serving engine. A ModelDeployment carries your vLLM (or SGLang, or TensorRT-LLM) container and its flags, and Modelplane places it on the right cluster. Switching engines is a change to your deployment, not to Modelplane. KServe manages within a cluster; Modelplane manages across clusters. The scope is different.
Do I need Crossplane already installed?
Yes. Modelplane is built on Crossplane and requires it — it uses Crossplane's function framework and shares its infrastructure providers. If you're already running Crossplane, you're extending an existing pattern. If you're not, you'll need to install it on your control cluster first. The good news: the control cluster is an ordinary Kubernetes cluster with no GPUs, so you're not paying for GPU capacity just to run the control plane.
What's the Upbound commercial angle?
Upbound has commercial products (Spaces, their managed control plane offering) built on top of Crossplane. Modelplane itself is Apache 2.0 with no usage caps, no token metering, and no commercial features. Upbound intends to donate it to a neutral open source foundation — same path Crossplane and Rook took. Bassam Tabbara (who built both of those) is the author.
How does inference autoscaling work with KEDA?
ModelDeployment exposes the standard Kubernetes scale subresource, which is exactly what KEDA targets. Set up a ScaledObject pointing at your queue depth or request rate, and KEDA scales the spec.replicas on your ModelDeployment. Modelplane's scheduler then places the new replicas across the fleet. No Modelplane-specific autoscaling config needed.
What about disaggregated prefill/decode for large models?
Supported. Set serving.mode: PrefillDecode on your engine and define separate prefill and decode containers. They run on the same cluster, hand off the KV cache over a fast fabric, and Modelplane configures the cluster-edge routing. The KV-transfer flags live in your engine config — Modelplane doesn't own them. Multi-node serving (LeaderWorkerSet) is also supported for models that don't fit a single GPU.
Modelplane is the project that was always going to exist — you could see the gap from a mile away. Every org running inference at scale has built some version of this privately. The interesting question is whether the open version gets there before the problem compounds further. Given that Bassam Tabbara — the person who built Crossplane and Rook, both now CNCF Graduated with production deployments at Apple, JPMorgan, and NASA — is the author, the architecture track record is solid.
Stand it up in dev now. The API is stable enough to evaluate, the 45-minute getting-started guide actually works, and the patterns you build today will carry into production.
For the Crossplane foundation this builds on: building internal platforms with Crossplane
For how ArgoCD fits into the GitOps sync pipeline: GitOps + AI Drift Detection: Catch It Before Prod
For the DRA GA context that makes Modelplane's scheduler possible: Kubernetes v1.36: You Can Finally Taint Individual GPUs