Modelplane Modelplane docs
Version
main v0.4 Latest v0.3 v0.2

ServingStack Custom Resource

On this page
This document is for an unreleased version of Modelplane.

This document applies to the Modelplane main branch and not to the latest release v0.4.

A ServingStack installs the serving substrate on a Kubernetes cluster: LeaderWorkerSet (Standard) or Grove and the KAI Scheduler (Dynamo), plus Gateway API, cert-manager, and Prometheus.

#Metadata

API version
infrastructure.modelplane.ai/v1alpha1
Kind
ServingStack
Scope
Namespaced
Short names
ss

#Example

Manifest
apiVersion: infrastructure.modelplane.ai/v1alpha1
kind: ServingStack
metadata:
  name: west-gke-stack
  namespace: platform
spec:
  cloud: GKE
  secrets:
    - type: Kubeconfig
      name: west-gke-kubeconfig
      key: kubeconfig
    - type: GoogleApplicationCredentials
      name: west-gke-sa-key
      key: private_key
  # Projected from the InferenceCluster: the name InferenceGateways reach this
  # cluster's gateway by, and the CAs whose client certificates it accepts.
  gateway:
    hostname: gateway-west-gke-1a2b3.modelplane-system.svc.cluster.local
    clientCAs:
      - name: eu
        certificate: |
          -----BEGIN CERTIFICATE-----
          MIIBszCCAVmgAwIBAgIQ…
          -----END CERTIFICATE-----

#Spec

ServingStackSpec defines the desired state of ServingStack.

# cloud required enum: GKE | EKS | AKS | Nebius | Vultr | Existing

The cloud the target cluster runs on. Selects the fixed set of components and versions this stack installs there, which is resolved per cloud at build time and changes only with a Modelplane release. Mirrors InferenceCluster.spec.cluster.source; the cluster composition sets it.

# gateway required object

Configuration for the cluster’s inference traffic gateway.

# className optional string 1–63 chars default: envoy

GatewayClass name. Override if the cluster already has a GatewayClass named envoy.

# clientCAs optional object[] ≤ 32 items
# certificate required string 1–16384 chars

The CA certificate, PEM encoded.

# name required string 1–253 chars

The InferenceGateway this CA belongs to.

# hostname required string 1–253 chars

The name this cluster’s gateway is reached by, projected from the InferenceCluster. The gateway serves a certificate for it, so an InferenceGateway can originate TLS and know it reached the right cluster.

# secrets required object[] 1–8 items
# key required string ≤ 253 chars

Key within the Secret that holds the credential data.

# name required string ≤ 253 chars

Name of the Secret.

# namespace optional string ≤ 253 chars

Namespace of the Secret, when it isn’t this ServingStack’s namespace. Set on cloud identity entries whose credential is the Secret the cloud provider’s ProviderConfig references.

# type required enum: Kubeconfig | GoogleApplicationCredentials | AWSWebIdentityCredentials | NebiusServiceAccountCredentials

The type of credential this secret contains. Kubeconfig is required. Any other value is a cloud identity type; when present, the serving stack authenticates to the cluster as that identity instead of using the kubeconfig’s embedded credentials.

# stack optional enum: Standard | Dynamo default: Standard

Which serving stack this installs. Standard (the default) is the Modelplane-composed serving layer: a Deployment or LeaderWorkerSet, Gateway API, and the endpoint picker. Dynamo swaps in NVIDIA’s components: Grove with the KAI Scheduler for multi-node gang scheduling, and a shared ModelExpress server for weight distribution. Propagated from the InferenceCluster.

#Status

# gateway optional object

Status of the cluster’s inference gateway.

# address optional string ≤ 256 chars

The gateway’s external address, once assigned by the cloud load balancer.

# caCertificate optional string ≤ 16384 chars

PEM certificate of the CA that signed this gateway’s serving certificate. An InferenceGateway validates the gateway’s serving certificate against it.