Getting Started

Kubernetes Quickstart

Get a model running on Kubernetes and send your first request.
View as Markdown

Select NVIDIA GPU or Intel GPU based on the accelerator hardware in your cluster, and follow that tab throughout the guide. The Intel GPU path uses the XPU runtime and tooling.

1

Check the prerequisites

You need:

  • A Kubernetes v1.30 or later cluster with NVIDIA GPU nodes
  • kubectl v1.30 or later configured for the cluster
  • Helm v3 or later
2

Install accelerator support

Install the NVIDIA GPU Operator:

$helm repo add nvidia https://helm.ngc.nvidia.com/nvidia --force-update
$helm repo update nvidia
$helm upgrade --install gpu-operator nvidia/gpu-operator \
>  --namespace gpu-operator \
>  --create-namespace \
>  --wait \
>  --timeout=10m

If your cluster provider installs the NVIDIA driver, add --set driver.enabled=false. Add --set toolkit.enabled=false only when the provider also configures the GPU container runtime.

3

Install Dynamo

Install the Dynamo platform chart in a dedicated namespace. Helm creates the namespace for you:

$export NAMESPACE=dynamo-system
$export DYNAMO_VERSION=1.3.0
$
$helm upgrade --install dynamo-platform \
>  oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform \
>  --version "$DYNAMO_VERSION" \
>  --namespace "$NAMESPACE" \
>  --create-namespace \
>  --wait \
>  --timeout=10m
$
$kubectl get pods --namespace "$NAMESPACE"
4

Create the model access secret

Qwen/Qwen3-0.6B is public and does not require a Hugging Face token. The DGDR profiler and example DGD expect a Secret named hf-token-secret, so create it with an empty token:

$kubectl create secret generic hf-token-secret \
>  --namespace "$NAMESPACE" \
>  --from-literal=HF_TOKEN= \
>  --dry-run=client -o yaml | kubectl apply -f -

For a gated or private model, replace the empty value with your Hugging Face token.

5

Deploy Qwen3 0.6B

Apply a DynamoGraphDeploymentRequest (DGDR). The DGDR profiles the model and creates the DynamoGraphDeployment (DGD) that serves it.

$cat <<EOF | kubectl apply --namespace "$NAMESPACE" -f -
$apiVersion: nvidia.com/v1beta1
$kind: DynamoGraphDeploymentRequest
$metadata:
$  name: qwen3-quickstart
$spec:
$  model: Qwen/Qwen3-0.6B
$  backend: auto
$  image: nvcr.io/nvidia/ai-dynamo/dynamo-planner:${DYNAMO_VERSION}
$EOF
6

Wait for the deployment

Model download and startup can take several minutes.

$kubectl wait \
>  --for=jsonpath='{.status.phase}'=Deployed \
>  dgdr/qwen3-quickstart \
>  --namespace "$NAMESPACE" \
>  --timeout=30m
$
$export DGD_NAME=$(kubectl get dgdr/qwen3-quickstart \
>  --namespace "$NAMESPACE" \
>  --output=jsonpath='{.status.dgdName}')
$export FRONTEND_SERVICE="${DGD_NAME}-frontend"
7

Send a request

Port-forward the Frontend, wait for its health endpoint, and send a chat completion request:

$kubectl port-forward \
>  --namespace "$NAMESPACE" \
>  "service/${FRONTEND_SERVICE}" 8000:8000 \
>  >/tmp/dynamo-port-forward.log 2>&1 &
$export PORT_FORWARD_PID=$!
$
$until curl --silent --fail http://localhost:8000/health >/dev/null; do
$  sleep 2
$done
$
$curl --silent --show-error http://localhost:8000/v1/chat/completions \
>  --header "Content-Type: application/json" \
>  --data '{
>    "model": "Qwen/Qwen3-0.6B",
>    "messages": [
>      {"role": "user", "content": "What is NVIDIA Dynamo?"}
>    ],
>    "max_tokens": 200
>  }' | python3 -m json.tool

What You Did

  • Installed the accelerator support required by your hardware.
  • Installed the Dynamo platform on your Kubernetes cluster.
  • Created either a DGDR that generated a DGD, or a DGD directly.
  • Sent an OpenAI-compatible request to the deployed model.

Clean Up

Stop the port-forward and remove the quickstart deployment:

$kill "$PORT_FORWARD_PID"
$kubectl delete dgdr/qwen3-quickstart --namespace "$NAMESPACE" --ignore-not-found
$kubectl delete dgd/"$DGD_NAME" --namespace "$NAMESPACE" --ignore-not-found