> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Install Gateway API Routing

Complete this installation only if you plan to use the Gateway API routing topology. The default
Dynamo Frontend topology works without Gateway API, the Gateway API Inference Extension (GAIE), or
an Endpoint Picker Plugin (EPP). To use the default topology, continue to
[KV-Aware Routing with the Dynamo Frontend](/dynamo/dev/kubernetes/model-deployment/kv-aware-routing/dynamo-frontend).

Install this integration after the [Dynamo platform](/dynamo/dev/kubernetes/installation/install-dynamo). The Dynamo operator needs
the GAIE `InferencePool` API before it can reconcile a `DynamoGraphDeployment` with an EPP component.

#### Choose a Gateway implementation

Dynamo requires a Gateway implementation that supports GAIE `InferencePool` resources and calls the
configured EPP before forwarding a request.

* Use **agentgateway** when the cluster does not already standardize on Istio or when you want a
  focused AI gateway deployment.
* Use **Istio** when the cluster already relies on Istio for ingress, service-mesh policy, or
  telemetry.

Other Gateway implementations can work when they support the same GAIE endpoint-picker contract.

#### Set the installation variables

Set the namespace where the Gateway and model routes will live. Keep the Dynamo version aligned with
the platform installation.

```bash
export NAMESPACE=my-model
export DYNAMO_SYSTEM_NAMESPACE=dynamo-system
export DYNAMO_VERSION=1.3.0
export AGW_NAMESPACE=agentgateway-system
```

#### Install the Gateway API layer

#### agentgateway

From the root of a Dynamo source checkout, run the repository installation script:

```bash
./deploy/inference-gateway/scripts/install_gaie_crd_agentgateway.sh
```

The script installs:

* Gateway API CRDs.
* GAIE CRDs.
* agentgateway with the inference extension enabled.
* An `AgentgatewayParameters` resource that prevents Istio sidecar injection into Gateway proxy pods.
* A `Gateway` named `inference-gateway` in `$NAMESPACE`.

The script pins compatible dependency versions. Review it before running it when your platform team
owns Gateway API or agentgateway lifecycle.

#### Istio

Install Gateway API and GAIE CRDs, then install Istio with the Gateway API Inference Extension
enabled. Follow the upstream
[Istio inference extension installation](https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api-inference-extension/)
for the Istio version approved by your platform.

Create the model namespace and an Istio Gateway:

```bash
kubectl create namespace "$NAMESPACE" \
  --dry-run=client -o yaml | kubectl apply -f -

cat <<EOF | kubectl apply -n "$NAMESPACE" -f -
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: inference-gateway
spec:
  gatewayClassName: istio
  listeners:
  - name: http
    port: 80
    protocol: HTTP
EOF
```

If Istio sidecars mediate traffic from the Gateway proxy to the EPP Service, enable operator-managed
`DestinationRule` generation:

```bash
helm upgrade -i dynamo-platform \
  oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform \
  --version "$DYNAMO_VERSION" \
  --namespace "$DYNAMO_SYSTEM_NAMESPACE" \
  --reuse-values \
  --set dynamo.serviceMesh.enabled=true \
  --set dynamo.serviceMesh.provider=istio
```

#### Verify the required APIs

Confirm that the cluster serves the Gateway, route, and inference-pool resources:

```bash
kubectl api-resources | grep -E 'gateways|httproutes|inferencepools'
kubectl get gatewayclass
```

Expected API groups include:

```text
gateway.networking.k8s.io
inference.networking.k8s.io
```

#### Verify the Gateway

Wait for the `inference-gateway` resource to become programmed:

```bash
kubectl wait gateway/inference-gateway -n "$NAMESPACE" \
  --for=condition=Programmed --timeout=180s

kubectl get gateway inference-gateway -n "$NAMESPACE"
```

If `Programmed` remains false, inspect the Gateway and the controller for your implementation:

```bash
kubectl describe gateway inference-gateway -n "$NAMESPACE"
kubectl get pods --all-namespaces | grep -E 'agentgateway|istio'
```

## Next Step

Add an EPP component and route an existing DGD with
[Route Requests with Gateway API](/dynamo/dev/kubernetes/model-deployment/kv-aware-routing/gateway-api).