> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Install Gateway API Inference Extension

> **Note**
>
> Complete this installation only if you plan to use the Gateway API routing topology. The default
> Dynamo Frontend topology works without Gateway API, the Gateway API Inference Extension (GAIE), or
> an Endpoint Picker Plugin (EPP). To use the default topology, continue to
> [the Dynamo Frontend](/dynamo/kubernetes/kv-aware-routing/using-the-dynamo-frontend).

Install this integration after the [Dynamo platform](/dynamo/kubernetes/installation/install-dynamo). The Dynamo operator needs
the GAIE `InferencePool` API before it can reconcile a `DynamoGraphDeployment` with an EPP component.

#### Choose a Gateway implementation

Dynamo requires a Gateway implementation that supports GAIE `InferencePool` resources and calls the
configured EPP before forwarding a request.

* Use **agentgateway** when the cluster does not already standardize on Istio or when you want a
  focused AI gateway deployment.
* Use **Istio** when the cluster already relies on Istio for ingress, service-mesh policy, or
  telemetry.

Other Gateway implementations can work when they support the same GAIE endpoint-picker contract.

#### Set the installation variables

Set `NAMESPACE` for the model, EPP, `InferencePool`, and `HTTPRoute`. With the agentgateway install
method the `Gateway` itself lives in `AGW_NAMESPACE` alongside its controller; the Istio method
creates the Gateway in `NAMESPACE`. Keep the Dynamo version aligned with the platform installation.

```bash
export NAMESPACE=my-model
export DYNAMO_SYSTEM_NAMESPACE=dynamo-system
export DYNAMO_VERSION=1.4.0
export AGW_NAMESPACE=agentgateway-system
```

#### Install the Gateway API layer

#### agentgateway

From the root of a Dynamo source checkout, run the repository installation script:

```bash
./deploy/inference-gateway/scripts/install_gaie_crd_agentgateway.sh
```

The script installs:

* Gateway API CRDs.
* GAIE CRDs.
* agentgateway with the inference extension enabled.
* An `AgentgatewayParameters` resource that prevents Istio sidecar injection into Gateway proxy pods.
* A `Gateway` named `inference-gateway` in `$AGW_NAMESPACE` (co-located with the agentgateway
  controller). Its `http` listener sets `allowedRoutes.namespaces.from: All`, so `HTTPRoute` objects
  in `$NAMESPACE` attach to it across namespaces.

The workload namespace `$NAMESPACE` still holds the model, EPP, `InferencePool`, and `HTTPRoute`.
Only the Gateway itself lives in `$AGW_NAMESPACE`.

The script pins compatible dependency versions. Review it before running it when your platform team
owns Gateway API or agentgateway lifecycle.

#### Istio

Install Gateway API and GAIE CRDs, then install Istio with the Gateway API Inference Extension
enabled. Follow the upstream
[Istio inference extension installation](https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api-inference-extension/)
for the Istio version approved by your platform.

Create the model namespace and an Istio Gateway:

```bash
kubectl create namespace "$NAMESPACE" \
  --dry-run=client -o yaml | kubectl apply -f -

cat <<EOF | kubectl apply -n "$NAMESPACE" -f -
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: inference-gateway
spec:
  gatewayClassName: istio
  listeners:
  - name: http
    port: 80
    protocol: HTTP
EOF
```

If Istio sidecars mediate traffic from the Gateway proxy to the EPP Service, enable operator-managed
`DestinationRule` generation:

```bash
helm upgrade -i dynamo-platform \
  "https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-${DYNAMO_VERSION}.tgz" \
  --namespace "$DYNAMO_SYSTEM_NAMESPACE" \
  --reuse-values \
  --set dynamo.serviceMesh.enabled=true \
  --set dynamo.serviceMesh.provider=istio
```

#### Verify the required APIs

Confirm that the cluster serves the Gateway, route, and inference-pool resources:

```bash
kubectl api-resources | grep -E 'gateways|httproutes|inferencepools'
kubectl get gatewayclass
```

Expected API groups include:

```text
gateway.networking.k8s.io
inference.networking.k8s.io
```

#### Verify the Gateway

The Gateway namespace depends on the install method: the agentgateway script creates it in
`$AGW_NAMESPACE`, while the Istio flow creates it in `$NAMESPACE`.

#### agentgateway

Wait for the `inference-gateway` resource to become programmed:

```bash
kubectl wait gateway/inference-gateway -n "$AGW_NAMESPACE" \
  --for=condition=Programmed --timeout=180s

kubectl get gateway inference-gateway -n "$AGW_NAMESPACE"
```

If `Programmed` remains false, inspect the Gateway and the controller:

```bash
kubectl describe gateway inference-gateway -n "$AGW_NAMESPACE"
kubectl get pods --all-namespaces | grep agentgateway
```

#### Istio

Wait for the `inference-gateway` resource to become programmed:

```bash
kubectl wait gateway/inference-gateway -n "$NAMESPACE" \
  --for=condition=Programmed --timeout=180s

kubectl get gateway inference-gateway -n "$NAMESPACE"
```

If `Programmed` remains false, inspect the Gateway and the controller:

```bash
kubectl describe gateway inference-gateway -n "$NAMESPACE"
kubectl get pods --all-namespaces | grep istio
```

## Next Step

Add an EPP component and route an existing DGD with
[GAIE with Dynamo](/dynamo/kubernetes/kv-aware-routing/using-gaie-with-dynamo).