> This page is for version 0.5.0.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo-helix/latest/llms.txt
> - 0.8.0: https://docs.nvidia.com/nemo-helix/v0.8.0/llms.txt
> - 0.7.1: https://docs.nvidia.com/nemo-helix/v0.7.1/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo-helix/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo-helix/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo-helix/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo-helix/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo-helix/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo-helix/v0.3.0/llms.txt
> - 0.2.0: https://docs.nvidia.com/nemo-helix/v0.2.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-helix/_mcp/server.

# Deploy Agents

Deploy a registered agent as a running service and invoke it through the
Agents gateway. An agent can run in one of three modes — as a local
subprocess (the default), or as a durable container on Docker or Kubernetes.

> **Note**
>
> Resource names for agents and deployments must contain only letters (a-z,
> A-Z), digits (0-9), underscores, hyphens, and dots. For example:
> `calculator-agent`, `my-agent`, `react-agent`.

The CLI and Python SDK connect to the deployment in your current CLI context. To connect to a local or remote deployment, see [Connect with CLI Contexts](/documentation/reference/cli-reference/connect-to-deployments).

#### CLI

```bash
# Confirm the deployment and workspace in the current context
nemo config view
```

#### Python SDK

```python
from nemo_helix_plugin.client.client import NemoClient

client = NemoClient.from_config()
```

> **Note**
>
> These examples use the `default` workspace. If your current CLI context uses a different workspace, run `nemo config set --workspace default` first. To check, run `nemo config view`.

---

## Deployment Modes

`nemo agents deploy --mode <mode>` selects the runtime backend for a
deployment. The default is `subprocess`; `docker` and `k8s` run the agent as a
durable container through the deployments plugin.

| Mode                   | Runs as                                     | Survives a platform restart | Requires                                             |
| ---------------------- | ------------------------------------------- | --------------------------- | ---------------------------------------------------- |
| `subprocess` (default) | A local FastAPI server on the platform host | No                          | Nothing extra                                        |
| `docker`               | A Docker container                          | Yes                         | A container image and a configured `docker` executor |
| `k8s`                  | A Kubernetes Deployment + Service           | Yes                         | A container image and a configured `k8s` executor    |

In every mode the agent is reached the same way — through the Agents gateway,
which resolves the agent's active deployment and proxies the request, so
clients do not need to know which mode the agent runs in.

---

## Subprocess Mode (Default)

The simplest path: the platform launches a FastAPI server for the
agent on its own host, assigns a port, watches its health, and tears it down
on `nemo agents undeploy`. No image or executor configuration is required.

#### CLI

```bash
# Confirm that the local Platform instance is ready
export NMP_BASE_URL=http://localhost:8080

curl -fsS --connect-timeout 2 --max-time 5 \
  "$NMP_BASE_URL/health/ready" >/dev/null || {
  echo "NeMo Helix is not ready at $NMP_BASE_URL"
  exit 1
}

# Register the calculator agent from agent.yaml
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

# Deploy as a local subprocess (waits for "running" by default)
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-deployment \
  --mode subprocess

# Invoke through the Agents gateway
nemo agents invoke \
  --agent-deployment calculator-agent-deployment \
  --input "What is 12 multiplied by 8?"
```

#### Python SDK

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

# Register the agent from an agent.yaml
with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

# Deploy as a local subprocess
client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-deployment",
)

# Invoke through the Agents gateway
response = client.agents.invoke(
    deployment="calculator-agent-deployment",
    input="What is 12 multiplied by 8?",
)
```

---

## Container Modes (Docker / Kubernetes)

Container modes give an agent a durable deployment that survives a platform
restart. Instead of a local process, the platform compiles the agent into a
generic deployment and hands it to the **deployments plugin**, which runs it on
the configured executor (Docker or Kubernetes) and projects the running
container's address back onto the agent deployment. The Agents gateway then
routes to that projected address.

### Prerequisites

**1. A container image for the agent.** Container modes run a packaged agent
runtime with the selected harness adapters and the agent's dependencies. Build
one with `nemo agents package`; the command detects `nemo-agents-spec-v1` and
selects the Platform agent image pipeline automatically. Image building
requires the `container` extra. Install it with:

```bash
uv sync --package nemo-agents-plugin --extra container
```

The examples below use the calculator agent that ships with the source
checkout. Run them from the repository root. Its config is located at
`plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml`.

#### CLI

```bash
# Build the calculator agent image and tag it locally
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:local

# For k8s, publish to a registry your cluster can pull from
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:1.0.0 \
  --publish \
  --registry <your-registry>
```

> **Note**
>
> `--mode docker` needs an image the platform's Docker daemon can run.
> `--mode k8s` needs an image the cluster nodes can pull (a registry image, or an
> image pre-loaded onto the nodes) — the k8s backend does not use image pull
> secrets. Pass the image with `--image`, or set
> `agents.deployments.default_image` in the platform configuration.
>
> For hand-built images that already contain an agent server, pass
> `--use-image-entrypoint` so the deployment preserves the image ENTRYPOINT/CMD
> instead of injecting the platform-packaged agent server command.
> The image must bind `0.0.0.0:$PORT`, serve `/health`, and read config from
> `AGENT_CONFIG_PATH` for Fabric or `NAT_CONFIG_PATH` for NAT.

**2. A configured deployments executor.** The platform operator defines named
executors in the platform configuration and points the agents plugin at them.
A minimal Docker + Kubernetes configuration:

```yaml
agents:
  deployments:
    # Names below must match deployments.executors[].name
    docker_executor: local-docker
    k8s_executor: k8s-local
    # Optional: the image used when --image is omitted
    default_image: ""
    # Container port the packaged agent runtime binds (and the readiness probe target)
    container_port: 8000

deployments:
  default_executor: local-docker
  executors:
    - name: local-docker
      backend: docker
      config:
        # Set false when running locally-built (not registry) images
        pull_images: false
    - name: k8s-local
      backend: k8s
      config:
        # Omit kubeconfig_path to use in-cluster ServiceAccount auth
        default_namespace: default
```

> **Note**
>
> The Kubernetes backend requires the `kubernetes` Python client in the platform
> image and a ServiceAccount with permission to manage Deployments, Services,
> ConfigMaps, and Pods in the target namespace. In the packaged Helm chart these
> run in the core controller, whose Role already grants those permissions.

### Deploy on Docker

#### CLI

```bash
# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

# --mode docker compiles to the deployments plugin's docker executor
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-docker \
  --mode docker \
  --image calculator-agent:local

# Reached through the Agents gateway, exactly like subprocess mode
nemo agents invoke \
  --agent-deployment calculator-agent-docker \
  --input "What is 12 multiplied by 8?"
```

#### Python SDK

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-docker",
    deployment_mode="docker",
    image="calculator-agent:local",
)

response = client.agents.invoke(
    deployment="calculator-agent-docker",
    input="What is 12 multiplied by 8?",
)
```

### Deploy on Kubernetes

Deployment is identical apart from `--mode k8s`. The deployments plugin creates
a Kubernetes Deployment and a ClusterIP Service, and projects the Service's
in-cluster DNS address (`<service>.<namespace>.svc.cluster.local:<port>`) onto
the agent deployment. Because the Agents gateway runs in-cluster, it routes to
that address directly.

#### CLI

```bash
# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-k8s \
  --mode k8s \
  --image <registry>/calculator-agent:1.0.0

nemo agents invoke \
  --agent-deployment calculator-agent-k8s \
  --input "What is 12 multiplied by 8?"
```

#### Python SDK

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-k8s",
    deployment_mode="k8s",
    image="<registry>/calculator-agent:1.0.0",
)

response = client.agents.invoke(
    deployment="calculator-agent-k8s",
    input="What is 12 multiplied by 8?",
)
```

---

## Model Access from a Deployed Agent

Regardless of mode, model traffic from inside the agent routes back through the
[Inference Gateway](/documentation/models-and-inference). The platform injects
the gateway URL when it deploys the agent, and the gateway resolves model
entity names to upstream providers and supplies their credentials. Two
conventions apply to `agent.yaml`:

* **Set `models.default.model` to the Inference Gateway entity name.** The
  models controller creates these names by replacing slashes and dots with
  hyphens (`nvidia/nemotron-3-nano-30b-a3b` becomes
  `nvidia-nemotron-3-nano-30b-a3b`).
* **Leave `base_url` unset for a Platform-routed model.** When `provider` is
  `nvidia`, `openai`, or `openai-compatible`, the deployment supplies the
  Inference Gateway URL. `api_key_env` names the environment variable expected
  by the selected harness; it does not contain a credential.

The calculator agent uses:

```yaml
models:
  default:
    provider: nvidia
    model: nvidia-nemotron-3-nano-30b-a3b
    api_key_env: NVIDIA_API_KEY
```

To make an external model available to the agent, register a provider first —
see [Deploy Models](/documentation/models-and-inference/deploy-models#add-external-providers)
for NVIDIA Build, OpenAI, and Anthropic examples.

> **Note**
>
> A deployed agent needs to reach the platform from **inside its container**. The
> deployment handles this automatically for both Docker and Kubernetes, so you
> normally don't need to configure anything. If an agent can't reach the platform,
> set `agents.deployments.gateway_url_override` to a URL that is reachable from
> inside the container.

#### Docker mode on Linux

On **Linux**, `host.docker.internal` doesn't resolve inside containers, so agent
invokes can fail with `openai.APIConnectionError`. Point the deployment at the
Docker bridge address (`172.17.0.1` by default) in `config.yaml`, then start the
platform bound to all interfaces:

```yaml
agents:
  deployments:
    gateway_url_override: http://172.17.0.1:8080
```

```bash
nemo services run --host 0.0.0.0 --port 8080
```

If your Docker bridge uses a non-default subnet, substitute its gateway address
(`docker network inspect bridge --format '{{ (index .IPAM.Config 0).Gateway }}'`).

---

## Inspect a Deployment

#### CLI

```bash
# Block until the deployment is running or failed
nemo agents deployments wait --agent calculator-agent

# List / inspect deployments
nemo agents deployments list
nemo agents deployments get calculator-agent-deployment
```

#### Python SDK

```python
import os

from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

deployment = client.agents.deployments.get("calculator-agent-deployment")
print(deployment["deployment_mode"], deployment["status"], deployment["endpoints"])
```

For a container-mode deployment, the deployment reports `deployment_mode`
(`docker` or `k8s`), a `status` of `running` once ready, and an `endpoints`
list carrying the container's routable address. Subprocess deployments carry a
loopback `endpoint` instead. The Agents gateway uses whichever the deployment's
mode provides, so invocation is identical across modes.

---

## Deployment Cleanup

#### CLI

```bash
# Stop the running deployment (removes the process/container/k8s objects)
nemo agents undeploy calculator-agent-deployment

# Remove the agent entity
nemo agents delete calculator-agent
```

#### Python SDK

```python
import os

from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

# Stop the running deployment (removes the process/container/k8s objects)
client.agents.deployments.delete("calculator-agent-deployment")

# Remove the agent entity
client.agents.delete("calculator-agent")
```