> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo-platform/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-platform/_mcp/server.

# Deploy Agents

Deploy a registered agent as a running service and invoke it through the
Agents gateway. An agent can run in one of three modes — as a local
subprocess (the default), or as a durable container on Docker or Kubernetes.

Resource names for agents and deployments must contain only letters (a-z,
A-Z), digits (0-9), underscores, hyphens, and dots. For example:
`calculator-agent`, `my-agent`, `react-agent`.

#### CLI

```bash
# Configure CLI (if not already done)
nemo config set --base-url "$NMP_BASE_URL" --workspace default
```

#### Python SDK

```python
import os
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)
```

---

## Deployment Modes

`nemo agents deploy --mode <mode>` selects the runtime backend for a
deployment. The default is `subprocess`; `docker` and `k8s` run the agent as a
durable container through the deployments plugin.

| Mode                   | Runs as                                     | Survives a platform restart | Requires                                             |
| ---------------------- | ------------------------------------------- | --------------------------- | ---------------------------------------------------- |
| `subprocess` (default) | A local FastAPI server on the platform host | No                          | Nothing extra                                        |
| `docker`               | A Docker container                          | Yes                         | A container image and a configured `docker` executor |
| `k8s`                  | A Kubernetes Deployment + Service           | Yes                         | A container image and a configured `k8s` executor    |

In every mode the agent is reached the same way — through the Agents gateway,
which resolves the agent's active deployment and proxies the request, so
clients do not need to know which mode the agent runs in.

---

## Agent Environments

An agent config declares what an agent needs (MCP servers, environment
variables, and the names of the credentials it reads) without pinning any of it
to a place to run. An **agent environment** supplies those specifics for a given
context. This allows one agent to be reused across different contexts. For instance,
a single agent can run with dev credentials and full tool access in one
environment, and with more restricted credentials, read-only tools, and a larger compute box in
another environment.

An environment is built from three resources:

| Resource           | Holds                                                                                                |
| ------------------ | ---------------------------------------------------------------------------------------------------- |
| `environment-spec` | Per-server secret refs, environment variables, and tool settings merged into the agent config.       |
| `compute-spec`     | The CPU/memory request. Referenced by name, or provided inline when you create the `environment`.    |
| `environment`      | Binds an `environment-spec` and an optional `compute-spec` into one named handle you deploy against. |

Secrets are referenced by name (`workspace/secret-name`) and resolved from the
platform [Secrets service](/documentation/get-started/core-concepts/manage-secrets)
at run time, so credential values never live in the agent config or the
environment spec.

### Example

#### Define the Agent

The agent config declares a `github` MCP server and the env var it reads its
credential from. It holds no secret and names no environment. See
[Agent Definition](/documentation/agents#agent-definition) for the full
`agent.yaml` reference.

```yaml
config_format: nemo-agents-spec-v1
name: research-agent
description: Researches a GitHub repo and summarizes its dependencies and risks.

instructions:
  system:
    content: |
      You are a repository research assistant. Use the github tools to read the
      repo and summarize what it does, its main dependencies, and any risks.

default_harness: deepagents
harnesses:
  deepagents:
    kind: deepagents
    settings:
      deepagents: {}

models:
  default:
    provider: nvidia
    model: nvidia-nemotron-3-nano-30b-a3b
    api_key_env: NVIDIA_API_KEY

mcp:
  servers:
    github:
      transport: stdio
      url: docker
      args: ["run", "-i", "--rm", "-e", "GITHUB_PERSONAL_ACCESS_TOKEN", "ghcr.io/github/github-mcp-server"]
```

Register it:

#### CLI

```bash
nemo agents create --name research-agent --agent-config research-agent/agent.yaml
```

#### Python SDK

```python
import yaml

with open("research-agent/agent.yaml") as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="research-agent",
    config=config,
    config_format=config["config_format"],
)
```

#### Create an Environment

The environment fulfills the agent's `github` server with a stored secret and a
tool scope, and provides an inline compute size. Create the secret first, then
the `environment-spec`, then the `environment` that references it.

#### CLI

```bash
printf '%s' "$GITHUB_PAT" | nemo secrets create research-github-pat --from-file -

nemo agents environment-specs create research \
  --spec '{"provider":"local","env":{"LOG_LEVEL":"debug"},"mcp":{"github":{"url":"docker","env":{"GITHUB_TOOLSETS":"repos,issues"},"secrets":{"GITHUB_PERSONAL_ACCESS_TOKEN":"default/research-github-pat"}}}}'

nemo agents environments create research \
  --environment-spec default/research \
  --spec '{"compute_spec":{"resources":{"limits":{"cpu":"2","memory":"4Gi"}}}}'
```

#### Python SDK

```python
client.secrets.create(name="research-github-pat", value=os.environ["GITHUB_PAT"])

client.agents.environment_specs.create(
    name="research",
    provider="local",
    env={"LOG_LEVEL": "debug"},
    mcp={
        "github": {
            "url": "docker",
            "env": {"GITHUB_TOOLSETS": "repos,issues"},
            "secrets": {"GITHUB_PERSONAL_ACCESS_TOKEN": "default/research-github-pat"},
        }
    },
)

client.agents.environments.create(
    name="research",
    environment_spec="default/research",
    compute_spec={"resources": {"limits": {"cpu": "2", "memory": "4Gi"}}},
)
```

#### Reference the Environment in an AgentDeployment

Deploy the agent under the environment. Its spec is merged into the agent
config, and its secret refs and compute are snapshotted onto the deployment.
Where both set the same field, the environment-spec's value wins.

#### CLI

```bash
nemo agents deploy \
  --agent research-agent \
  --name research-deployment \
  --environment default/research

nemo agents deployments get research-deployment
```

#### Python SDK

```python
client.agents.deployments.create(
    agent="research-agent",
    name="research-deployment",
    environment="default/research",
)

client.agents.deployments.get("research-deployment")
```

---

## Subprocess Mode (Default)

The simplest path: the platform launches a FastAPI server for the
agent on its own host, assigns a port, watches its health, and tears it down
on `nemo agents undeploy`. No image or executor configuration is required.

#### CLI

```bash
# Confirm that the local Platform instance is ready
export NMP_BASE_URL=http://localhost:8080

curl -fsS --connect-timeout 2 --max-time 5 \
  "$NMP_BASE_URL/health/ready" >/dev/null || {
  echo "NeMo Platform is not ready at $NMP_BASE_URL"
  exit 1
}

# Register the calculator agent from agent.yaml
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

# Deploy as a local subprocess (waits for "running" by default)
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-deployment \
  --mode subprocess

# Invoke through the Agents gateway
nemo agents invoke \
  --agent-deployment calculator-agent-deployment \
  --input "What is 12 multiplied by 8?"
```

#### Python SDK

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

# Register the agent from an agent.yaml
with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

# Deploy as a local subprocess
client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-deployment",
)

# Invoke through the Agents gateway
response = client.agents.invoke(
    deployment="calculator-agent-deployment",
    input="What is 12 multiplied by 8?",
)
```

---

## Container Modes (Docker / Kubernetes)

Container modes give an agent a durable deployment that survives a platform
restart. Instead of a local process, the platform compiles the agent into a
generic deployment and hands it to the **deployments plugin**, which runs it on
the configured executor (Docker or Kubernetes) and projects the running
container's address back onto the agent deployment. The Agents gateway then
routes to that projected address.

### Prerequisites

**1. A container image for the agent.** Container modes run a packaged agent
runtime with the selected harness adapters and the agent's dependencies. Build
one with `nemo agents package`; the command detects `nemo-agents-spec-v1` and
selects the Platform agent image pipeline automatically. Image building
requires the `container` extra. Install it with:

```bash
uv sync --package nemo-agents-plugin --extra container
```

The examples below use the calculator agent that ships with the source
checkout. Run them from the repository root. Its config is located at
`plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml`.

#### CLI

```bash
# Build the calculator agent image and tag it locally
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:local

# For k8s, publish to a registry your cluster can pull from
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:1.0.0 \
  --publish \
  --registry <your-registry>
```

`--mode docker` needs an image the platform's Docker daemon can run.
`--mode k8s` needs an image the cluster nodes can pull (a registry image, or an
image pre-loaded onto the nodes) — the k8s backend does not use image pull
secrets. Pass the image with `--image`, or set
`agents.deployments.default_image` in the platform configuration.

For hand-built images that already contain an agent server, pass
`--use-image-entrypoint` so the deployment preserves the image ENTRYPOINT/CMD
instead of injecting the platform-packaged agent server command.
The image must bind `0.0.0.0:$PORT`, serve `/health`, and read config from
`AGENT_CONFIG_PATH` for Fabric or `NAT_CONFIG_PATH` for NAT.

**2. A configured deployments executor.** The platform operator defines named
executors in the platform configuration and points the agents plugin at them.
A minimal Docker + Kubernetes configuration:

```yaml
agents:
  deployments:
    # Names below must match deployments.executors[].name
    docker_executor: local-docker
    k8s_executor: k8s-local
    # Optional: the image used when --image is omitted
    default_image: ""
    # Container port the packaged agent runtime binds (and the readiness probe target)
    container_port: 8000

deployments:
  default_executor: local-docker
  executors:
    - name: local-docker
      backend: docker
      config:
        # Set false when running locally-built (not registry) images
        pull_images: false
    - name: k8s-local
      backend: k8s
      config:
        # Omit kubeconfig_path to use in-cluster ServiceAccount auth
        default_namespace: default
```

The Kubernetes backend requires the `kubernetes` Python client in the platform
image and a ServiceAccount with permission to manage Deployments, Services,
ConfigMaps, and Pods in the target namespace. In the packaged Helm chart these
run in the core controller, whose Role already grants those permissions.

### Deploy on Docker

#### CLI

```bash
# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

# --mode docker compiles to the deployments plugin's docker executor
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-docker \
  --mode docker \
  --image calculator-agent:local

# Reached through the Agents gateway, exactly like subprocess mode
nemo agents invoke \
  --agent-deployment calculator-agent-docker \
  --input "What is 12 multiplied by 8?"
```

#### Python SDK

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-docker",
    deployment_mode="docker",
    image="calculator-agent:local",
)

response = client.agents.invoke(
    deployment="calculator-agent-docker",
    input="What is 12 multiplied by 8?",
)
```

### Deploy on Kubernetes

Deployment is identical apart from `--mode k8s`. The deployments plugin creates
a Kubernetes Deployment and a ClusterIP Service, and projects the Service's
in-cluster DNS address (`<service>.<namespace>.svc.cluster.local:<port>`) onto
the agent deployment. Because the Agents gateway runs in-cluster, it routes to
that address directly.

#### CLI

```bash
# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml

nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-k8s \
  --mode k8s \
  --image <registry>/calculator-agent:1.0.0

nemo agents invoke \
  --agent-deployment calculator-agent-k8s \
  --input "What is 12 multiplied by 8?"
```

#### Python SDK

```python
import os

import yaml
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

with open(
    "plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml"
) as f:
    config = yaml.safe_load(f)
client.agents.create(
    name="calculator-agent",
    config=config,
    config_format=config["config_format"],
)

client.agents.deployments.create(
    agent="calculator-agent",
    name="calculator-agent-k8s",
    deployment_mode="k8s",
    image="<registry>/calculator-agent:1.0.0",
)

response = client.agents.invoke(
    deployment="calculator-agent-k8s",
    input="What is 12 multiplied by 8?",
)
```

---

## Model Access from a Deployed Agent

Regardless of mode, model traffic from inside the agent routes back through the
[Inference Gateway](/documentation/models-and-inference). The platform injects
the gateway URL when it deploys the agent, and the gateway resolves model
entity names to upstream providers and supplies their credentials. Two
conventions apply to `agent.yaml`:

* **Set `models.default.model` to the Inference Gateway entity name.** The
  models controller creates these names by replacing slashes and dots with
  hyphens (`nvidia/nemotron-3-nano-30b-a3b` becomes
  `nvidia-nemotron-3-nano-30b-a3b`).
* **Leave `base_url` unset for a Platform-routed model.** When `provider` is
  `nvidia`, `openai`, or `openai-compatible`, the deployment supplies the
  Inference Gateway URL. `api_key_env` names the environment variable expected
  by the selected harness; it does not contain a credential.

The calculator agent uses:

```yaml
models:
  default:
    provider: nvidia
    model: nvidia-nemotron-3-nano-30b-a3b
    api_key_env: NVIDIA_API_KEY
```

To make an external model available to the agent, register a provider first —
see [Deploy Models](/documentation/models-and-inference/tutorials/deploy-models#add-external-providers)
for NVIDIA Build, OpenAI, and Anthropic examples.

A deployed agent needs to reach the platform from **inside its container**. The
deployment handles this automatically for both Docker and Kubernetes, so you
normally don't need to configure anything. If an agent can't reach the platform,
set `agents.deployments.gateway_url_override` to a URL that is reachable from
inside the container.

#### Docker mode on Linux

On **Linux**, `host.docker.internal` doesn't resolve inside containers, so agent
invokes can fail with `openai.APIConnectionError`. Point the deployment at the
Docker bridge address (`172.17.0.1` by default) in `config.yaml`, then start the
platform bound to all interfaces:

```yaml
agents:
  deployments:
    gateway_url_override: http://172.17.0.1:8080
```

```bash
nemo services run --host 0.0.0.0 --port 8080
```

If your Docker bridge uses a non-default subnet, substitute its gateway address
(`docker network inspect bridge --format '{{ (index .IPAM.Config 0).Gateway }}'`).

---

## Inspect a Deployment

#### CLI

```bash
# Block until the deployment is running or failed
nemo agents deployments wait --agent calculator-agent

# List / inspect deployments
nemo agents deployments list
nemo agents deployments get calculator-agent-deployment
```

#### Python SDK

```python
import os

from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

deployment = client.agents.deployments.get("calculator-agent-deployment")
print(deployment["deployment_mode"], deployment["status"], deployment["endpoints"])
```

For a container-mode deployment, the deployment reports `deployment_mode`
(`docker` or `k8s`), a `status` of `running` once ready, and an `endpoints`
list carrying the container's routable address. Subprocess deployments carry a
loopback `endpoint` instead. The Agents gateway uses whichever the deployment's
mode provides, so invocation is identical across modes.

---

## Deployment Cleanup

#### CLI

```bash
# Stop the running deployment (removes the process/container/k8s objects)
nemo agents undeploy calculator-agent-deployment

# Remove the agent entity
nemo agents delete calculator-agent
```

#### Python SDK

```python
import os

from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

# Stop the running deployment (removes the process/container/k8s objects)
client.agents.deployments.delete("calculator-agent-deployment")

# Remove the agent entity
client.agents.delete("calculator-agent")
```