Deploy Agents

Deploy a registered agent as a running service and invoke it through the Agents gateway. An agent can run in one of three modes — as a local subprocess (the default), or as a durable container on Docker or Kubernetes.

$
# Configure CLI (if not already done)
$
nemo config set --base-url "$NMP_BASE_URL" --workspace default

Deployment Modes

nemo agents deploy --mode <mode> selects the runtime backend for a deployment. The default is subprocess; docker and k8s run the agent as a durable container through the deployments plugin.

In every mode the agent is reached the same way — through the Agents gateway, which resolves the agent’s active deployment and proxies the request, so clients do not need to know which mode the agent runs in.

Agent Environments

An agent config declares what an agent needs (MCP servers, environment variables, and the names of the credentials it reads) without pinning any of it to a place to run. An agent environment supplies those specifics for a given context. This allows one agent to be reused across different contexts. For instance, a single agent can run with dev credentials and full tool access in one environment, and with more restricted credentials, read-only tools, and a larger compute box in another environment.

An environment is built from three resources:

Secrets are referenced by name (workspace/secret-name) and resolved from the platform Secrets service at run time, so credential values never live in the agent config or the environment spec.

Example

Define the Agent

The agent config declares a github MCP server and the env var it reads its credential from. It holds no secret and names no environment. See Agent Definition for the full agent.yaml reference.

1
config_format: nemo-agents-spec-v1
2
name: research-agent
3
description: Researches a GitHub repo and summarizes its dependencies and risks.
4
5
instructions:
6
  system:
7
    content: |
8
      You are a repository research assistant. Use the github tools to read the
9
      repo and summarize what it does, its main dependencies, and any risks.
10
11
default_harness: deepagents
12
harnesses:
13
  deepagents:
14
    kind: deepagents
15
    settings:
16
      deepagents: {}
17
18
models:
19
  default:
20
    provider: nvidia
21
    model: nvidia-nemotron-3-nano-30b-a3b
22
    api_key_env: NVIDIA_API_KEY
23
24
mcp:
25
  servers:
26
    github:
27
      transport: stdio
28
      url: docker
29
      args: ["run", "-i", "--rm", "-e", "GITHUB_PERSONAL_ACCESS_TOKEN", "ghcr.io/github/github-mcp-server"]

Register it:

$
nemo agents create --name research-agent --agent-config research-agent/agent.yaml

Create an Environment

The environment fulfills the agent’s github server with a stored secret and a tool scope, and provides an inline compute size. Create the secret first, then the environment-spec, then the environment that references it.

$
printf '%s' "$GITHUB_PAT" | nemo secrets create research-github-pat --from-file -
$
$
nemo agents environment-specs create research \
>
  --spec '{"provider":"local","env":{"LOG_LEVEL":"debug"},"mcp":{"github":{"url":"docker","env":{"GITHUB_TOOLSETS":"repos,issues"},"secrets":{"GITHUB_PERSONAL_ACCESS_TOKEN":"default/research-github-pat"}}}}'
$
$
nemo agents environments create research \
>
  --environment-spec default/research \
>
  --spec '{"compute_spec":{"resources":{"limits":{"cpu":"2","memory":"4Gi"}}}}'

Reference the Environment in an AgentDeployment

Deploy the agent under the environment. Its spec is merged into the agent config, and its secret refs and compute are snapshotted onto the deployment. Where both set the same field, the environment-spec’s value wins.

$
nemo agents deploy \
>
  --agent research-agent \
>
  --name research-deployment \
>
  --environment default/research
$
$
nemo agents deployments get research-deployment

Subprocess Mode (Default)

The simplest path: the platform launches a FastAPI server for the agent on its own host, assigns a port, watches its health, and tears it down on nemo agents undeploy. No image or executor configuration is required.

$
# Confirm that the local Platform instance is ready
$
export NMP_BASE_URL=http://localhost:8080
$
$
curl -fsS --connect-timeout 2 --max-time 5 \
>
  "$NMP_BASE_URL/health/ready" >/dev/null || {
>
  echo "NeMo Platform is not ready at $NMP_BASE_URL"
>
  exit 1
>
}
$
$
# Register the calculator agent from agent.yaml
$
nemo agents create \
>
  --name calculator-agent \
>
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml
$
$
# Deploy as a local subprocess (waits for "running" by default)
$
nemo agents deploy \
>
  --agent calculator-agent \
>
  --name calculator-agent-deployment \
>
  --mode subprocess
$
$
# Invoke through the Agents gateway
$
nemo agents invoke \
>
  --agent-deployment calculator-agent-deployment \
>
  --input "What is 12 multiplied by 8?"

Container Modes (Docker / Kubernetes)

Container modes give an agent a durable deployment that survives a platform restart. Instead of a local process, the platform compiles the agent into a generic deployment and hands it to the deployments plugin, which runs it on the configured executor (Docker or Kubernetes) and projects the running container’s address back onto the agent deployment. The Agents gateway then routes to that projected address.

Prerequisites

1. A container image for the agent. Container modes run a packaged agent runtime with the selected harness adapters and the agent’s dependencies. Build one with nemo agents package; the command detects nemo-agents-spec-v1 and selects the Platform agent image pipeline automatically. Image building requires the container extra. Install it with:

$
uv sync --package nemo-agents-plugin --extra container

The examples below use the calculator agent that ships with the source checkout. Run them from the repository root. Its config is located at plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml.

$
# Build the calculator agent image and tag it locally
$
nemo agents package \
>
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
>
  --tag calculator-agent:local
$
$
# For k8s, publish to a registry your cluster can pull from
$
nemo agents package \
>
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
>
  --tag calculator-agent:1.0.0 \
>
  --publish \
>
  --registry <your-registry>

2. A configured deployments executor. The platform operator defines named executors in the platform configuration and points the agents plugin at them. A minimal Docker + Kubernetes configuration:

1
agents:
2
  deployments:
3
    # Names below must match deployments.executors[].name
4
    docker_executor: local-docker
5
    k8s_executor: k8s-local
6
    # Optional: the image used when --image is omitted
7
    default_image: ""
8
    # Container port the packaged agent runtime binds (and the readiness probe target)
9
    container_port: 8000
10
11
deployments:
12
  default_executor: local-docker
13
  executors:
14
    - name: local-docker
15
      backend: docker
16
      config:
17
        # Set false when running locally-built (not registry) images
18
        pull_images: false
19
    - name: k8s-local
20
      backend: k8s
21
      config:
22
        # Omit kubeconfig_path to use in-cluster ServiceAccount auth
23
        default_namespace: default

Deploy on Docker

$
# Run from the repository root (see Prerequisites)
$
nemo agents create \
>
  --name calculator-agent \
>
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml
$
$
# --mode docker compiles to the deployments plugin's docker executor
>
nemo agents deploy \
>
  --agent calculator-agent \
>
  --name calculator-agent-docker \
>
  --mode docker \
>
  --image calculator-agent:local
>
>
# Reached through the Agents gateway, exactly like subprocess mode
>
nemo agents invoke \
>
  --agent-deployment calculator-agent-docker \
>
  --input "What is 12 multiplied by 8?"

Deploy on Kubernetes

Deployment is identical apart from --mode k8s. The deployments plugin creates a Kubernetes Deployment and a ClusterIP Service, and projects the Service’s in-cluster DNS address (<service>.<namespace>.svc.cluster.local:<port>) onto the agent deployment. Because the Agents gateway runs in-cluster, it routes to that address directly.

$
# Run from the repository root (see Prerequisites)
$
nemo agents create \
>
  --name calculator-agent \
>
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml
$
$
nemo agents deploy \
>
  --agent calculator-agent \
>
  --name calculator-agent-k8s \
>
  --mode k8s \
>
  --image <registry>/calculator-agent:1.0.0
$
$
nemo agents invoke \
>
  --agent-deployment calculator-agent-k8s \
>
  --input "What is 12 multiplied by 8?"

Model Access from a Deployed Agent

Regardless of mode, model traffic from inside the agent routes back through the Inference Gateway. The platform injects the gateway URL when it deploys the agent, and the gateway resolves model entity names to upstream providers and supplies their credentials. Two conventions apply to agent.yaml:

• Set models.default.model to the Inference Gateway entity name. The models controller creates these names by replacing slashes and dots with hyphens (nvidia/nemotron-3-nano-30b-a3b becomes nvidia-nemotron-3-nano-30b-a3b).
• Leave base_url unset for a Platform-routed model. When provider is nvidia, openai, or openai-compatible, the deployment supplies the Inference Gateway URL. api_key_env names the environment variable expected by the selected harness; it does not contain a credential.

The calculator agent uses:

1
models:
2
  default:
3
    provider: nvidia
4
    model: nvidia-nemotron-3-nano-30b-a3b
5
    api_key_env: NVIDIA_API_KEY

To make an external model available to the agent, register a provider first — see Deploy Models for NVIDIA Build, OpenAI, and Anthropic examples.

Docker mode on Linux

On Linux, host.docker.internal doesn’t resolve inside containers, so agent invokes can fail with openai.APIConnectionError. Point the deployment at the Docker bridge address (172.17.0.1 by default) in config.yaml, then start the platform bound to all interfaces:

1
agents:
2
  deployments:
3
    gateway_url_override: http://172.17.0.1:8080

$
nemo services run --host 0.0.0.0 --port 8080

If your Docker bridge uses a non-default subnet, substitute its gateway address (docker network inspect bridge --format '{{ (index .IPAM.Config 0).Gateway }}').

Inspect a Deployment

$
# Block until the deployment is running or failed
$
nemo agents deployments wait --agent calculator-agent
$
$
# List / inspect deployments
$
nemo agents deployments list
$
nemo agents deployments get calculator-agent-deployment

For a container-mode deployment, the deployment reports deployment_mode (docker or k8s), a status of running once ready, and an endpoints list carrying the container’s routable address. Subprocess deployments carry a loopback endpoint instead. The Agents gateway uses whichever the deployment’s mode provides, so invocation is identical across modes.

Deployment Cleanup

$
# Stop the running deployment (removes the process/container/k8s objects)
$
nemo agents undeploy calculator-agent-deployment
$
$
# Remove the agent entity
$
nemo agents delete calculator-agent