Deploy Agents

A deployment runs a registered agent as a long-running service. Clients reach every deployment the same way, through the Agents gateway, whatever runtime it uses.

The CLI and Python SDK connect to the deployment in your current CLI context. To connect to a local or remote deployment, see Connect with CLI Contexts.

# Confirm the deployment and workspace in the current context
nemo config view

Runtimes

Select a runtime with nemo agents deploy --mode:

Every mode except subprocess runs through the deployments plugin. For when the platform itself needs Docker, see Local Docker and Executors.

--mode openshell uses agents.deployments.openshell_executor with either gateway compute driver.

Subprocess

The platform starts the agent on its own host, assigns a port, and watches its health. No image or executor configuration is needed.

nemo agents deploy --agent calculator-agent --name calculator-agent-deployment
nemo agents invoke --agent calculator-agent --input "What is 12 multiplied by 8?"

nemo agents deploy waits until the deployment is running or failed and exits non-zero on failure. Pass --no-wait to return immediately.

Docker

The bundled local configuration defines a local-docker executor and makes it the default, so --mode docker works without extra configuration. Deploy with an agent image:

nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-docker \
  --mode docker \
  --image calculator-agent:local

To use another executor, add it to deployments.executors and set agents.deployments.docker_executor to its name. A deployments.executors list in your configuration replaces the default list, so keep the executors you already use. Set agents.deployments.default_image to deploy without --image.

agents:
  deployments:
    gateway_url_override: http://172.17.0.1:8080

nemo services run --host 0.0.0.0 --port 8080

docker network inspect bridge --format '{{ (index .IPAM.Config 0).Gateway }}'

Kubernetes

Helm installs define a local-k8s executor in the release namespace and make it the default, so --mode k8s works there without extra configuration. Elsewhere, add a k8s executor and point the Agents plugin at it. A deployments.executors list replaces the default list, so keep the executors you already use:

agents:
  deployments:
    k8s_executor: k8s-local
deployments:
  executors:
    - name: local-docker               # keep existing executors
      backend: docker
      config:
        pull_images: false
    - name: k8s-local
      backend: k8s
      config:
        default_namespace: default     # omit kubeconfig_path to use in-cluster auth

nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-k8s \
  --mode k8s \
  --image <registry>/calculator-agent:1.0.0

The deployments plugin creates a Deployment and a ClusterIP Service, and the Agents gateway routes to the Service’s in-cluster address. Cluster nodes must be able to pull the image: the k8s backend does not use image pull secrets. The platform needs a ServiceAccount that can manage Deployments, Services, ConfigMaps, and Pods in the namespace; the Helm chart’s core controller already has one.

Auth-Enabled Platforms

On a platform with authentication enabled, docker and k8s deployments get an auth-proxy sidecar that authenticates the agent’s calls to the platform and the Inference Gateway. Deploy from an authenticated CLI or SDK client so the deployment can act on your behalf. The sidecar uses the platform API image by default; set deployments.auth_proxy_image if that image name does not resolve in your environment.

The sidecar forwards to platform.base_url, not to agents.deployments.gateway_url_override, so that URL must be reachable from inside the agent container. On macOS, Docker deployments rewrite a loopback platform.base_url to host.docker.internal; on Linux, set it to an address the container can reach, such as the Docker bridge. With workload token exchange enabled, a remote platform.base_url must use HTTPS; plain HTTP is accepted only for loopback.

Readiness and Startup Timeouts

docker and k8s agent containers serve /health on agents.deployments.container_port (default 8000). For agents that start slowly, raise agents.deployments.readiness_probe_timeout_seconds (default 1) and agents.deployments.readiness_probe_failure_threshold (default 12), then restart the platform and redeploy; existing containers keep their old settings. A deployment that stays in STARTING longer than deployments.controller.starting_timeout_seconds (default 3600, 0 disables it) is marked failed.

The controller measures this timeout from the start of the current uninterrupted STARTING period. A deployment that loses readiness after being READY gets a fresh timeout window. Status-message updates while it remains STARTING do not reset the window; recovering readiness ends it.

Build and Publish Agent Images

nemo agents package builds an image from an agent.yaml and the files beside it. It runs locally and needs a running Docker daemon:

nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:1.0.0 \
  --publish \
  --registry nvcr.io/my-org

The image installs only the default_harness, runs as a non-root agent user, and serves the agent on port 8000. Authenticate Docker to the registry before you publish. Pass --pyproject to install the agent’s Python project into the image.

uv build --package nemo-helix --wheel --out-dir dist
export NEMO_AGENTS_WHEEL=LATEST   # newest wheel in dist/

To build an agent that is already registered, run the build as a platform job:

nemo agents package-agent --agent calculator-agent --tag calculator-agent:1.0.0 --registry nvcr.io/my-org

The job builds from the agent’s stored config and <name>-ethos fileset on the platform host, so it needs a subprocess execution profile and, to push, a host that is logged in to the registry. Images are tagged nemo-agents/<workspace>/<tag>; read the final reference from the job’s package_result result.

Agent Environments

An agent config declares what it needs (MCP servers, environment variables, the names of the credentials it reads) without binding them to one place. An agent environment supplies those values, so the same agent can run with different credentials, tools, or compute.

Secrets are referenced as workspace/secret-name and resolved from the Secrets service, so credential values never appear in the agent config.

For an agent that declares a github MCP server:

printf '%s' "$GITHUB_PAT" | nemo secrets create research-github-pat --from-file -
nemo agents environment-specs create research \
  --spec '{"mcp":{"github":{"url":"docker","env":{"GITHUB_TOOLSETS":"repos,issues"},"secrets":{"GITHUB_PERSONAL_ACCESS_TOKEN":"default/research-github-pat"}}}}'
nemo agents environments create research \
  --environment-spec default/research \
  --spec '{"compute_spec":{"resources":{"limits":{"cpu":"2","memory":"4Gi"}}}}'
nemo agents deploy --agent research-agent --environment default/research

The environment is merged into the agent config and snapshotted onto the deployment when it is created. Where both set the same field, the environment wins. The same environments work for execute-agent jobs.

Interact with a Deployment

# One request, routed to the agent's active deployment
nemo agents invoke --agent calculator-agent --input "What is 2 to the 10th power?"
# A multi-turn session that you can resume later
nemo agents chat --agent-deployment calculator-agent-deployment --session-name calc
nemo agents sessions list --agent-deployment calculator-agent-deployment

Any OpenAI-compatible client can call the gateway directly:

curl -s "${NHX_BASE_URL:-http://localhost:8080}/apis/agents/v2/workspaces/default/agents/calculator-agent/-/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "What is 12 multiplied by 8?"}]}'

Replace agents/<agent> with deployments/<deployment> to target one deployment. The example defaults to a local platform at http://localhost:8080. If your CLI context points at a remote deployment, export its URL as NHX_BASE_URL first, and for an authenticated deployment add -H "Authorization: Bearer $(nemo auth token)".

Inspect a Deployment

nemo agents deployments list
nemo agents deployments get calculator-agent-deployment
nemo agents deployments wait --agent calculator-agent
nemo agents logs --agent calculator-agent --follow

nemo agents logs reads subprocess logs from the platform host, so run it there. For other modes, use the agent’s Logs tab in NeMo Studio. For nemo-agents-spec-v1 agents, the subprocess log holds the agent server’s startup, validation, and request output. The harness’s own output is saved per invocation under the deployment’s artifacts/.fabric/ directory on the platform host.

Update a Deployed Agent

Agent configs are immutable, so redeploy replaces the agent in one step: it validates the new config, undeploys, recreates the agent from the new config, and deploys it again with the previous runtime flags:

nemo agents redeploy --agent calculator-agent --agent-config ./calculator-agent/agent.yaml

An invalid config leaves everything untouched. If a later step fails, re-run the command to finish.

Clean Up

nemo agents undeploy calculator-agent-deployment   # or --agent calculator-agent for all of its deployments
nemo agents delete calculator-agent