> This page is for version 0.8. > For other versions, use one of these documentation indexes: > - Latest (default): https://docs.nvidia.com/nemo-helix/latest/llms.txt > - 0.9: https://docs.nvidia.com/nemo-helix/v0.9/llms.txt > - 0.8: https://docs.nvidia.com/nemo-helix/v0.8/llms.txt > - 0.7: https://docs.nvidia.com/nemo-helix/v0.7/llms.txt > - 0.6: https://docs.nvidia.com/nemo-helix/v0.6/llms.txt > - 0.5: https://docs.nvidia.com/nemo-helix/v0.5/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-helix/_mcp/server. # Deploy Agents > Run a registered agent as a subprocess, Docker container, or Kubernetes Deployment, and build and publish agent images. A deployment runs a [registered agent](/nemo-helix/v0.8/documentation/agents/register-agents) as a long-running service. Clients reach every deployment the same way, through the Agents gateway, whatever runtime it uses. The CLI and Python SDK connect to the deployment in your current CLI context. To connect to a local or remote deployment, see [Connect with CLI Contexts](/nemo-helix/v0.8/documentation/reference/cli-reference/connect-to-deployments). #### CLI ```bash # Confirm the deployment and workspace in the current context nemo config view ``` #### Python SDK ```python from nemo_helix_plugin.client.client import NemoClient client = NemoClient.from_config() ``` > **Note** > > These examples use the `default` workspace. If your current CLI context uses a different workspace, run `nemo config set --workspace default` first. To check, run `nemo config view`. ## Runtimes Select a runtime with `nemo agents deploy --mode`: | Mode | Runs as | Survives a platform restart | Requires | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ------------------------------------------------------------------------------ | | `subprocess` (default) | A process on the platform host | No | The agent's harness installed in the platform environment | | `docker` | A Docker container | Yes | An agent image and a Docker executor | | `k8s` | A Kubernetes Deployment and ClusterIP Service | Yes | An image the cluster can pull and a Kubernetes executor | | `openshell` | An OpenShell sandbox whose network policy denies everything except the platform; see [Sandboxed Deployments](/nemo-helix/v0.8/documentation/agents/deploy-agents/sandboxed-deployments) | Yes | An image packaged with `--sandbox-runtime openshell` and an OpenShell executor | Every mode except `subprocess` runs through the deployments plugin. For when the platform itself needs Docker, see [Local Docker and Executors](/nemo-helix/v0.8/documentation/get-started/local-docker-and-executors). ### Subprocess The platform starts the agent on its own host, assigns a port, and watches its health. No image or executor configuration is needed. #### CLI ```bash nemo agents deploy --agent calculator-agent --name calculator-agent-deployment nemo agents invoke --agent calculator-agent --input "What is 12 multiplied by 8?" ``` #### Python SDK ```python from nemo_agents_plugin.sdk import AgentsResource from nemo_helix_plugin.client.client import NemoClient agents = AgentsResource(NemoClient.from_config()) agents.deployments.create(agent="calculator-agent", name="calculator-agent-deployment") # Once agents.deployments.get("calculator-agent-deployment")["status"] is "running": response = agents.invoke( deployment="calculator-agent-deployment", input="What is 12 multiplied by 8?", ) ``` `nemo agents deploy` waits until the deployment is `running` or `failed` and exits non-zero on failure. Pass `--no-wait` to return immediately. ### Docker The bundled local configuration defines a `local-docker` executor and makes it the default, so `--mode docker` works without extra configuration. Deploy with an [agent image](#build-and-publish-agent-images): #### CLI ```bash nemo agents deploy \ --agent calculator-agent \ --name calculator-agent-docker \ --mode docker \ --image calculator-agent:local ``` #### Python SDK ```python agents.deployments.create( agent="calculator-agent", name="calculator-agent-docker", deployment_mode="docker", image="calculator-agent:local", ) ``` To use another executor, add it to `deployments.executors` and set `agents.deployments.docker_executor` to its name. A `deployments.executors` list in your configuration replaces the default list, so keep the executors you already use. Set `agents.deployments.default_image` to deploy without `--image`. > **Note** > > On Linux, `host.docker.internal` does not resolve inside containers, so model calls from the agent > can fail with `APIConnectionError`. Point agents at the Docker bridge and start the platform on all > interfaces: > > ```yaml > agents: > deployments: > gateway_url_override: http://172.17.0.1:8080 > ``` > > ```bash > nemo services run --host 0.0.0.0 --port 8080 > ``` > > If your Docker bridge uses a different subnet, use its gateway address instead: > > ```bash > docker network inspect bridge --format '{{ (index .IPAM.Config 0).Gateway }}' > ``` ### Kubernetes Helm installs define a `local-k8s` executor in the release namespace and make it the default, so `--mode k8s` works there without extra configuration. Elsewhere, add a `k8s` executor and point the Agents plugin at it. A `deployments.executors` list replaces the default list, so keep the executors you already use: ```yaml agents: deployments: k8s_executor: k8s-local deployments: executors: - name: local-docker # keep existing executors backend: docker config: pull_images: false - name: k8s-local backend: k8s config: default_namespace: default # omit kubeconfig_path to use in-cluster auth ``` ```bash nemo agents deploy \ --agent calculator-agent \ --name calculator-agent-k8s \ --mode k8s \ --image /calculator-agent:1.0.0 ``` The deployments plugin creates a Deployment and a ClusterIP Service, and the Agents gateway routes to the Service's in-cluster address. Cluster nodes must be able to pull the image: the `k8s` backend does not use image pull secrets. The platform needs a ServiceAccount that can manage Deployments, Services, ConfigMaps, and Pods in the namespace; the Helm chart's core controller already has one. ### Auth-Enabled Platforms On a platform with authentication enabled, `docker` and `k8s` deployments get an `auth-proxy` sidecar that authenticates the agent's calls to the platform and the Inference Gateway. Deploy from an authenticated CLI or SDK client so the deployment can act on your behalf. The sidecar uses the platform API image by default; set `deployments.auth_proxy_image` if that image name does not resolve in your environment. The sidecar forwards to `platform.base_url`, not to `agents.deployments.gateway_url_override`, so that URL must be reachable from inside the agent container. On macOS, Docker deployments rewrite a loopback `platform.base_url` to `host.docker.internal`; on Linux, set it to an address the container can reach, such as the Docker bridge. With workload token exchange enabled, a remote `platform.base_url` must use HTTPS; plain HTTP is accepted only for loopback. ### Readiness and Startup Timeouts `docker` and `k8s` agent containers serve `/health` on `agents.deployments.container_port` (default `8000`). For agents that start slowly, raise `agents.deployments.readiness_probe_timeout_seconds` (default `1`) and `agents.deployments.readiness_probe_failure_threshold` (default `12`), then restart the platform and redeploy; existing containers keep their old settings. A deployment that stays in `STARTING` longer than `deployments.controller.starting_timeout_seconds` (default `3600`, `0` disables it) is marked failed. ## Build and Publish Agent Images `nemo agents package` builds an image from an `agent.yaml` and the files beside it. It runs locally and needs a running Docker daemon: | Command | Result | | -------------------------------------------------------------------------------------- | ------------------------------------------------------ | | `nemo agents package --agent --no-build` | A `Dockerfile` and `.dockerignore` beside `agent.yaml` | | `nemo agents package --agent --tag ` | A local image | | `nemo agents package --agent --tag --publish --registry ` | A local image, pushed to the registry | ```bash nemo agents package \ --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \ --tag calculator-agent:1.0.0 \ --publish \ --registry nvcr.io/my-org ``` The image installs only the `default_harness`, runs as a non-root `agent` user, and serves the agent on port 8000. Authenticate Docker to the registry before you publish. Pass `--pyproject` to install the agent's Python project into the image. > **Note** > > The image pins the installed `nemo-helix` version, which a source checkout cannot resolve from a > package index. From a checkout, build a wheel first and point packaging at it: > > ```bash > uv build --package nemo-helix --wheel --out-dir dist > export NEMO_AGENTS_WHEEL=LATEST # newest wheel in dist/ > ``` To build an agent that is already registered, run the build as a platform job: ```bash nemo agents package-agent --agent calculator-agent --tag calculator-agent:1.0.0 --registry nvcr.io/my-org ``` The job builds from the agent's stored config and `-ethos` fileset on the platform host, so it needs a `subprocess` execution profile and, to push, a host that is logged in to the registry. Images are tagged `nemo-agents//`; read the final reference from the job's `package_result` result. ## Agent Environments An agent config declares what it needs (MCP servers, environment variables, the names of the credentials it reads) without binding them to one place. An **agent environment** supplies those values, so the same agent can run with different credentials, tools, or compute. | Resource | Holds | | ------------------ | ---------------------------------------------------------------------------------------------- | | `environment-spec` | Environment variables, secret references, and MCP server settings merged into the agent config | | `compute-spec` | CPU and memory requests and limits for `docker` and `k8s` deployments | | `environment` | An `environment-spec` plus an optional `compute-spec`, deployed against by name | Secrets are referenced as `workspace/secret-name` and resolved from the [Secrets service](/nemo-helix/v0.8/documentation/get-started/core-concepts/manage-secrets), so credential values never appear in the agent config. For an agent that declares a `github` MCP server: ```bash printf '%s' "$GITHUB_PAT" | nemo secrets create research-github-pat --from-file - nemo agents environment-specs create research \ --spec '{"mcp":{"github":{"url":"docker","env":{"GITHUB_TOOLSETS":"repos,issues"},"secrets":{"GITHUB_PERSONAL_ACCESS_TOKEN":"default/research-github-pat"}}}}' nemo agents environments create research \ --environment-spec default/research \ --spec '{"compute_spec":{"resources":{"limits":{"cpu":"2","memory":"4Gi"}}}}' nemo agents deploy --agent research-agent --environment default/research ``` The environment is merged into the agent config and snapshotted onto the deployment when it is created. Where both set the same field, the environment wins. The same environments work for [execute-agent jobs](/nemo-helix/v0.8/documentation/agents/execute-agents-as-jobs). ## Interact with a Deployment ```bash # One request, routed to the agent's active deployment nemo agents invoke --agent calculator-agent --input "What is 2 to the 10th power?" # A multi-turn session that you can resume later nemo agents chat --agent-deployment calculator-agent-deployment --session-name calc nemo agents sessions list --agent-deployment calculator-agent-deployment ``` Any OpenAI-compatible client can call the gateway directly: ```bash curl -s "${NHX_BASE_URL:-http://localhost:8080}/apis/agents/v2/workspaces/default/agents/calculator-agent/-/v1/chat/completions" \ -H "Content-Type: application/json" \ -d '{"messages": [{"role": "user", "content": "What is 12 multiplied by 8?"}]}' ``` Replace `agents/` with `deployments/` to target one deployment. The example defaults to a local platform at `http://localhost:8080`. If your CLI context points at a remote deployment, export its URL as `NHX_BASE_URL` first, and for an authenticated deployment add `-H "Authorization: Bearer $(nemo auth token)"`. ## Inspect a Deployment ```bash nemo agents deployments list nemo agents deployments get calculator-agent-deployment nemo agents deployments wait --agent calculator-agent nemo agents logs --agent calculator-agent --follow ``` `nemo agents logs` reads subprocess logs from the platform host, so run it there. For other modes, use the agent's **Logs** tab in [NeMo Studio](/nemo-helix/v0.8/documentation/studio/agents). For `nemo-agents-spec-v1` agents, the subprocess log holds the agent server's startup, validation, and request output. The harness's own output is saved per invocation under the deployment's `artifacts/.fabric/` directory on the platform host. ## Update a Deployed Agent Agent configs are immutable, so `redeploy` replaces the agent in one step: it validates the new config, undeploys, recreates the agent from the new config, and deploys it again with the previous runtime flags: ```bash nemo agents redeploy --agent calculator-agent --agent-config ./calculator-agent/agent.yaml ``` An invalid config leaves everything untouched. If a later step fails, re-run the command to finish. ## Clean Up ```bash nemo agents undeploy calculator-agent-deployment # or --agent calculator-agent for all of its deployments nemo agents delete calculator-agent ```