Deploy Agents

Deploy a registered agent as a running service and invoke it through the Agents gateway. An agent can run in one of three modes — as a local subprocess (the default), or as a durable container on Docker or Kubernetes.

# Configure CLI (if not already done)
nemo config set --base-url "$NMP_BASE_URL" --workspace default

Deployment Modes

nemo agents deploy --mode <mode> selects the runtime backend for a deployment. The default is subprocess; docker and k8s run the agent as a durable container through the deployments plugin.

In every mode the agent is reached the same way — through the Agents gateway, which resolves the agent’s active deployment and proxies the request, so clients do not need to know which mode the agent runs in.

Subprocess Mode (Default)

The simplest path: the platform launches a FastAPI server for the agent on its own host, assigns a port, watches its health, and tears it down on nemo agents undeploy. No image or executor configuration is required.

# Confirm that the local Platform instance is ready
export NMP_BASE_URL=http://localhost:8080
curl -fsS --connect-timeout 2 --max-time 5 \
  "$NMP_BASE_URL/health/ready" >/dev/null || {
  echo "NeMo Platform is not ready at $NMP_BASE_URL"
  exit 1
}
# Register the calculator agent from agent.yaml
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml
# Deploy as a local subprocess (waits for "running" by default)
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-deployment \
  --mode subprocess
# Invoke through the Agents gateway
nemo agents invoke \
  --agent-deployment calculator-agent-deployment \
  --input "What is 12 multiplied by 8?"

Container Modes (Docker / Kubernetes)

Container modes give an agent a durable deployment that survives a platform restart. Instead of a local process, the platform compiles the agent into a generic deployment and hands it to the deployments plugin, which runs it on the configured executor (Docker or Kubernetes) and projects the running container’s address back onto the agent deployment. The Agents gateway then routes to that projected address.

Prerequisites

1. A container image for the agent. Container modes run a packaged agent runtime with the selected harness adapters and the agent’s dependencies. Build one with nemo agents package; the command detects nemo-agents-spec-v1 and selects the Platform agent image pipeline automatically. Image building requires the container extra. Install it with:

uv sync --package nemo-agents-plugin --extra container

The examples below use the calculator agent that ships with the source checkout. Run them from the repository root. Its config is located at plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml.

# Build the calculator agent image and tag it locally
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:local
# For k8s, publish to a registry your cluster can pull from
nemo agents package \
  --agent plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml \
  --tag calculator-agent:1.0.0 \
  --publish \
  --registry <your-registry>

2. A configured deployments executor. The platform operator defines named executors in the platform configuration and points the agents plugin at them. A minimal Docker + Kubernetes configuration:

agents:
  deployments:
    # Names below must match deployments.executors[].name
    docker_executor: local-docker
    k8s_executor: k8s-local
    # Optional: the image used when --image is omitted
    default_image: ""
    # Container port the packaged agent runtime binds (and the readiness probe target)
    container_port: 8000
deployments:
  default_executor: local-docker
  executors:
    - name: local-docker
      backend: docker
      config:
        # Set false when running locally-built (not registry) images
        pull_images: false
    - name: k8s-local
      backend: k8s
      config:
        # Omit kubeconfig_path to use in-cluster ServiceAccount auth
        default_namespace: default

Deploy on Docker

# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml
# --mode docker compiles to the deployments plugin's docker executor
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-docker \
  --mode docker \
  --image calculator-agent:local
# Reached through the Agents gateway, exactly like subprocess mode
nemo agents invoke \
  --agent-deployment calculator-agent-docker \
  --input "What is 12 multiplied by 8?"

Deploy on Kubernetes

Deployment is identical apart from --mode k8s. The deployments plugin creates a Kubernetes Deployment and a ClusterIP Service, and projects the Service’s in-cluster DNS address (<service>.<namespace>.svc.cluster.local:<port>) onto the agent deployment. Because the Agents gateway runs in-cluster, it routes to that address directly.

# Run from the repository root (see Prerequisites)
nemo agents create \
  --name calculator-agent \
  --agent-config plugins/nemo-agents/examples/nemo-agent-config/calculator-agent/agent.yaml
nemo agents deploy \
  --agent calculator-agent \
  --name calculator-agent-k8s \
  --mode k8s \
  --image <registry>/calculator-agent:1.0.0
nemo agents invoke \
  --agent-deployment calculator-agent-k8s \
  --input "What is 12 multiplied by 8?"

Model Access from a Deployed Agent

Regardless of mode, model traffic from inside the agent routes back through the Inference Gateway. The platform injects the gateway URL when it deploys the agent, and the gateway resolves model entity names to upstream providers and supplies their credentials. Two conventions apply to agent.yaml:

• Set models.default.model to the Inference Gateway entity name. The models controller creates these names by replacing slashes and dots with hyphens (nvidia/nemotron-3-nano-30b-a3b becomes nvidia-nemotron-3-nano-30b-a3b).
• Leave base_url unset for a Platform-routed model. When provider is nvidia, openai, or openai-compatible, the deployment supplies the Inference Gateway URL. api_key_env names the environment variable expected by the selected harness; it does not contain a credential.

The calculator agent uses:

models:
  default:
    provider: nvidia
    model: nvidia-nemotron-3-nano-30b-a3b
    api_key_env: NVIDIA_API_KEY

To make an external model available to the agent, register a provider first — see Deploy Models for NVIDIA Build, OpenAI, and Anthropic examples.

Docker mode on Linux

On Linux, host.docker.internal doesn’t resolve inside containers, so agent invokes can fail with openai.APIConnectionError. Point the deployment at the Docker bridge address (172.17.0.1 by default) in config.yaml, then start the platform bound to all interfaces:

agents:
  deployments:
    gateway_url_override: http://172.17.0.1:8080

nemo services run --host 0.0.0.0 --port 8080

If your Docker bridge uses a non-default subnet, substitute its gateway address (docker network inspect bridge --format '{{ (index .IPAM.Config 0).Gateway }}').

Inspect a Deployment

# Block until the deployment is running or failed
nemo agents deployments wait --agent calculator-agent
# List / inspect deployments
nemo agents deployments list
nemo agents deployments get calculator-agent-deployment

For a container-mode deployment, the deployment reports deployment_mode (docker or k8s), a status of running once ready, and an endpoints list carrying the container’s routable address. Subprocess deployments carry a loopback endpoint instead. The Agents gateway uses whichever the deployment’s mode provides, so invocation is identical across modes.

Deployment Cleanup

# Stop the running deployment (removes the process/container/k8s objects)
nemo agents undeploy calculator-agent-deployment
# Remove the agent entity
nemo agents delete calculator-agent