About Agents

An agent on NeMo Platform calls tools, accesses models through shared Platform infrastructure, and runs as a service you can register, deploy, and invoke. New agents use the Platform-managed nemo-agents-spec-v1 agent.yaml contract. Legacy NVIDIA NeMo Agent Toolkit (NAT) workflow configurations remain supported, including NAT-specific evaluation and optimization commands. Both formats are managed through the nemo agents command group.

How It Works

A platform-managed agent consists of three components:

1. The agent entity. nemo agents create stores the agent YAML as an entity in the workspace. The same configuration can be redeployed without re-registering it.
2. The deployment controller. nemo agents deploy passes the stored config to the Agents service controller. By default it launches a FastAPI server, assigns a port, watches its health, and tears it down on nemo agents undeploy. With --mode docker or --mode k8s it instead runs the agent as a durable container through the deployments plugin — see Deploy Agents.
3. The Agents gateway. The gateway resolves the agent to its current running deployment and proxies client requests to it, including streaming responses. From a client’s perspective, the agent is an OpenAI-compatible endpoint owned by NeMo Platform.

Model traffic from inside the agent process routes back through the Inference Gateway, which resolves model entity names to upstream providers and supplies their credentials. This is why agent configs do not carry base_url or api_key values — the deployment injects the gateway URL automatically, and the gateway looks up the rest.

A virtual model is a platform-managed wrapper around one or more backend models. An agent can point at a virtual model entity while the virtual model handles routing, format translation, and guardrails behind the scenes — no changes to the agent’s YAML required.

Applying Changes Through Candidate Agents

The Agents v2 API has no in-place patch for a stored agent config. When the optimize or secure workflows propose a change — a model swap, a routing split, a guardrailed virtual model — they apply it by creating a sibling agent with the new config, deploying it, and running an evaluation against it. The original agent stays untouched until you decide to promote the candidate.

The sibling-candidate pattern is the recommended way to apply changes: you always have a baseline to compare against, and you can roll back by undeploying the candidate.

Agent Lifecycle

Agents are managed end-to-end through the nemo agents command group:

To run an agent.yaml directly without registering it on the platform, pass --agent-config <path> to nemo agents invoke or nemo agents run.

Legacy NAT-only commands:

Agent Definition

Platform-managed agents

About NeMo Fabric

NeMo Fabric gives users one configurable, observable way to run applications across multiple agent harnesses. It standardizes configuration, lifecycle management, and results without requiring a separate integration for every harness. To learn more about Fabric, see NeMo Fabric.

NeMo Platform uses Fabric as the runtime wrapper around your agent so the platform can deploy it and route its model traffic through shared infrastructure.

Intended behavior is recorded in ETHOS.md. Runtime behavior is described by the platform-managed agent.yaml:

The calculator agent uses DeepAgents as its harness and the Platform Inference Gateway for model access. It answers arithmetic and numeric comparison requests and records ATIF and ATOF telemetry with NeMo Relay:

1
config_format: nemo-agents-spec-v1
2
name: calculator-agent
3
description: Calculator agent executed with DeepAgents
4
5
instructions:
6
  system:
7
    content: |
8
      You are a concise calculator agent. Solve arithmetic and numeric
9
      comparison requests yourself and return only the answer.
10
11
default_harness: deepagents
12
13
harnesses:
14
  deepagents:
15
    kind: deepagents
16
    settings:
17
      deepagents: {}
18
19
models:
20
  default:
21
    provider: nvidia
22
    model: nvidia-nemotron-3-nano-30b-a3b
23
    api_key_env: NVIDIA_API_KEY
24
25
skills:
26
  paths: []
27
28
mcp:
29
  servers: {}
30
31
tools:
32
  blocked: []
33
34
environment:
35
  workspace: ./workspace
36
  artifacts: ./artifacts
37
38
telemetry:
39
  enabled: true
40
  provider: relay
41
  output_dir: ./artifacts/relay
42
  project: calculator-agent
43
  atif:
44
    enabled: true
45
    filename_template: trajectory-{session_id}.atif.json
46
    storage:
47
      - type: http
48
        endpoint: http://localhost:8080/apis/intake/v2/workspaces/default/ingest/atif
49
        timeout_millis: 3000
50
  atof:
51
    enabled: true
52
    filename: events.atof.jsonl
53
    mode: append

NVIDIA Agent Toolkit (NAT) workflows (Legacy)

About NeMo Agent Toolkit

NVIDIA NeMo Agent Toolkit is a flexible, lightweight, and unifying library that allows you to easily connect existing enterprise agents to data sources and tools across any framework.

NeMo Platform uses NAT as the runtime wrapper around your agent so the platform can deploy it, evaluate it, optimize it, and route its model traffic through shared infrastructure. For the toolkit itself, see the NeMo Agent Toolkit documentation.

An agent’s behavior is described by a NAT workflow YAML with three top-level sections:

ReAct is a common agent pattern where the model alternates between a reasoning step and a tool call until it has enough information to answer. It is a good default when you want the agent to decide which tool to use next based on what it has already learned. A minimal ReAct agent looks like this:

1
functions:
2
  wiki:
3
    _type: wiki_search
4
  clock:
5
    _type: current_datetime
6
7
llms:
8
  llm:
9
    _type: openai
10
    model_name: nvidia-nemotron-3-nano-30b-a3b
11
12
workflow:
13
  _type: react_agent
14
  tool_names: [wiki, clock]
15
  llm_name: llm

The Inference Gateway is the local platform’s model proxy. Agents send model requests to it instead of to provider APIs directly, so the platform can resolve model names, attach credentials, and route through middleware on the agent’s behalf. Two conventions apply when a config targets a deployed agent:

• Model names use the Inference Gateway entity form, with slashes and dots converted to hyphens (nvidia/nemotron-3-nano-30b-a3b becomes nvidia-nemotron-3-nano-30b-a3b). The Inference Gateway resolves the entity to the upstream provider that owns it.
• Leave base_url and api_key unset on openai and nim LLMs. NeMo Platform injects an Inference Gateway URL when it deploys the agent, and the gateway retrieves upstream credentials from the secrets service. Setting base_url explicitly bypasses both.

Common Tasks

• Agent Ethos: ETHOS.md, the durable contract for intended behavior.
• Deploy Agents: run an agent as a local subprocess or as a durable container on Docker or Kubernetes, and invoke it through the Agents gateway.
• Observe Agents: ingest and query agent telemetry with NeMo Intake, then review traces, feedback, and evaluator results.
• Optimize Agents: Fabric-backed numeric HPO (nemo agents optimize), plus model-routing / skill / prompt suggestions for deployed agents.
• Secure Agents: check guardrail coverage and scan recent telemetry for sensitive data.
• Plugins and Skills: understand how agent, middleware, and coding-agent integrations extend the local platform.
• Agentic Metrics: evaluate tool use, goal completion, topic adherence, answer accuracy, and trajectories.
• Agent Configuration: use agents as online evaluation targets.