Run a War-Game
Agent Governance is released with early access availability and is subject to limited support and potential API changes in future releases.
This page runs a full attack, defend and validate cycle against an agent. It covers both starting points: an agent registered on the platform, and an agent project the platform does not manage at all, supplied as a directory or an uploaded archive.
The cycle produces proposed fixes. Choosing which to keep and writing them back to the agent is covered in Review and Apply Mitigations.
Prerequisites
- Local services running with the models and jobs controllers:
Bind to
0.0.0.0so the sandbox can reach the Inference Gateway throughhost.docker.internal. - Docker running on the host.
- The OpenShell gateway installed with its native installer. Installing OpenShell as a standalone CLI tool gives you the command without the gateway service.
- At least one model provider registered in the workspace.
Provision the Agent hardening plugin once per machine, then confirm the prerequisites:
setup creates the agent-hardener and garak virtual environments and the inference
credential. doctor is read-only and checks all five: the agent-hardener venv, the
garak venv, the inference credential, the Docker daemon and the OpenShell gateway.
1. Create a Manifest
A manifest is the war-game’s target. Creating one resolves the agent and stores the result, so every later run attacks the same thing.
Registered agent
Unregistered project
The agent must be registered with nemo agents create. It does not have to be
deployed: the Agent hardening plugin reads the stored config and builds its own sandboxed copy.
The manifest is named after the agent, and that name is the --manifest-id
every later command takes. init also writes agent-hardener.yaml as a readable
rendering. Editing that file does not change a --manifest-id run, which reads
the saved manifest — but run --config reads the local file, so edits do take
effect there.
Declare the hosts your agent’s tools call:
A bare host opens port 443 only. Write host:80 for plain HTTP. Hosts cannot
be discovered automatically for an agent whose tool code lives in an installed
package, so declare them here.
If the agent reads non-secret environment variables, set them at the same time:
--env is repeatable and splits on the first = only, so values may contain
=. Keep credentials out of it: --env values are stored in plaintext on the
manifest, while --secrets stores only the names and resolves the values from
the platform Secrets store when the run starts.
If a tool’s host is not allow-listed, the sandbox drops the call and the model usually answers from its own knowledge instead. The run then reports success while never exercising the tool path. Check egress before trusting a clean result.
2. Generate a Benign Suite
The benign suite is the set of ordinary requests the agent should still answer
after hardening. nemo agent-hardener run consumes a suite and never generates one,
so create it before the first run:
This opens an interview about what the agent does, then lets you review and edit the generated requests. The result is cached on the manifest and reused by later runs.
Two non-interactive forms:
--yes accepts the interview’s suggested answers. --no-interactive skips the
interview and generates from rules alone, which suits CI.
3. Run the Cycle
The run attacks a sandboxed copy of the agent, generates fixes with both defenders, then replays the attacks and the benign suite against those fixes.
Common options:
These flags apply to a single launch and never edit the saved manifest, so a run can deviate from the frozen baseline without breaking comparability. To change the stored defaults instead:
Check recent runs:
4. Keep the Manifest Current
Editing the agent does not change an existing manifest. Pick up the changes deliberately:
Egress, secrets, models, defenders and the cached benign suite are all preserved. Only the target is rebuilt.
Applying a mitigation refreshes the manifest for you, so a run, harden, apply, run sequence measures the change you just made.
Run It in Studio
Studio drives the same cycle and is the only surface that can apply fixes back to the agent.
In NeMo Studio, open Governance → Agent Hardening in the workspace sidebar.
- Manifests → New Manifest. Choose a source: a registered agent, or
Upload Project for an agent the platform does not manage. An uploaded
project is a single
.zipcontaining an installable agent project, and Studio inspects it to pre-fill the workflow and port. - Accept or correct the detected port and secrets, add any egress hosts, then Create.
- Run war-game on the manifest. The run opens on its Swarm tab.
- Watch the graph advance per phase with the live agent feed beside it. Click any node for its prompts and model calls.
- When the run finishes, use the Harden panel. See Review and Apply Mitigations.
The Studio UI ships inside the plugin, so installing the plugin is what puts it in Studio. There is no feature flag.
Next Steps
- Review and Apply Mitigations: choose which fixes to keep and write them back to the agent.
- Agent Hardening CLI Reference: every command and flag.
- Troubleshooting: setup, sandbox and victim failures.