Skip to navigation

Agent Skills

View as Markdown

The Alpamayo1.5 NIM image includes three Agent Skills that teach compatible agents how to operate the service safely:

SkillPurpose
start-alpamayo-nimCheck prerequisites, launch the container, wait for readiness, and diagnose or restart an unhealthy backend.
run-alpamayo-trajectoryValidate camera and fixed 16-row ego-motion inputs, choose HTTP or gRPC, submit trajectory inference, and inspect its outputs.
run-alpamayo-vqaSubmit visual question answering requests with one or more images over HTTP.

The skills follow the open Agent Skills directory format. Each skill is a folder containing SKILL.md and optional product metadata. No plugin or marketplace installation is required.

Extract the Skills

Set IMG_NAME to the Alpamayo image, then create a temporary container and copy the skills and sample scene without starting the NIM:

export IMG_NAME=nvcr.io/nim/nvidia/alpamayo1.5:1.0.0
mkdir -p alpamayo-agent-assets
asset_container=$(docker create "$IMG_NAME")
docker cp "$asset_container:/opt/nim/skills" alpamayo-agent-assets/skills
docker cp "$asset_container:/workspace/sample_data" alpamayo-agent-assets/sample_data
docker rm "$asset_container"

If an alpamayo1.5 container is already running, copy the same paths directly from that container instead.

Install the Skills in Codex

Codex discovers user-scoped skills under $HOME/.agents/skills. Copy the extracted directories there:

mkdir -p "$HOME/.agents/skills"
cp -R alpamayo-agent-assets/skills/. "$HOME/.agents/skills/"

Start a new Codex thread after installation. Use /skills to inspect the available skills, or invoke one explicitly by including its $ name in the prompt. For example:

Use $start-alpamayo-nim to launch $IMG_NAME with the cache at
/tmp/alpamayo-nim-cache and wait until it is ready.
Use $run-alpamayo-trajectory with
alpamayo-agent-assets/sample_data to run one gRPC trajectory request.
Use $run-alpamayo-vqa to ask what the vehicle ahead is doing in
alpamayo-agent-assets/sample_data/images/front_wide_t3.png.

Agents can also select a skill automatically when the request matches the skill description. Explicit invocation is recommended for the first run so the chosen workflow is visible.

Run an End-to-End Agent Workflow

Use the skills in this order:

  1. Invoke $start-alpamayo-nim to verify the GPU, cache, ports, and API key, launch the container, and wait for /v1/health/ready.
  2. Invoke $run-alpamayo-trajectory or $run-alpamayo-vqa with the extracted sample files or your own inputs.
  3. Use the trajectory skill to validate both ego-motion arrays as exactly 16 rows before sending a request. The history length remains 16 for reduced image layouts.
  4. If an inference request fails, check readiness before retrying. Stop routing requests when readiness is not HTTP 200; unrecoverable CUDA failures require a container restart.

Do not include the value of NGC_API_KEY in an agent prompt, transcript, or command line. Export it in the shell and let the startup skill pass the environment variable by name. Review container start, stop, and restart actions before authorizing them in environments that host other workloads.

Use the Skills with Other Agents

The extracted directories can be used by other Agent Skills-compatible tools. Install them in the directory that the tool documents for user-scoped skills. If a tool does not support Agent Skills discovery, attach the relevant SKILL.md as instructions for that agent and preserve its safety constraints.

Use Documentation for Advanced Workflows

The bundled skills cover direct Docker startup and common trajectory and VQA requests over HTTP and gRPC. For complex, deployment-specific, or application-specific workflows, give the agent access to the documentation for the same NIM version and direct it to the relevant pages, such as Configuring a NIM, API Reference, Sampling Control, Support Matrix, and Bring Your Own Checkpoint.

A link is useful only when the agent can retrieve its contents. Otherwise, attach the relevant pages or provide their contents. Do not include credentials, private checkpoint data, or unrelated logs, and review generated commands and configuration changes before running them.