Is this page helpful?

DeepStream Coding Agent#

Building production‑grade vision AI pipelines has traditionally demanded deep expertise in GStreamer internals, GPU‑accelerated inference, and multi‑stream orchestration. DeepStream’s AI Agent support changes that equation — it brings the knowledge to the developer, right inside the IDE.

By pairing AI coding assistants such as Cursor, Claude Code, and Codex with DeepStream‑specific skills and MCP tooling, developers can describe what they want in natural language and receive architecturally sound, SDK‑idiomatic code in return.

DeepStream offers two complementary AI agent integrations:

DeepStream Agentic Skills — A suite of structured knowledge packages, each targeting a specific DeepStream workflow. Skills cover pyservicemaker (DeepStream’s Python pipeline API) code generation, gst-launch pipeline building, Nsight Systems profiling, vision-model onboarding, MV3DT reference-app operations, SOP microservice development, and AutoMagicCalib camera calibration. The full skill catalog, install instructions, and example prompts are in the DeepStream GitHub repository.
Inference Builder MCP — A Model Context Protocol server that connects Inference Builder directly to AI coding agents. Through natural‑language prompts, the agent can generate inference pipeline configurations, build Docker images, and run smoke tests — all without leaving the conversation. See Inference Builder MCP for details.

Together, these integrations compress the path from idea to running pipeline, letting developers focus on application logic while the agent handles boilerplate, configuration, and SDK plumbing.

Note

Code generated with AI coding assistants is a development starting point. All generated code must go through your full software development lifecycle (SDLC) — including code review, testing, and security validation — before production use.

DeepStream Agentic Skills#

The DeepStream GitHub repository ships a suite of agentic skills under the skills/ directory. Each skill follows the standard SKILL.md convention supported by AI coding assistants such as Cursor, Claude Code, and Codex, and can be installed at the user level (available across all projects) or at the workspace level (scoped to a single project).

Note

The six DeepStream pipeline skills require only an AI coding assistant — no GPU, DeepStream SDK installation, or special hardware is needed. The AutoMagicCalib skills additionally require an NVIDIA GPU, Docker, NVIDIA Container Toolkit, and NGC image access on the host where the calibration stack runs. DeepStream SDK is only required on the system where you run the generated code; see Installation for setup instructions.

To install a skill, clone the repository and copy the skill directory into your tool’s skills folder:

git clone https://github.com/NVIDIA/DeepStream
cp -r DeepStream/skills/<skill-name> ~/.claude/skills/   # Claude Code
# or ~/.cursor/skills/  for Cursor, ~/.codex/skills/  for Codex

See the GitHub README for per-tool paths and symlink-based update instructions.

DeepStream pipeline and application skills:

Skill	Mode	Use when you want to…
`deepstream-dev`	Reference-rich	Hand-author or refine a `pyservicemaker` / GStreamer DeepStream pipeline — the agent consults condensed SDK references and guardrails for accurate API usage.
`deepstream-generate-pipeline`	Interactive + retrieval	Generate a ready-to-run `gst-launch-1.0` pipeline by answering a short questionnaire; the agent retrieves and adapts from 270+ verified pipelines and validates the result before presenting it.
`deepstream-profile-pipeline`	Measure-then-derive	Profile a pipeline with Nsight Systems, derive optimal configs from the measured inference plateau batch and HW ceiling, and report per-plugin NVTX timings. Requires the `deepstream:9.1-triton-multiarch` container; other DS images strip the NVTX injector and produce empty traces.
`deepstream-import-vision-model`	Autonomous orchestration	Onboard any HuggingFace or NGC object-detection model end-to-end: ONNX export, TensorRT engine build, multi-stream DeepStream benchmark, and a PDF report — fully unattended.
`deepstream-run-mv3dt`	Reference-app operations	Run the DeepStream Multi-View 3D Tracking (MV3DT) reference app on shipped samples or synchronized MP4 datasets, including calibration handoff, Kafka metadata, and OSD/Bird’s Eye View (BEV) outputs.
`deepstream-sop`	Microservice scaffold + evaluate-and-fix loop	Build, deploy, evaluate, debug, and measure latency on a DeepStream SOP (Standard Operating Procedure) inference microservice for GPU-accelerated operator step-sequence compliance via Generic Event Boundary Detection (GEBD) + Vision Language Model (VLM). Supports file, RTSP, and Basler GigE camera inputs with SSE and Kafka output.

AutoMagicCalib camera calibration skills (see AutoMagicCalib):

Skill	Mode	Use when you want to…
`amc-setup-calibration-stack`	Deployment runbook	Launch the AutoMagicCalib microservice and web UI from NGC release images via Docker Compose.
`amc-run-sample-calibration`	Validation runbook	Verify a running AutoMagicCalib stack with the bundled synthetic sample dataset.
`amc-run-video-calibration`	Calibration runbook	Calibrate a camera rig from user-provided pre-recorded MP4 files via the AutoMagicCalib REST API.
`amc-run-rtsp-calibration`	RTSP capture + calibration	Calibrate a camera rig from live RTSP streams by recording them via a capture service and submitting them to the AutoMagicCalib REST API.

For per-skill install instructions, bundled reference documents, and example prompts, see the skills/ directory in the DeepStream GitHub repository.

Tip

All skills work best in Agent mode. In agent mode, the AI assistant automatically selects and activates the relevant skill based on the task context — no manual file referencing needed.