> This page is for version 9.2 (default). > For other versions, use one of these documentation indexes: > - 9.2 (default): https://docs.nvidia.com/deepstream/9.2/llms.txt > - Archives: https://docs.nvidia.com/deepstream/archives/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/deepstream/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/deepstream/_mcp/server. # Gst-nvinfereval The Gst-nvinfereval plugin is a pass-through element that evaluates live model accuracy against ground-truth data. It auto-detects whether it is downstream of a single inference model (Mode A) or a [Gst-nvmodelmux](/text/DS_plugin_gst-nvmodelmux) primary/shadow pair (Mode B), and computes AP\@0.5, AP\@0.5:0.95, precision, recall, and F1 per model, per class, over configurable evaluation epochs. In Mode B, it can gate automatic promotion of a shadow model to primary based on configurable KPI thresholds and a minimum AP margin over the primary. The buffer is never modified; the element only reads `InferenceProvenanceMeta` when present and forwards it to the evaluation engine. ## Inputs and Outputs * Inputs * GStreamer buffer containing NvDsBatchMeta with InferenceProvenanceMeta (from an upstream Gst-nvinfer or Gst-nvmodelmux) * Control parameters * gt-file, config-file, eval-interval, iou-threshold, cumulative * min-ap-margin, kpi-ap50, kpi-precision, kpi-recall, promotion-mode (A/B promotion gating, Mode B only) * report-file, show-gt-overlay, show-ap-table, exclude-classes, exclude-undetected * Output * The same GStreamer buffer, unmodified, passed through to the next element * A GstMessage posted at each epoch boundary with the evaluation report (and optionally a JSON report file) ## Features The following table summarizes the features of the plugin. **Gst-nvinfereval plugin features** | Feature | Description | Release | | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | Live accuracy evaluation | Computes AP\@0.5, AP\@0.5:0.95, precision, recall, and F1 per model and per class against ground truth over configurable evaluation epochs | DS 9.2 | | Mode A / Mode B auto-detection | Auto-detects whether it sits downstream of a single inference model or a Gst-nvmodelmux primary/shadow pair | DS 9.2 | | A/B promotion gating | In Mode B, gates automatic promotion of a shadow model to primary using a minimum AP margin and configurable KPI thresholds (AP\@0.5, precision, recall) | DS 9.2 | | Ground-truth format auto-detection | Auto-detects the ground-truth format (KITTI directory \| NVSchema JSONL \| COCO JSON \| TAO JSON) | DS 9.2 | | Per-model / per-class config overrides | Supports per-model \[model-\] config-file sections that override GT files, KPI thresholds, and promotion margins | DS 9.2 | ## Gst Properties The following table describes the Gst properties of the Gst-nvinfereval plugin. **Gst-nvinfereval plugin gst properties** | Property | Meaning | Type and Range | Example Notes | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | ---------------------------------- | | gt-file | Default ground-truth file or directory for all models (format auto-detected: KITTI dir, NVSchema JSONL, COCO JSON, or TAO JSON); per-model overrides go in the config-file \[model-\] sections; leave empty to run in no-GT relative-comparison mode | String | gt-file=/path/to/gt | | config-file | Path to a GKeyFile INI config (see configs/config_nvinfereval.txt); keys in the file override the corresponding GObject properties, and per-model \[model-\] sections set model-specific GT files, KPI thresholds, and promotion margins; leave empty to use GObject properties only | String | config-file=config_nvinfereval.txt | | eval-interval | Number of Primary frames per evaluation epoch, counted independently per model_name; shadow frames are excluded from the count so the epoch always spans exactly this many Primary frames regardless of whether a shadow model is present | Integer, 1 to 4,294,967,295 | eval-interval=1000 | | iou-threshold | Minimum IoU for a prediction to count as a true positive in precision/recall/F1 computation; AP\@0.5 and AP\@0.5:0.95 always use their fixed standard thresholds | Float, 0.0 to 1.0 | iou-threshold=0.5 | | min-ap-margin | Default margin by which the shadow's AP\@0.5 must exceed the primary's AP\@0.5 to trigger promotion; per-model override available in config-file | Float, 0.0 to 1.0 | min-ap-margin=0.02 | | kpi-ap50 | Default minimum AP\@0.5 the shadow must reach for promotion (0.0 = disabled); per-model override available in config-file | Float, 0.0 to 1.0 | kpi-ap50=0.0 | | kpi-precision | Default minimum mean precision the shadow must reach (0.0 = disabled); per-model override available in config-file | Float, 0.0 to 1.0 | kpi-precision=0.0 | | kpi-recall | Default minimum mean recall the shadow must reach (0.0 = disabled); per-model override available in config-file | Float, 0.0 to 1.0 | kpi-recall=0.0 | | promotion-mode | Action taken when the shadow wins an epoch (Mode B only): 0 (promote) shadow becomes primary and A/B ends; 1 (swap) shadow becomes primary, old primary becomes shadow, A/B continues; 2 (report-only) metrics reported, no REST call, pure monitoring | Integer, 0 to 2 | promotion-mode=0 | | report-file | Optional path to write JSON epoch reports (appended); leave empty to disable file output (a GstMessage is always posted) | String | report-file=/tmp/eval_report.jsonl | | show-gt-overlay | Draws ground-truth bounding boxes as green rectangles via nvosd; useful for debugging GT alignment | Boolean | show-gt-overlay=0 | | show-ap-table | Prints the per-class AP/precision/recall table to the console at each epoch boundary | Boolean | show-ap-table=0 | | exclude-classes | Comma-separated list of class labels to exclude from mean AP/P/R; excluded classes are still shown per-class with an \[EXCLUDED] note | String | exclude-classes=unknown,Box | | exclude-undetected | Automatically excludes from mean AP/P/R any class for which the model produced zero detections (TP=0, FP=0); such classes appear in the per-class table with \[UNDETECTED] | Boolean | exclude-undetected=1 | | cumulative | When TRUE, metrics are computed over all frames since pipeline start (accumulator never resets); when FALSE, each epoch is an independent tumbling window of eval-interval Primary frames | Boolean | cumulative=1 |