> This page is for version 3.3.0 (default).
> For other versions, use one of these documentation indexes:
> - 3.3.0 (default): https://docs.nvidia.com/vss/3.3.0/llms.txt
> - Dev: https://docs.nvidia.com/vss/dev/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/vss/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/vss/_mcp/server.

# Dataset Post-Processing

Post-processing is also a crucial step to verify the correctness of all generated data and convert the ground truth data into a specific format compatible with the model.

Generated data for MTMC training includes as follows:

* RGB images.
* 2D & 3D bboxes.
* Depth maps.
* Camera calibration.

## Prerequisites

With conda installed, run the following from the repository root on Ubuntu/Debian:

```bash
conda create -n postpro python=3.10 -y
conda activate postpro
python -m pip install -r tools/sdg-postprocessing/requirements.txt
sudo apt update && sudo apt install -y ffmpeg
ffmpeg -version
```

Post-processing is divided into two steps:

**Data Sanity Check**:

* Sanity checks for RGB, depth maps, and ground truth focuses on identifying issues such as dropped frames or data corruption.
* Sanity check for calibration focuses on identifying the correctness of calibration, including intrinsic matrix, extrinsic matrix, point pairs array, projection matrix, and homography matrix.
* Sanity check for b frame focuses on identifying if the b-frames are present in a video.

Navigate to the `tools/sdg-postprocessing/data_sanity_check` path:

```bash
cd /path/to/video-search-and-summarization/tools/sdg-postprocessing/data_sanity_check/
```

```bash
# RGB image sanity check
python dataset_sanity_check_rgb.py --base_dir /path/to/dataset --total_frames 9000

# Depth map sanity checks
# Raw depth maps (.npy) and converted .png files should both be validated
python dataset_sanity_check_npy.py --base_dir /path/to/dataset --total_frames 9000
python dataset_sanity_check_png.py --base_dir /path/to/dataset --total_frames 9000

# Ground truth sanity check
python dataset_sanity_check_json.py --base_dir /path/to/dataset --total_frames 9000

# Calibration sanity check
mkdir -p /path/to/calibration-sanity-output
python dataset_sanity_check_calibration.py --base_dir /path/to/dataset --calibration /path/to/dataset/calibration.json --output_dir /path/to/calibration-sanity-output

# Video b-frame sanity check
bash dataset_sanity_check_videos.sh /path/to/dataset
```

**Data Conversion**:

Conversions for depth maps, RGB images, and ground truth are designed to save storage space and reduce redundant ground truth data, while schema conversion is performed for downstream tasks such as E2E model & pipeline evaluation.

Navigate to the `tools/sdg-postprocessing/data_conversion` path:

```bash
cd /path/to/video-search-and-summarization/tools/sdg-postprocessing/data_conversion/
```

```bash
# Depth map conversion
python convert_npy_to_png_depthmap.py /path/to/dataset

# Video conversion
bash convert_images_to_videos_no_bframes.sh /path/to/dataset

# Ground truth conversion
python convert_ground_truth.py /path/to/dataset --calibration /path/to/dataset/calibration.json --output /path/to/dataset/ground_truth --max_frames 9000 --xform_info /path/to/xform_info.json
```

> **Note**
>
> Generate `xform_info.json` only when ground-truth conversion requires parent-Xform rotation overrides:
>
> 1. Load the scene in Isaac Sim.
>
> 2. Open the Script Editor.
>
>    ![Script Editor location](/vss/_fern-img/33309fcead651fe96e8feb965396b8032a11d8bec0390cabc42779740295e87e.webp)
>
> 3. In the Script Editor, open `tools/sdg-postprocessing/utils/export_xform_semantics.py` via **File** > **Open**, and set `USER_OUTPUT_PATH` to your desired output directory or JSON file path.
>
> 4. Click **Run** to export the JSON file to the configured path.
>
> If the scene does not need parent-Xform rotation overrides, omit the `--xform_info` option from the ground-truth conversion command.