> This page is for NVIDIA Synthetic Video Detector NIM, version 2.0.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 2.0.0 (Latest) (default): https://docs.nvidia.com/nim/maxine/synthetic-video-detector/2.0.0/llms.txt
> - 1.0.0: https://docs.nvidia.com/nim/maxine/synthetic-video-detector/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/maxine/llms.txt.

# PCIe Multi-GPU Systems

On hosts with more than one visible GPU, the NIM assigns each request to a GPU and keeps that request's CUDA shared memory and Triton model instance on the same device. This **pinned** mode is the default. It is correct without changing the host IOMMU configuration, and it is the higher-throughput path on typical PCIe multi-GPU systems.

To let Triton's scheduler place each batch on any visible GPU instead, set `NV_AI4M_SVD_PIN_MODEL_TO_SHM_GPU=false`. That **unpinned** mode requires IOMMU passthrough (`iommu=pt`). If passthrough is not active, the NIM logs a warning and stays in the pinned mode rather than starting a configuration that can corrupt cross-device shared-memory reads.

Accepted false values are `false`, `0`, `no`, and `off` (case-insensitive). An unset, empty, or unrecognized value keeps the pinned default.

```bash
docker run -it --rm --name=synthetic-video-detector-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NV_AI4M_SVD_PIN_MODEL_TO_SHM_GPU=false \
  -p 8000:8000 \
  -p 8001:8001 \
  nvcr.io/nim/nvidia/synthetic-video-detector:latest
```

## IOMMU Passthrough

In unpinned mode, Triton schedules model instances across GPUs and the NIM uses peer-to-peer memory copy to transfer data between GPUs whenever this feature is available; that is, when `cudaDeviceCanAccessPeer()` returns true.

However, on bare-metal Linux systems with PCIe topology, IOMMU-enabled peer-to-peer memory copy is not supported. For more information, refer to [Host IOMMU Hardware, PCI Access Control Services, and VMs](https://docs.nvidia.com/cuda/cuda-programming-guide/03-advanced/multi-gpu-systems.html#host-iommu-hardware-pci-access-control-services-and-vms). When IOMMU is enabled, set it to passthrough (by setting the Linux kernel parameter `iommu=pt`) before using unpinned Triton scheduling. An IOMMU that is disabled outright is not equivalent to passthrough; cross-device shared-memory reads can still return corrupted data.

The following are the steps to set IOMMU to passthrough in GRUB:

1. Determine whether IOMMU is enabled by running the following command:

   ```bash
   dmesg | grep -e DMAR -e IOMMU
   ```

   - **No matching output:** IOMMU is not enabled. The default pinned mode still works. To use `NV_AI4M_SVD_PIN_MODEL_TO_SHM_GPU=false`, enable IOMMU in passthrough as described in the following steps.
   - **Matching output:** IOMMU is enabled. Continue to the next step.

2. Determine whether IOMMU is set to passthrough by running the following command:

   ```bash
   dmesg | grep -i -e iommu=pt -e iommu.*passthrough
   ```

   - **No matching output:** Continue with the GRUB configuration steps.
   - **Matching output:** IOMMU passthrough is already enabled. No further action is required.

3. Open `/etc/default/grub` file for edit and add `iommu=pt` to `GRUB_CMDLINE_LINUX` option. For example:

   ```text
   .....

   GRUB_CMDLINE_LINUX="crashkernel=auto quiet iommu=pt"

   .....
   ```

4. Based on the system's OS, use `grub-mkconfig` or `grub2-mkconfig` to generate the configuration file:

   - On systems with BIOS:

     ```bash
     grub-mkconfig -o /boot/grub2/grub.cfg #on ubuntu, debian
     grub2-mkconfig -o /boot/grub2/grub.cfg #on centos, rockylinux
     ```

   - On systems with UEFI:

     ```bash
     grub-mkconfig -o /boot/efi/EFI/<os_name>/grub.cfg #on ubuntu, debian

     grub2-mkconfig -o /boot/efi/EFI/<os_name>/grub.cfg #on centos, rockylinux

        #replace <os_name> with ubuntu, centos, debian, or rocky
     ```

5. On Ubuntu and Debian, you might need to install `grub-mkconfig`:

   ```bash
   apt install grub-common
   ```

6. Reboot the system:

   ```bash
   systemctl reboot
   ```

7. Verify that IOMMU is set to passthrough by repeating step 2.