NVIDIA Grace Blackwell with ConnectX-8 GPUDirect RDMA Reference Code#

NVIDIA® GPUDirect® RDMA technology provides a direct path for data exchange between the GPU and a peer device using the standard PCI Express (PCIe) features. This document provides information about using the reference code to enable GPUDirect RDMA between the NVIDIA Blackwell GPU and ConnectX-8 (CX8) using Data Direct technology on the NVIDIA Grace Blackwell platform.

Document History#

Date

Description

July 16, 2025

  • Initial release.

August 14, 2025

  • Updated cookbook to be generic to Grace Blackwell

  • Updated Kernel reference to official branch

  • Updated PCIe topology to make it mode readable

  • Updated ACS image with bits highlighted

  • Updated DOCA 3.0 and 3.1 instructions

August 28, 2025

  • Updated Kernel reference

  • Updated DOCA version to use that has new tool ‘rdma_topo’

  • Removed reference to doca_data_direct_ctl

  • Updated ACS section with new ACS flag settings

  • Added PCIe topology images

December 15, 2025

  • Updated Kernel reference

  • Updated DOCA 3.2

  • Added new section to enable data direct on VF

January 28, 2026

  • Update terminology and fix typos
On this page