NVIDIA HPC-X Software Toolkit Rev 2.21.0

Changes and New Features

HPC-X current version provides the following changes and new features:

Category

Change Description

Hybrid Mode for Hardware DCS Offload

Added support for hardware DCS offload in hybrid mode, which combines existing software DCS with hardware DCS offload.

To enable this feature, set UCX_DC_MLX5_TX_POLICY=dcs_hybrid

GGA Transport for High-Speed Data Transfer

Added support for GGA transport, a high-speed DMA copy engine that enables efficient data transfer between host memory and the DPU's internal memory.

Multi-Node NVLink with Optimized Protocol Selection

Added support for multi-node NVLink, with automatic detection and selection of the most efficient data transfer protocols.

IP Address Filtering for RoCE Devices

Added support for filtering RoCE devices based on their IP address, allowing the selection of specific network subnets.

To configure the filter, set UCX_IB_ROCE_SUBNETSwith a list of subnets. For example: UCX_IB_ROCE_SUBNETS=5.4.3.2/16,1.2.3.4/24.

Automatic Selection of GPU Bounce Buffers for Large Message Transfers

Added support for an optimization that automatically selects GPU bounce buffers for large message transfers when these buffers offer performance benefits over host memory buffers.

Single Memory Key Creation Using ODP for Enhanced Efficiency

Added support for a feature that leverages the ODP capability to create a single memory key for the entire process's virtual address space. This reduces the number of allocated memory keys, helping to bypass firmware limitations.

To enable this feature, set UCX_GVA_ENABLE=y

MLNX_OFED to DOCA-OFED Transition

Starting this version, the host driver is part of the NVIDIA DOCA package.

DOCA-OFED is a DOCA-Host profile that includes the same components, drivers, and tools as MLNX_OFED. Installing DOCA-OFED will result with the same file system on the host as MOFED.

For further information, please see NVIDIA MLNX_OFED to DOCA-OFED Transition Guide .

HPC-X Content

Updated HPC-X Content section to reflect the communication libraries versions embedded in this HPC-X release.

  • NVIDIA Scalable Hierarchical Aggregation and Reduction Protocol (SHARP) v3.9.0

  • UCX v1.18.0

© Copyright 2024, NVIDIA. Last updated on Jan 21, 2025.