Overview Guide#
The NVIDIA® Magnum IO GPUDirect® Storage Overview Guide provides a high-level overview of GPUDirect® Storage (GDS).
GPUDirect® Storage (GDS) enables a direct data path for direct memory access (DMA) transfers between GPU memory and storage, which avoids a bounce buffer through the CPU. Using this direct path can relieve system bandwidth bottlenecks and decrease the latency and utilization load on the CPU.
This guide provides you guidance to help you enable filesystems for GDS, and some insights about the features of a file system and how it relates to GDS. The guide also outlines the functionalities, considerations, and software architecture about GDS. This high-level introduction sets the stage for deeper technical information in the cuFile API Reference Guide for GDS users who need to modify the kernel.
Benefits for a Developer#
GDS provides the following benefits for application developers:
Enables a direct path between GPU memory and storage.
Can relieve bandwidth bottlenecks, reduce the latency, and reduce the load on CPUs for data transferral.
Reduces the performance impact and dependence on CPUs to process storage data transfer.
Performance force multiplier on top of the compute advantage for computational pipelines that are fully migrated to the GPU so that the GPU, rather than the CPU, has the first and last touch of data that moves between storage and the GPU.
Supports interoperability with other OS-based file access, which enables data to be transferred to and from the device by using traditional file IO, which is then accessed by middleware or an application program that uses the cuFile APIs.
The cuFile APIs and their implementations provide the following benefits:
A family of APIs that provide CUDA® applications with the best-performing access to local or distributed file and block storage.
When transferring to and from the GPU, increased performance and lower CPU load relative to existing standard Linux file IO.
Greater ease of use by removing the need for the careful expert management of memory allocation and data movement.
An API sequence that is simpler relative to existing implicit file-GPU data movement methods which require a more complex management of memory and data movement on and between the CPU and GPU.
Generality across a variety of storage types that span various local and distributed file systems, block interfaces, and namespace systems, including standard Linux and third-party solutions.
Primary API to do file I/O independent of memory type for GPU applications.
Ability to target memory that has been allocated with any of cudaMalloc or cuMemAlloc or cuMemCreate/cuMemMap to the GPU, or cudaHostAlloc or cudaMallocHost to the CPU. GDS peer-to-peer mode does not support migratable memory, allocated with
cudaMallocManagedor system-allocated memory (malloc, stack, etc.) on systems where UVM support is available such as Grace-Hopper. GDS will support IO to these memory allocations using a compatible path using internal GPU/CPU bounce buffers.
The Stream subset of the cuFile APIs provides the following benefits:
Asynchronous offloaded operations are ordered with respect to a CUDA stream.
IO after compute: The GPU kernel produces data before it is transferred to IO.
Compute after IO: After the data transfer is complete, the GPU kernel can proceed.
Available concurrency across streams.
Using different CUDA streams allows the possibility of concurrent execution and the concurrent use of multiple DMA engines.
Intended Uses#
cuFile features can be used in the following ways:
cuFile implementations boost throughput when IO between storage and GPU memory is a performance bottleneck.
This condition arises in cases where the compute pipeline has been migrated to the GPU from the CPU, so that the first and last agents to touch data, before or after transfers with storage, execute on the GPU.
cuFile APIs are currently explicit and are used for reading or writing between storage and buffers that completely fit into the available GPU physical memory.
Rather than fine-grained random access, the cuFile APIs are a suitable match for coarse-grained streaming transfers.
For fine-grained accesses, the underlying software overheads for making a kernel transition and going through the operating system can be amortized.