14. RDMA Peer Memory Configuration#

cuObjClient uses the JSON configuration selected by the CUFILE_ENV_PATH_JSON environment variable. The properties.rdma_peer_type field selects the RDMA peer-memory type used with CUDA device memory.

CUFILE_ENV_PATH_JSON=/path/to/cuobj.json
{
  "properties": {
    "rdma_peer_type": "dmabuf"
  }
}

rdma_peer_type defaults to "nvidia_peermem". Valid values are "dmabuf" and "nvidia_peermem". Set "rdma_peer_type": "dmabuf" to request DMA-BUF support. DMA-BUF use requires platform support for DMA-BUF.

The same selection can be made with the CUFILE_DMABUF_ENABLE environment variable. Set CUFILE_DMABUF_ENABLE=TRUE to request "dmabuf", or CUFILE_DMABUF_ENABLE=FALSE to request "nvidia_peermem". The environment variable is applied after the JSON configuration.

This setting does not make CUDA managed memory registerable. CUDA managed memory is supported only with unregistered cuObjGet() or cuObjPut() operations.