Device Memory is a verbs API that allows using on-chip memory, located on the device, as a data buffer for send/receive and RDMA operations. The device memory can be mapped and accessed directly by user and kernel applications, and can be allocated in various sizes, registered as memory regions with local and remote access keys for performing the send/ receive and RDMA operations. Using the device memory to store packets for transmission can significantly reduce transmission latency compared to the host memory.

Device Memory Experimental Verbs 

• ibv_exp_alloc_dm
• ibv_exp_memcpy_dm
• ibv_exp_free_dm
• ibv_exp_reg_mr
  • ibv_exp_reg_mr_dm

Device Memory RDMA-Core Verbs

• Ibv_alloc_dm
• Ibv_reg_dm_mr
• ibv_memcpy_to_dm
• ibv_memcpy_from_dm
• ibv_free_dm

Relevant Man Pages

ibv_alloc_dm: https://github.com/linux-rdma/rdma-core/blob/master/libibverbs/man/ibv_alloc_dm.3

Example

See example in libibverbs/examples/rc_pingpong.c