Device Memory is a verbs API that allows using on-chip memory, located on the device, as a data buffer for send/receive and RDMA operations. The device memory can be mapped and accessed directly by user and kernel applications, and can be allocated in various sizes, registered as memory regions with local and remote access keys for performing the send/ receive and RDMA operations. Using the device memory to store packets for transmission can significantly reduce transmission latency compared to the host memory.
Device Memory Experimental Verbs
Device Memory RDMA-Core Verbs
Relevant Man Pages
See example in libibverbs/examples/rc_pingpong.c