RDMA Aware Networks Programming User Manual v1.7

RDMA Protection Domain Operations

Template: struct ibv_mr {}rdma_reg_msgs{*}(struct rdma_cm_id *id, void *addr, size_t length)

Input Parameters:
id A reference to the communication identifier where the message buffer(s) will be used

addr The address of the memory buffer(s) to register

length The total length of the memory to register

Output Parameters:

ibv_mr A reference to an ibv_mr struct of the registered memory region

Return Value: A reference to the registered memory region on success or NULL on failure

Description: rdma_reg_msgs registers an array of memory buffers for sending or receiving messages or for RDMA operations. The registered memory buffers may then be posted to an rdma_cm_id using rdma_post_send or rdma_post_recv. They may also be specified as the target of an RDMA read operation or the source of an RDMA write request.
The memory buffers are registered with the protection domain associated with the rdma_cm_id. The start of the data buffer array is specified through the addr parameter and the total size of the array is given by the length.
All data buffers must be registered before being posted as a work request. They must be deregistered by calling rdma_dereg_mr.

Template: struct ibv_mr * rdma_reg_read(struct rdma_cm_id *id, void *addr, size_t length)

Input Parameters:

id A reference to the communication identifier where the message buffer(s) will be used

addr The address of the memory buffer(s) to register

length The total length of the memory to register

Output Parameters:

ibv_mr A reference to an ibv_mr struct of the registered memory region

Return Value: A reference to the registered memory region on success or NULL on failure. If an error occurs, errno will be set to indicate the failure reason.

Description: rdma_reg_read Registers a memory buffer that will be accessed by a remote RDMA read operation. Memory buffers registered using rdma_reg_read may be targeted in an RDMA read request, allowing the buffer to be specified on the remote side of an RDMA connection as the remote_addr of rdma_post_read, or similar call.
rdma_reg_read is used to register a data buffer that will be the target of an RDMA read operation on a queue pair associated with an rdma_cm_id. The memory buffer is registered with the protection domain associated with the identifier. The start of the data buffer is specified through the addr parameter, and the total size of the buffer is given by length.
All data buffers must be registered before being posted as work requests. Users must deregister all registered memory by calling the rdma_dereg_mr.

See Also rdma_cm(7), rdma_create_id(3), rdma_create_ep(3), rdma_reg_msgs(3), rdma_reg_write(3), ibv_reg_mr(3), ibv_dereg_mr(3), rdma_post_read(3)

Template: struct ibv_mr {}rdma_reg_write{*}(struct rdma_cm_id *id, void *addr, size_t length)

Input Parameters:

id A reference to the communication identifier where the message buffer(s) will be used

addr The address of the memory buffer(s) to register

length The total length of the memory to register

Output Parameters:

ibv_mr A reference to an ibv_mr struct of the registered memory region

Return Value: A reference to the registered memory region on success or NULL on failure. If an error occurs, errno will be set to indicate the failure reason.

Description: rdma_reg_write registers a memory buffer which will be accessed by a remote RDMA write operation. Memory buffers registered using this routine may be targeted in an RDMA write request, allowing the buffer to be specified on the remote side of an RDMA connection as the remote_addr of an rdma_post_write or similar call.
The memory buffer is registered with the protection domain associated with the rdma_cm_id. The start of the data buffer is specified through the addr parameter, and the total size of the buffer is given by the length.
All data buffers must be registered before being posted as work requests. Users must deregister all registered memory by calling the rdma_dereg_mr.

See Also rdma_cm(7), rdma_create_id(3), rdma_create_ep(3), rdma_reg_msgs(3), rdma_reg_read(3), ibv_reg_mr(3), ibv_dereg_mr(3), rdma_post_write(3)

Template: int rdma_dereg_mr(struct ibv_mr *mr)

Input Parameters:

mr A reference to a registered memory buffer

Output Parameters:

None

Return Value: 0 on success, -1 on error. If the call fails, errno will be set to indicate the reason for the failure.

Description: rdma_dereg_mr deregisters a memory buffer which has been registered for RDMA or message operations. This routine must be called for all registered memory associated with a given rdma_cm_id before destroying the rdma_cm_id.

Template: int rdma_create_srq(struct rdma_cm_id *id, struct ibv_pd *pd, struct ibv_srq_init_attr *attr)

Input Parameters:

id The RDMA communication identifier

pd Optional protection domain for the shared request queue (SRQ)

attr Initial SRQ attributes

Output Parameters:

attr The actual capabilities and properties of the created SRQ are returned through this structure

Return Value: 0 on success, -1 on error. If the call fails, errno will be set to indicate the reason for the failure.

Description: rdma_create_srq allocates a shared request queue associated with the rdma_cm_id, id. The id must be bound to a local RMDA device before calling this routine. If the protection domain, pd,
is provided, it must be for that same device. After being allocated, the SRQ will be ready to handle posting of receives. If a pd is NULL, then the rdma_cm_id will be created using a default protection domain. One default protection domain is allocated per RDMA device. The initial SRQ attributes are specified by the attr parameter.
If a completion queue, CQ, is not specified for the XRC SRQ, then a CQ will be allocated by the rdma_cm for the SRQ, along with corresponding completion channels. Completion channels and CQ data created by the rdma_cm are exposed to the user through the rdma_cm_id structure. The actual capabilities and properties of the created SRQ will be returned to the user through the attr parameter.
An rdma_cm_id may only be associated with a single SRQ.

Template: void rdma_destroy_srq(struct rdma_cm_id *id)

Input Parameters:

id The RDMA communication identifier whose associated SRQ we wish to destroy.

Output Parameters:

None

Return Value: none

Description: rdma_destroy_srq destroys an SRQ allocated on the rdma_cm_id, id. Any SRQ associated with an rdma_cm_id must be destroyed before destroying the rdma_cm_id, id.

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.