Resource Domain Experimental Verbs
Resource domain is a verb object that may be associated with QP and CQ objects upon creation to enhance data-path performance.
Thread model - defines the threading module for the objects (QP/CQ) associated with this resource domain.
|
IBV_EXP_THREAD_SAFE |
Access to the associated objects are thread safe. Such objects may be accessed by any thread without concern for thread safety issues. |
|
IBV_EXP_THREAD_UNSAFE |
Access to the associated objects are not thread safe. Access to such objects must be coordinated by the calling threads. |
|
IBV_EXP_THREAD_SINGLE |
Access to the associated objects are from only one thread over the lifetime of the object. In addition, different objects associated with the same resource domain must be called by the same thread. |
Message model - defines whether associated objects connection is optimized for low latency or high bandwidth.
|
IBV_EXP_MSG_FORCE_LOW_LATENCY |
Forces the provider to optimize for low latency |
|
IBV_EXP_MSG_LOW_LATENCY |
Suggests the provider to optimize for low latency. |
|
IBV_EXP_MSG_HIGH_BW |
Suggests the provider to optimize for high bandwidth. |
|
IBV_EXP_MSG_DEFAULT |
Uses the provider's default message model. |
To create a resource domain, use:
static inline struct ibv_exp_res_domain *ibv_exp_create_res_domain(
struct ibv_context *context, struct ibv_exp_res_domain_init_attr *attr)
To
destroy a resource domain, use:
static inline int ibv_exp_destroy_res_domain( struct ibv_context *context,
struct ibv_exp_res_domain *res_dom, struct ibv_exp_destroy_res_domain_attr *attr)
Use the res_domain field and relevant comp_mask in the ibv_exp_cq_init_attr and ibv_exp_qp_init_attr structs to pass resource domain to the CQ and QP upon their creation.
Example - creating a CQ and QP that may be called from one thread only:
struct ibv_exp_res_domain_init_attr res_domain_attr;
struct ibv_exp_res_domain *res_domain;
struct ibv_exp_cq_init_attr cq_attr;
struct ibv_exp_qp_init_attr qp_attr;
…
res_domain_attr.comp_mask = IBV_EXP_RES_DOMAIN_THREAD_MODEL |
IBV_EXP_RES_DOMAIN_MSG_MODEL;
res_domain_attr.thread_model = IBV_EXP_THREAD_SINGLE;
res_domain_attr.msg_model = IBV_EXP_MSG_HIGH_BW;
res_domain = ibv_exp_create_res_domain(ctx->context, &res_domain_attr);
if (!_domain) {
fprintf(stderr, "Can't create resource domain\n");
exit(1);
}
…
cq_attr.res_domain = res_domain;
cq_attr.comp_mask |= IBV_EXP_CQ_INIT_ATTR_RES_DOMAIN;
cq = ibv_exp_create_cq(context, num_ entries, NULL, NULL, 0, &cq_attr);
…
qp_attr.res_domain = res_domain;
qp_attr.comp_mask |= IBV_EXP_QP_INIT_ATTR_RES_DOMAIN;
qp = ibv_exp_create_qp(context, &qp_attr);