RoCE Time-Stamping allows you to stamp packets when they are sent to the wire or when they are received from the wire. The time stamp is given in raw hardware cycles, but can easily be converted into hardware-referenced nanoseconds-based-time. Additionally, it enables you to query the hardware for the hardware time, thus stamp other application’s event and compare time.
RoCE Time-Stamping Experimental vs. RDMA-Core Verbs and Device Attributes
| Experimental | RDMA-Core | ||
|---|---|---|---|
| Verbs | |||
| ibv_exp_query_values | ibv_query_rt_values_ex | ||
IBV_WXP_VALUES_HW_CLOCK_NS | ibv_query_rt_values_ex | ||
IBV_EXP_VALUES_HW_CLOCK | ibv_query_rt_values_ex | IBV_VALUES_MASK_RAW_CLOCK | |
IBV_EXP_VALUES_CLOCK_INFO | mlx5dv_get_clock_info | ||
| Ibv_exp_cqe_ts_to_ns | mlx5dv_ts_to_ns | ||
| Ibv_exp_create_cq | ibv_create_cq_ex | ||
IBV_EXP_CQ_TIMESTAMP | IBV_WC_EX_WITH_COMPLETION_TIMESTAM | ||
| IBV_EXP_CQ_TIMESTAMP_TO_SYS_TIME | IBV_WC_EX_WITH_COMPLETION_TIMESTAMP_WALLCLOCK | ||
| Ibv_exp_poll_cq | ibv_start_poll | ||
|
| ||
| Ibv_exp_query_device | ibv_query_device_ex | ||
| timestamp_mask | completion_timestamp_mask | ||
| hca_core_clock | hca_core_clock | ||
| Device Attributes | |||
| IBV_EXP_DEVICE_ATTR_WITH_TIMESTAMP_MASK | Device attributes are not needed in RDMA-Core. | ||
IBV_EXP_DEVICE_ATTR_WITH_HCA_CORE_CLOC | |||
Relevant Man Pages
- ibv_query_rt_values_ex: https://github.com/linux-rdma/rdma-core/blob/master/libibverbs/man/ibv_query_rt_values_ex.3
- mlx5dv_get_clock_info: https://github.com/linux-rdma/rdma-core/blob/master/providers/mlx5/man/mlx5dv_get_clock_info.3
- mlx5dv_ts_to_ns: https://github.com/linux-rdma/rdma-core/blob/master/providers/mlx5/man/mlx5dv_ts_to_ns.3
- ibv_create_cq_ex: https://github.com/linux-rdma/rdma-core/blob/master/libibverbs/man/ibv_create_cq_ex.3
Example
See example in libibverbs/examples/rc_pingpong.c
