The wqdump utility dumps device internal work queues. A work queue is an object containing a Queue Pair Context (QPC) which contains control information required by the device to execute I/O operations on that QP, and a work queue buffer which is a virtually-contiguous memory buffer allocated when creating the QP.
The dumped data can be used for hardware troubleshooting. It can be applied on ConnectX adapter cards family and Connect-IB adapter devices.
wqdump on ConnectX-3 and ConnectX-3 Pro is not supported against in-band devices.
wqdump Usage
The mst driver must be started prior to running the wqdump utility. To start the wqdump utility:
- Start the mst driver (mst start or mst restart).
Run wqdump:
# WQDump <-d|--device DeviceName> <--source ContextType> [--gvmi Gvmi] [--qp ContextNumber] <--dump DumpType> [--fi StartIndex] [--num NumberOfItems] [--format Format] [--address Address] [--size Size] [-v|--version] [-h|--help] [--clear_semaphore] [--gw_access]
where:
| --d|--device DeviceName | Device name |
| --source ContextType | Type of context to dump. Options are: Snd, Rcv, Cmp, Srq, Eqe, Connect-X3/Pro: MCG, 5th generation devices: MKC, SXDC, FullQp ConnectX-5: CMAS_QP_WQE, CMAS_QP_SWQ, CMAS_SRQ_WQE, CMAS_CQ_BUFF, CMAS_QP_DBR, CMAS_QP_SDB, CMAS_SRQ_DB, CMAS_CQ_DBR, CMAS_CQ_ARM, CMAS_EQ_BUFF, CMAS_TAG_MATCH. CMAS_INLINE |
| --gvmi Gvmi | Guest VM ID (5th generation devices) |
| --qp ContextNumber | Context number to dump |
| --dump DumpType | Dump Type. Options are: WQ, QP, WQ_QP, ALL_QPC, ALL_VALID_QPNS, ICM |
| --fi StartIndex | Index of first element to dump, (Default:0) |
| --num NumberOfItems | Number of elements to dump from buffer, (Default: keep reading) |
| --format Format | Output format: options are : text, raw, dw, (Default: text) |
| --address Address | Memory Address |
| --size Size | Memory size in bytes |
| -v|--version | Show tool version and exit |
| -h|--help | Show usage |
| --clear_semaphore | Force clear semaphore |
| --gw_access | Force get QPC by GW access (Connect-X 3/Pro) |
4th Generation Device Examples:
Print all valid qpns
The example below will dump all valid qpns of type mcg context.
# wqdump -d /dev/mst/mt4099_pci_cr0 --source mcg --dump ALL_VALID_QPNS
Dump mcg qp
The example below will dump mcg context number 0x10.
# wqdump -d /dev/mst/mt4099_pci_cr0 --source mcg --dump QP -qp 0x10
Dump other qpns
The example below will dump snd context number 0x10 in a raw format.
# wqdump -d /dev/mst/mt4099_pci_cr0 --source snd --dump QP -qp 0x10 --format raw
Dump wq
The example below will dump send work queue buffer number 0x42.
# wqdump -d /dev/mst/mt4099_pci_cr0 --source snd --dump wq -qp 0x42
Dump mcg qp by GW access
The example below will dump mcg context number 0x10 by GW access.
# wqdump -d /dev/mst/mt4103_pci_cr0 --source mcg --dump QP -qp 0x10 --gw_access
5th Generation Device Examples:
FullQp: The QP context of the Rcv and Snd with the common part. Note, FullQp does not have dump as WQ.
• Get opened contexts from the first 20 indexes:
# wqdump -d /dev/mst/mt4117_pciconf0 --source FullQp --dump ALL_VALID_QPNS --num 20 Numbers of valid contexts (in the range 0x0 - 0x13): index 0x00000003 index 0x00000006 index 0x00000007 index 0x0000000b index 0x0000000c index 0x00000013 ----------------------------------------- Number of valid contexts: 6 -----------------------------------------
• Show the QP Context (RAW):
# wqdump -d /dev/mst/mt4117_pciconf0 --source FullQp --dump QP --qp 0x0000000b --format raw == Common Part (Not Connected) == 0. 80000000 0000001e 05000000 00000000 1. 70000000 00000000 00000000 00000000 2. 0000000b 00ffffff 00000000 00000000 3. 00000000 00000000 00000000 80010000 -------------------------- Send Qpc gvmi 0000 QP Index 0000000b 0. 80000000 0000001e 05000000 00000000 1. 70000000 00000000 00000000 00000000 2. 0000000b 00ffffff 00000000 00000000 3. 00000000 00000000 00000000 80010000 -------------------------- == Responder Part (Not Connected (mac)) == Recv Qpc gvmi 0000 QP Index 0000000b 0. b8eccd02 00000000 d8f5cd02 00000000 1. d0010000 00000000 40000000 00000000 2. e0e03903 00000000 f0e03903 00000000 3. 00000000 00000000 00000000 00000000 --------------------------
SRQ
• Opened QPs:
# wqdump -d /dev/mst/mt4115_pciconf0 --source Srq --dump all_valid_qpns Numbers of valid contexts (in the range 0x0 - 0xffffff): gvmi 0x0000 index 0x00000060 gvmi 0x0000 index 0x00000061 gvmi 0x0000 index 0x00000066
• Dump WQs:
0x00000066# wqdump -d /dev/mst/mt4117_pciconf0 --source Srq --dump wq --qp 0x60 [Element Index 0] -------------- SRQ Next ------------------ next_wqe_index : 0x1 signature : 0x0 -------------- scatter entry (0) --------------- byte_count : 0x0 wqe_inline : 0x0 local_key : 0x0 local_address_63_32 : 0x0 local_address_31_0 : 0x0 [Element Index 0x1] -------------- SRQ Next ------------------ next_wqe_index : 0x0 signature : 0x0 -------------- scatter entry (0) --------------- byte_count : 0x0 wqe_inline : 0x0 local_key : 0x0 local_address_63_32 : 0x0 local_address_31_0 : 0x0
CMAS
• Opened contexts from some CMAS type (CMAS_EQ_BUFF for ex):
#wqdump -d /dev/mst/mt4119_pciconf0 --source CMAS_EQ_BUFF --dump ALL_VALID_QPNS Numbers of valid contexts (in the range 0x0 - 0xffffff): gvmi 0x0000 index 0x00000002 gvmi 0x0000 index 0x00000010 gvmi 0x0000 index 0x00000011 gvmi 0x0000 index 0x00000012 gvmi 0x0000 index 0x00000013 gvmi 0x0000 index 0x00000014 gvmi 0x0000 index 0x00000015
• Dump raw data:
# wqdump -d /dev/mst/mt4119_pciconf0 --source CMAS_EQ_BUFF --dump QP --qp 0x15 --format raw CMAS gvmi 0000 CMAS Index 00000015 0. 80000002 00000000 00000000 e4f80000 1. 00000000 00000000 00000000 00000000 2. 00000000 00000000 00000000 00000000 3. 00000000 00000000 00000000 00000000 --------------------------
