itrace Utility
The itrace utility extracts and prints trace messages generated by the firmware of a ConnectX-2/ ConnectX-3/ConnectX-3 Pro adapter cards. These trace messages inform developers of software drivers about internal status, events, critical errors, etc., for each iRISC. Trace messages generated by iRISCs are stored in the trace buffer. The trace buffer is located in host memory for MemFree adapter cards (i.e., without on-board memory), and in adapter memory for adapter cards with on-board memory.
The utility is a command line application controlled by command line parameters. It prints trace messages in text format to the console.
In order to print the firmware traces, the following are required:
Debug firmware is burnt and loaded on the device
The driver is up, meaning:
For adapters with on-board memory: The SYS_ENABLE command has been executed
For adapters without on-board memory (MemFree): The RUN_FW command has been executed
The desired trace mask is set (see the -m flag below)
The mst driver must be started prior to running itrace tool. To start itrace:
Start the mst driver (mst start or mst restart).
Enter the following command:
# itrace [options...] IRISC_NAME
where:
IRISC_NAME | The iRISC for which traces are to be printed. This can be specified once anywhere in the command line as a special option without the leading hyphen. Run ‘itrace -h’ to get a list of iRISC names for each adapter device. |
-h, --help | Displays help about itrace usage. |
-m --mask=TRACE_MASK | Sets the Trace Mask. |
To enable generating trace messages for an iRISC, the trace_mask register must be set according to the specifications in the device’s Programmer’s Reference Manual. Setting or clearing bits of the trace_mask register enables or disables, respectively, the generation of specific types of trace messages.
The trace_mask parameter must be either a hexadecimal or a decimal number and its value will be written into the trace_mask register. Changing the trace_mask parameter will not change or remove messages previously stored in the trace buffer, so disabled types of messages can still be displayed by itrace if they were previously generated.
-w, --wait | Runs itrace in wait mode. itrace will exit only if you press <Ctrl-C>. This is not the default behavior of itrace. Without the -w option, itrace will exit if there have been no |
-d, --device=DEVICE | Specifies the name of the mst device driver for accessing the cr-space. The default value is:/dev/mst/mt4099_pci_cr0. |
-l, --nolock | Ignore NSI GW lock |
--nomap | Sets the itrace to not access memory directly (via memory mapping) for reading the trace buffer, but to use the adapter memory access Gateway instead. By default, itrace |
--no-propel | Sets the itrace not to animate the propeller in wait mode (-w option). By default, animation is enabled. |
-v, --version | Prints the MFT version and exits. |
-c, --color | Enables color in trace output. |
-D, --dump | Dumps the trace buffer and exits. This option is useful for debugging itrace; it dumps the contents of the trace buffer in row format. |
--start=START_NO | Sets first message number to display. |
--debug=TSTRING | Control trace. See: --help-debug. |
--help-debug | Prints trace usage. |
For Linux, device names should be listed with the /dev/mst prefix. For Windows, no prefix is required.
Example:
itrace -d /dev/mst/mt4099_pci_cr0 sx1
itrace: read memory (174712 bytes), each dot denotes 2048 bytes:
[.....................................................................................]
IRISC Trace Viewer (Mellanox ConnectX), mft 4.1.0-26, built on Aug 16 2015, 17:32:24. Git SHA
Hash: dd3f359
FW Version: 2.34.8420 09/08/2015 19:44:8
(00000003 c1b59e4e) SCHD: exeqpc_valid2freed(0x0) vec_busy_valid=0x00000010
(00000004 dda895e4) SCHD: SQP:0x000400 exes_super_scheduler:busy_done
(00000005 dda89760) SCHD: writing QpState SQPSTATE_GOOD_IDLE!!!!
(00000006 dda89868) SCHD: exeqpc_valid2freed(0x0) vec_busy_valid=0x00000010
(00000007 dda97ccf) SCHD: SQP:0x000400 exes_super_scheduler:busy_
(00000008 dda97e47) SCHD: writing QpState SQPSTATE_GOOD_IDLE!!!!
(00000009 dda97f4f) SCHD: exeqpc_valid2freed(0x0) vec_busy_valid=0x00000010
(0000000a dda9a8f6) SCHD: SQP:0x000400 exes_super_scheduler:busy
(0000000b dda9aa6e) SCHD: writing QpState SQPSTATE_GOOD_IDLE!!!!
(0000000c dda9ab79) SCHD: exeqpc_valid2freed(0x0) vec_busy_valid=0x00000010
(0000000d ddaaadc1) SCHD: SQP:0x000400 exes_super_scheduler:busy_
(00000029 ddaee521) INFO: IPCdata[00]=0x01abcd0a(0000002a ddaee60c) INFO: IPCdata[01]=0x00000014
(0000002b ddaee8ce) MAD: exes_mad: QPN=0x000000, nda_nds=0x7c58d014
(0000002c ddaee9f2) SCHD: SQP:0x000000 sqpc_access_db_algorithm: INC
(0000002d ddaef0d5) SCHD: exes_scheduler: try to insert
(0000002e ddaef2d9) SCHD: SQP:0x000000 exes_scheduler chosen
(0000002f ddaef6aa) SCHD: EXES_GO(0x0)..