NVIDIA Tegra
NVIDIA Jetson Linux Developer Guide
32.4.3 Release

 

Program Trace Macrocell

 
Using the PTM
Decode Trace Example
A Program Trace Macrocell (PTM) is a real-time trace module providing instruction tracing of an ARM® core. It traces way-points in:
All indirect branches conditional
Unconditional direct branches
All exceptions
PTM also generates processing elements. This way the PTM module is triggered when there is a deviation from the normal code flow. The ETM v4 in NVIDIA® Jetson™ TX2 series and NVIDIA® Jetson™ TX1 devices, and protocols use waypoints and correlate the trace to the code image, vmlinux. These PTM packets can be decoded later using mem_parser to obtain the complete CPU instruction flow.
PTM is advantageous may be necessary for:
Debugging - On many production systems it is not possible to connect JTAG devices. In these cases you can use PTM to check which instructions ran before the CPU crashed.
Code Profiling - Because there is no Watchdog Timer (WDT) reset, code profiling checks are possible, provided a start and end address.
Performance Analysis - PTM has no performance overhead as compared with ftrace. PTM can also provide instruction counts and timestamps that can be used for performance analysis.
PTM is a component of the ARM coresight framework. Detailed information about the framework is available at:
https://www.kernel.org/doc/Documentation/trace/coresight.txt
To configure PTM drivers
1. Verify that the coresight drivers have been enabled in the kernel as follows:
# zcat /proc/config.gz | grep -i coresight | grep "=y"
CONFIG_CORESIGHT=y
CONFIG_CORESIGHT_LINKS_AND_SINKS=y
CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
CONFIG_CORESIGHT_SINK_TPIU=y
CONFIG_CORESIGHT_SINK_ETBV10=y
CONFIG_CORESIGHT_SOURCE_ETM4X=y
2. Verify that the devices defined in the DTB exist in:
/sys/bus/coresight/devices/.
3. Execute the commands:
# ll /sys/bus/coresight/devices/ | cut -d " " -f10,11,12
8010000.funnel_major -> ../../../devices/8010000.funnel_major/8010000.funnel_major/
8030000.etf -> ../../../devices/8030000.etf/8030000.etf/
8040000.replicator -> ../../../devices/8040000.replicator/8040000.replicator/
8050000.etr -> ../../../devices/8050000.etr/8050000.etr/
8060000.tpiu -> ../../../devices/8060000.tpiu/8060000.tpiu/
8820000.funnel_minor -> ../../../devices/8820000.funnel_minor/8820000.funnel_minor/
9010000.funnel_bccplex -> ../../../devices/9010000.funnel_bccplex/9010000.funnel_bccplex/
9840000.ptm -> ../../../devices/9840000.ptm/9840000.ptm/
9940000.ptm -> ../../../devices/9940000.ptm/9940000.ptm/
9a40000.ptm -> ../../../devices/9a40000.ptm/9a40000.ptm/
9b40000.ptm -> ../../../devices/9b40000.ptm/9b40000.ptm/

Using the PTM

You can enable and disable trace capture at runtime. The content of the ETF/ETR on Jetson TX2 or Jetson TX1 devices, during the period that trace capture is enabled, can be read back to parse.
Note:
PTM is not supported for Denver cores.
To collect PTM trace on A57 Core 0
On the target, execute the command:
# echo 1 > /sys/bus/coresight/devices/8030000.etf/enable_sink
# echo 1 > /sys/bus/coresight/devices/9840000.ptm/enable_source
[ 246.479271] coresight-tmc 8030000.etf: TMC enabled
[ 246.484362] coresight-funnel 8010000.funnel_major: FUNNEL inport 0 enabled
[ 246.491890] coresight-funnel 9010000.funnel_bccplex: FUNNEL inport 1 enabled
[ 246.499761] coresight-etm4x 9840000.ptm: ETM tracing enabled
# cat /sys/bus/coresight/devices/8030000.etf/status
Depth: 0x2000
Status: 0x1
RAM read ptr: 0x5b60
RAM wrt ptr: 0x5b60
Trigger cnt: 0x0
Control: 0x1
Flush status: 0x0
Flush ctrl: 0x133
Mode: 0x0
PSRC: 0x0
DEVID: 0x580
# echo 0 > /sys/bus/coresight/devices//9840000.ptm/enable_source
[ 264.323280] coresight-etm4x 9840000.ptm: ETM tracing disabled
[ 264.329400] coresight-funnel 9010000.funnel_bccplex: FUNNEL inport 1 disabled
[ 264.336998] coresight-funnel 8010000.funnel_major: FUNNEL inport 0 disabled
[ 264.360311] coresight-tmc 8030000.etf: TMC disabled
# dd if=/dev/8030000.etf of=~/cstrace-t186.bin
On the host, execute the command:
$ ./mem_parser --formatter --etb <dumped_file_with_trace buffer> --elf <vmlinux_with_path> > <output_file_with_instructions>
For example:
$ ./mem_parser --formatter --etb ~/cstrace-t186.bin --elf $TEGRA_TOP/out/l4t-t186ref-debug-aarch64/nvidia/kernel/vmlinux > ./cstrace-t186.disassemble.txt
To collect PTM trace on other three A57 cores:
The correspond commands for other 3 cores in A57 are as follows.
Core 1 on A57:
echo 1 > /sys/bus/coresight/devices/8030000.etf/enable_sink
echo 1 > /sys/bus/coresight/devices/9940000.ptm/mode [set EXC bit i.e. bit 24 in ETMTECR1 to trace all memory. Refer *etm*architecture*.pdf, ignore this if you don't know the result.]
echo 1 > /sys/bus/coresight/devices/9940000.ptm/enable_source (enable tracing)
cat /sys/bus/coresight/devices/8030000.etf/status [Will show RAM wrt ptr moving]
echo 0 > /sys/bus/coresight/devices//9940000.ptm/enable_source (disable tracing)
Core 2 on A57:
echo 1 > /sys/bus/coresight/devices/8030000.etf/enable_sink
echo 1 > /sys/bus/coresight/devices/9a40000.ptm/mode [set EXC bit i.e. bit 24 in ETMTECR1 to trace all memory. Refer *etm*architecture*.pdf, ignore this if you don't know the result.]
echo 1 > /sys/bus/coresight/devices/9a40000.ptm/enable_source (enable tracing)
cat /sys/bus/coresight/devices/8030000.etf/status [Will show RAM wrt ptr moving]
echo 0 > /sys/bus/coresight/devices//9a40000.ptm/enable_source (disable tracing)
Core 3 on A57:
echo 1 > /sys/bus/coresight/devices/8030000.etf/enable_sink
echo 1 > /sys/bus/coresight/devices/9b40000.ptm/mode [set EXC bit i.e. bit 24 in ETMTECR1 to trace all memory. Refer *etm*architecture*.pdf, ignore this if you don't know the result.]
echo 1 > /sys/bus/coresight/devices/9b40000.ptm/enable_source (enable tracing)
cat /sys/bus/coresight/devices/8030000.etf/status [Will show RAM wrt ptr moving]
echo 0 > /sys/bus/coresight/devices//9b40000.ptm/enable_source (disable tracing)

Decode Trace Example

The following example is for mem_parser Version 2.0.9
Parsing Pogram Trace Macrocell buffer (ETM4.x) ......
etb buffer size: 32768
>>>>>>>>>>>>>>>>>>>>>>>>>>> CPU: ID = 0x20 start:
etb buffer size: 30404
..........
..........
ATOM packet format 2:
ATOMS: N, E,
<0fc7:da >
<__do_softirq+0x130>
ffffffc0000a8b98: 910022b5
ffffffc0000a8b9c: 1ad326d6
ffffffc0000a8ba0: 5ac002d3
ffffffc0000a8ba4: 6b1f02df
ffffffc0000a8ba8: 5ac01273
ffffffc0000a8bac: 1a9307f3
E ffffffc0000a8bb0: 35fffb73 cbnz x19, ffffffc0000a8b1c <__do_softirq+0xb4>
<__do_softirq+0xb4>
ffffffc0000a8b1c: 928000e0
ffffffc0000a8b20: 8b33cc00
ffffffc0000a8b24: 8b0002b5
ffffffc0000a8b28: 91080357<__do_softirq+0xb4>