Jetson Module Adaptation and Bring-Up: Jetson Xavier NX Series ¶
This topic is for engineers who are developing production software for an NVIDIA Jetson Xavier NX series module. It describes how to port NVIDIA Jetson Linux and Bootloader from a Jetson Xavier NX Developer Kit to a custom hardware platform and bring up a working system on the new platform.
The examples described include code for the Jetson Xavier NX Developer Kit (P3518).
For information on customizing the configuration files, see the section MB1 Platform Configuration .
Two other topics are closely related to this one:
-
Jetson Module Adaptation and Bring-Up: Checklists contains lists of steps to perform to complete the parts of the adaptation task.
Board Configuration ¶
Jetson Xavier NX series has the following variants:
-
Jetson Xavier NX (development, P3668-0000), with 8 GB of RAM and an on-module microSD card socket. Is intended only for software development, and is sold only as a component of Jetson Xavier NX Developer Kit. You can program it by flashing a microSD card in place, or by flashing it on a host computer and then installing it on the module.
-
Jetson Xavier NX (production, P3668-0001), with 8 GB of RAM but no microSD card socket. This module is sold as a separate unit. It is intended for production use but may also be used in a Jetson Xavier NX Developer Kit. You program it in the conventional way, by flashing software to on-board eMMC memory.
-
Jetson Xavier NX 16GB (P3668-0003), with 16 GB of RAM. This variant is identical to P3668-0001 in all respects except RAM size.
Jetson Xavier NX Developer Kit, P3518, includes a P3509-0000 reference carrier board as well as the P3668-0000 SOM. The SOM and carrier board each have an EEPROM where the board ID is saved. The SOM can be used without any software configuration modifications.
Jetson Xavier NX series modules have a provision for attaching a heatsink or fansink. The module provided with a Jetson Xavier NX Developer Kit has a fansink already attached. See the Jetson Xavier NX Thermal Design Guide for guidance on designing or procuring a thermal solution for production Jetson Xavier NX series modules.
Before using the SOM with a carrier board other than P3509, you must change the kernel device tree, MB1 configuration, ODM data, and flashing configuration to accommodate the new carrier board instead of P3509. An EEPROM ID for your custom board is not required.
You can also use the reference carrier board P3449-0000 (included in the Jetson Nano Developer Kit) with a Jetson Xavier NX series module. The Jetson Linux installer detects the type of module and carrier board and automatically installs the appropriate drivers and other device-specific components.
Board Naming ¶
To support a Jetson Xavier NX series module with your custom carrier board, you
must assign the module/carrier board combination a
board name
. This
name must consist of lower case letters, numerals, hyphens, and
underscores. You must set the board name in the environment variable
BOARD
on your host system.
Some examples of valid board names are
p3509-0000+p3668-0000
,
p3518-0000-devkit
, and
devboard
.
The installation process uses the board name in filenames and pathnames, including names in the device tree and proc file system.
The placeholder
is used to represent the board name throughout
this topic. Where a command contains
,
substitute the name of your custom board.
Note
Do not modify files whose filenames and pathnames contain your system’s default board name. The default board name refers to the carrier board provided with your Jetson developer kit. If it appears in a file’s name, that file contains code or data that is specific to that board. Make copies of such files that are named with your custom board’s board name, and modify them to support your custom board.
Root File System Configuration ¶
Jetson Linux can use any standard or customized Linux root file system (rootfs) that is appropriate for its targeted embedded applications.
However, certain settings must be configured in the rootfs’s boot-up framework to set default configuration after boot, or some of the core functionalities will not run as expected.
For example:
-
The
nv.shandnvfb.shboot-up scripts do some platform-specific configuration in the kernel. -
The Xorg and X libraries must be correctly configured for the target device.
-
The nvpmodel clock and frequency must be configured for the target device.
Jetson Linux provides these rootfs configurations and customizations in this directory and its subdirectories:
Linux_for_Tegra/nv_tegra/
You must incorporate the relevant customization for your target rootfs from this location.
Note
For the sample Ubuntu root file system provided by NVIDIA, this customization is applied using the script
Linux_for_Tegra/apply_binaries.sh
.
MB1 Configuration Changes ¶
Multiple
.cfg
files define boot time configuration of the hardware. They
are applied by Bootloader. The MB1 boot configuration tables are available at:
/bootloader/t186ref/BCT
Where
is the top-level BSP flashing directory.
Pinmux Changes ¶
If your carrier board schematic differs from that of the reference carrier board in Jetson Xavier NX Developer Kit, you must change the pinmux configuration applied by the software.
To define your board’s pinmux configuration, download the Jetson Xavier NX pinmux table from the Jetson Download Center. Be sure to get the right version of the table for your SOM.
The pinmux table is a spreadsheet that:
-
Shows the locations and default pinmux settings
-
Acts as a data source for software that defines the pinmux settings in the source code or device tree
You must customize the spreadsheet for the configuration of your board, then convert the
.dtsi
file generated by Excel to a
.cfg
file. For instructions, see the README file at:
Linux_for_Tegra/kernel/pinmux/t19x/
You must perform the same conversion for
gpio.dtsi
and
padvoltage.dtsi
.
GPIO Changes ¶
If you designed your own carrier board, to translate from SOM connector pins to actual GPIO numbers you must understand the GPIO mapping formula below. The translated GPIO numbers can be controlled by the driver.
To check a GPIO number ¶
This example procedure checks the GPIO number of signal name
SPI0_CS1
.
-
Open the Jetson Xavier NX pinmux table. (See Pinmux Changes , above.)
-
Search the table for
SPI0_CS1. -
Confirm that the Customer Usage field contains the GPIO name
GPIO3_PZ.07. -
Search PZ.07 using the following command:
cat /sys/kernel/debug/gpio | grep PZ.07
-
For example Output:
gpio-xxx (PZ.07 )
-
GPIO number of
SPI0_CS1is xxx.
Note: To use a pin as GPIO, make sure that E_IO_HV field is disabled in corresponding pinmux register of the GPIO pin. You can disable the field 3.3V Tolerance Enable in pinmux spreadsheet and reflash the board with the updated pinmux file.
PMIC Changes ¶
The PMIC configuration file configures the initial PMIC in the P3668 SOM. It also defines some GPIO expander-based GPIO regulator settings in the P3509 carrier board configurations. Review this configuration file to replace any references to the P3509 carrier board with references to your custom board. If required, include any regulator information to enable this file.
For example, the following settings configure PMIC to turn on SD4 on I2C
address
0x3c
. The example is from the configuration file
tegra194-mb1-bct-pmic-p3668-0001-a00.cfg
:
######################## System Configurations ####
# PMIC FPS to turn SD4 (VDD_DDR_1V1) on in time slot 0
# PMIC FPS to set GPIO2 (EN_DDR_VDDQ) high in time slot 1
# Set SLPEN = 1 and CLRSE on POR reset
pmic.system.block[0].type = -1; # I2C
pmic.system.block[0].controller-id = 4;
pmic.system.block[0].slave-add = 0x78; # 7BIt:0x3c
pmic.system.block[0].reg-data-size = 8;
pmic.system.block[0].reg-add-size = 8;
pmic.system.block[0].block-delay = 10;
pmic.system.block[0].commands[0].0x53.0x38 = 0x00; #_SD4 FPS UP slot 0
pmic.system.block[0].commands[1].0x55.0x38 = 0x10; #_GPIO2 FPS UP slot 2
pmic.system.block[0].commands[2].0x41.0x1C = 0x1C; #_SLPEN=1, CLRSE = 11
Porting the Linux Kernel ¶
It is assumed that you are using a P3668 SOM connected to an unmodified P3509
carrier board; the QSPI, PMIC, DDR, and routing of lines are all unchanged.
The modifications you are
making are for the SOM and the carrier board. Consequently, based on the
peripherals present on your carrier board, you can modify the
.dts
files
by disabling/enabling the controllers and changing the supplies.
To port the kernel configuration code (the device tree) to your platform, modify one of the distributed configuration files to describe the design of your platform.
The configuration files are available at:
/hardware/nvidia/platform/t19x/
/hardware/nvidia/soc/t19x
The final DTB file used is:
tegra194-p3668-0000-p3509-0000.dtb
By reading the above file, you see which other
.dtsi
files are
referenced by include statements. The following table describes common
.dtsi
files that may be modified to reflect hardware design changes.
|
Types of Changes |
DTSI Filename or location |
|---|---|
|
Power supply changes |
|
|
Regulator parameter changes |
|
|
Display panel and node changes |
For details, see the topic Display Configuration and Bringup . |
|
ODM data based feature configuration |
|
|
NVIDIA SoC controller state to enable/disable a controller |
|
|
Panels related .dts files |
|
Verify that no other
.dts
or
.dtsi
file, including these
.dts
files,
overrides any changes you make.
As a best practice, create your own set of
.dts
files based on the
NVIDIA
®
Jetson AGX Xavier™
files already present. Rename your
newly created files to the name of your board.
Note
Use
fdtdump
or
dtc
to generate a
.dts
file from the final
.dtb
file and check whether your changes have taken effect.
The command usage is:
$ dtc -I dtb -O dts tegra194-p3668-all-p3509-0000.dtb > tegra194-p3668-all-p3509-0000.dts
$ fdtdump dts tegra194-p3668-all-p3509-0000.dtb >tegra194-p3668-all-p3509-0000.dts
PCIe Controller Configuration ¶
This section provides information about PCIe controller configurations.
The PCIe host controller is based on Synopsis Designware PCIe intellectual property, and thus inherits all the common properties defined in the information file at:
$(KERNEL_TOP)/nvidia/Documentation/devicetree/bindings/pci/nvidia,tegra19x-pcie.txt
PCIe Controller Features ¶
Xavier SoC has six PCIe controllers that are supported by the HSIO and NVHS UPHY bricks. All of these controllers cannot be enabled at the same time, refer to UPHY Lane Assignment for a list of the supported PCIe configurations.
Here is a list of the PCIe controllers that are supported by Jetson Xavier NX series and their specifications:
|
PCIe Controller |
Controller mode |
Supported link width |
Supported link speed |
|---|---|---|---|
|
PCIe C0 |
Root port |
x4 |
Gen4 |
|
PCIe C1 |
Root port |
x1 |
Gen4 |
|
PCIe C3 |
Root port |
x1 |
Gen4 |
|
PCIe C5 |
Dual mode |
up to x8 |
Gen4 |
-
Dual Mode PCIe Controllers : The C5 Controller supports dual mode, and it can be configured as a Root Port or an Endpoint. Mode selection is fixed during flashing and dynamic switching of the mode during runtime is not possible.
-
ASPM : All controllers support ASPM L0s, L1, L1.1, and L1.2.
Porting Universal Serial Bus ¶
The Jetson Xavier NX series modules can support one enhanced SuperSpeed Universal Serial Bus (USB) port and up to three High Speed USB ports. The Jetson Xavier NX P3509 carrier board is designed and verified for one USB3.1 port. If you design your own carrier board, verify the compatibility between P3509 and your own board by consulting the NVIDIA team.
USB Structure ¶
An enhanced SuperSpeed USB port has nine pins:
-
VBUS.
-
GND.
-
D+.
-
D−.
-
Two differential signal pairs for SuperSpeed data transfer.
-
One ground (
GND_DRAIN) for drain wire termination and managing EMI, RFI, and signal integrity.
The
D+
/
D−
signal pins connect to UTMI pads. The
SSTX
/
SSRX
signal pins
connect to UPHY and are handled by a single UPHY lane. As UPHY lanes are
shared between PCIE, SATA, UFS, and XUSB, UPHY lanes must be assigned
according to the custom carrier board’s requirements.
Required Device Tree Changes ¶
This section gives step-by-step guidance for checking schematics and configuring USB ports in the device tree. All of the examples are based on the design of the Jetson Xavier NX P3509 carrier board.
For a Host-Only Port ¶
This section uses U7, a USB 3.1 Gen2 Realtek Enhanced SuperSpeed on-board hub, model number RTS5489-GR (hereafter “ Realtek hub ”) as an example of a host-only port.
Go Through the Schematics ¶
Note
The P3509 carrier board’s schematic file is included in the Jetson Xavier NX Developer Kit Carrier Board Design Files , available from the Jetson Download Center.
Check the Realtek hub on the P3509 carrier board and find the socket location wired to the P3668 SOM.
-
USB2 signal pins
D+/D-(USP_D_P/USP_D_N) wire out from U7 and lead to module socket pins 117 (USB1_D) and 115 (USB1_D).
-
USB3 differential pairs (
USP_TX_*andUSP_RX_*) wire out from U7 and lead to module socket pins 161 (SBSS_RX) and 168 (USBSS_TX).
Through the schematic, you can conclude that for U7:
-
The USB2 signal pair is wired to UTMI pad 1 (USB2 port 1).
-
The USB3 signal pairs are wired to UPHY lane 1 (USB 3.1 port 2).
The xusb_padctl Node ¶
The device tree’s
xusb_padctl
node follows the conventions of
pinctrl-bindings.txt
. It contains two groups, named
pads
and
ports
,
which describe USB2 and USB3 signals along with parameters and port
numbers. The name of each parameter description subnode in
pads
and
ports
must be in the form
, where
is
"usb2"
or
"usb3"
, and
is the associated port number.
The properties of the
pads
subnode are:
-
nvidia,function: A string containing the name of the function to mux to the pin or group. Must be"xusb".
The properties of the
ports
subnode are:
-
mode: A string that describes USB port capability. A port for USB2 must have this property. It must be one of these values:-
host -
peripheral -
otg
-
-
nvidia,usb2-companion: USB2 port (0, 1, or 2) to which the port is mapped. A port for USB3 must have this property. -
nvidia,oc-pin: The overcurrentVBUSpin the port is using. The value must be positive or zero.Note
Overcurrent detection and handling for U7, a Realtek hub on the P3509 carrier board, are controlled by the hub itself. Therefore, you need not set this property.
-
vbus-supply:VBUSregulator for the corresponding UTMI pad. Set to&battery_regfor a dummy regulator.Note
The Realtek hub is always connected to the root hub port on a P3509, so you need not control hub power. You just need enable it with
VDD_3V3_SYS. Therefore, you must set dummy regulators for U7 on the P3509 carrier board.
For the detailed information about
xusb_padctl
, see the documentation at:
kernel/kernel-5.10/Documentation/devicetree/bindings/phy/nvidia,xusb-padctl.txt
As an example, consider U7, the Realtek hub, which is always connected to USB2 port 1 and USB3 port 2 on the root hub. Create a pad/port node and property list for U7 based on the device tree structure described above:
xusb_padctl: xusb_padctl@3520000 {
...
pads {
usb2 {
lanes {
usb2-1 {
nvidia,function = "xusb";
status = "okay";
};
...
};
};
usb3 {
lanes {
usb3-2 {
nvidia,function = "xusb";
status = "okay";
};
};
};
};
ports {
...
usb2-1 {
mode = "host";
vbus-supply = <&battery_reg>;
status = "okay";
};
...
usb3-0 {
nvidia,usb2-companion = <1>;
status = "okay";
};
};
};
Under the xHCI Node ¶
The Jetson Xavier NX xHCI controller complies with xHCI specifications, which support both USB 2.0 High Speed / Full Speed / Low Speed and USB 3.1 SuperSpeed protocols.
The properties of the node are:
-
phys: Must contain an entry for each entry inphy-names. -
phy-names: Must include an entry for each PHY used by the controller. Names must be in the form- "usb2"or"usb3", and -
nvidia,xusb-padctl: A pointer to thexusb-padctlnode.
For detailed information about xHCI, see the documentation at:
kernel/kernel-5.10/Documentation/devicetree/bindings/usb/nvidia,tegra-xhci.txt
Consider U7, the Realtek hub, as an example. Create an xHCI node and property list for U7 based on the device tree structure described above:
tegra_xhci: xhci@3610000 {
...
phys = <&{/xusb_padctl@3520000/pads/usb2/lanes/usb2-1}>,
<&{/xusb_padctl@3520000/pads/usb3/lanes/usb3-2}>;
phy-names = "usb2-1", "usb3-2";
nvidia,xusb-padctl = <&xusb_padctl>;
status = "okay";
...
};
For an On-The-Go Port ¶
USB On-The-Go, which is often abbreviated USB OTG or just OTG , is a specification that allows USB to act as a host or a device in the same port. A USB OTG port can switch back and forth between the roles of host and device.
This section takes J5, USB2.0 Micro B connector, as an example of an OTG port.
An OTG port adds a fifth pin to the standard USB connector, called the ID pin . An OTG cable has an A-plug on one end and a B-plug on the other end. The A-plug’s ID pin is grounded, while the B-plug’s ID pin is floating. A device with an A-plug inserted becomes and OTG A-device (a host), and a device with a B-plug inserted becomes a B-device (a device).
Note
Because its ID pin is floating, J5 is fixed in the device role in the Jetson Xavier NX Developer Kit. It cannot function as a host, for example, to connect a flash drive, a keyboard, or a mouse.
Go Through the Schematics ¶
Note
The P3509 carrier board’s schematic file is included in Jetson Xavier NX Developer Kit Carrier Board Design Files , available from the Jetson Download Center.
Check the USB connectors on the P3509 carrier board and find the socket location wired to the P3668 SOM.
-
USB2 signal pins D+/D− (
DPandDN) wire out from J5 and lead to pin 111 (USB0_D) and pin 109 (USB0_D) on the SOM socket.
The USB 2.0 Micro B connector, J5, supports only HighSpeed mode, and does not have USB3 signal pairs.
From the schematic, you can see that for J5:
-
The USB2 signal pair is wired to UTMI pad 0 (USB2 port 0).
The USB Connector Class ¶
A USB connector class represents a physical USB connector. It should be a child of a USB interface controller or a separate node when it is attached to the MUX and USB interface controllers.
Generally, port switching between the roles of an OTG port is controlled
by the host driver (xHCI) and device driver (xUDC), and can be defined
by the state of the ID pin and the
VBUS_DETECT
pin.
Taking
GPIO_M3
as the
VBUS_DETECT
pin and
GPIO_Q0
as the ID pin, for example:
-
Find the corresponding GPIO states on the
VBUS_DETECTpin and the ID pin.Generally, the ID pin is designed as internal pull high (logical high). With an A-plug connected the ID pin is pulled to ground (logical low), while with a B-plug connected or no cable connected it remains logical high.
The operation of the
VBUS_DETECTpin depends on the device’s design. Consider the schematic in the following diagram, for example:
With a B-plug connected,
VBUS_DETECTis logical low, because VBUS is provided from an external power supply. When no cable is connected, it is logical high.Note
VBUS_DETECTis initially logical high, then logical low becauseVBUSis provided by the host controller. Therefore, the state of theVBUS_DETECTpin does not matter when the OTG port is operating in host mode. -
Create the table of GPIO states and their corresponding output cable states:
GPIO_Q0 (ID)
GPIO_M3 (VBUS_DETECT)
Data Role
1
1
Not Connected
0
0
HOST
0
1
HOST
1
0
DEVICE
Under the Connector Node ¶
Port switching between the roles of an OTG port is defined by the state
of the ID pin and the
VBUS_DETECT
pin and the settings of the USB
connector class.
The properties of the
extcon
node are:
-
compatible: Value must be"Value must be "gpio-usb-b-connector". -
Label:Symbolic name for the connector. -
type: Size of the connector, should be specified in case of non-full size ‘usb-a-connector’ or ‘usb-b-connector’ compatible connectors. -
id-gpios:An input gpio for USB ID pin. -
vbus-gpios: An input gpio for the USB VBus pin, used to detect the presence of VBUS 5V. -
cable-connected-on-boot: Name of the output cable connected on boot.It should be
USB_ROLE_NONE,USB_ROLE_HOST, andUSB_ROLE_DEVICE. If not specified, the system assumes that no cable is to be connected. -
wakeup-source: A Boolean; true if the device can wake up the system.For detailed information about
USBConnector, see the documentation at:``kernel/kernel-5.10/Documentation/devicetree/bindings/connector/usb-connector.yaml``
-
Create a
USBConnectordevice node and property list based on the device tree structure described above and the table of GPIO states and corresponding output cable states forGPIO_Q0andGPIO_M3:vxusb_padctl: xusb_padctl@3520000 { ... ports { usb2-0 { ... Connector { compatible = "gpio-usb-b-connector"; label = "micro-USB"; type = "micro"; vbus-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(M, 3) GPIO_ACTIVE_LOW>; id-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(Q, 0) GPIO_ACTIVE_HIGH>; }; ... }; }; ...
The USB 2.0 Micro B connector, J5, has the connector’s ID pin floating
and the
VBUS_DETECT
pin of the connector wired out to
GPIO00
, which
corresponds to
GPIO_PZ1
. As a result, J5 can only function in the device role.
This table describes the GPIO states on J5 and the corresponding output: cable states:
|
GPIO_PZ1 (VBUS_DETECT) |
Data Role |
|---|---|
|
1 |
Not Connected |
|
0 |
Device |
This is the
USBConnectorClass
device node and property list based on the device tree structure and the table of GPIO states and corresponding output cable states for GPIO_PZI that is customized for Jetson Xavier NX, where the ID pin that is floating the port is fixed in the device role:
xusb_padctl: xusb_padctl@3520000 {
...
ports {
usb2-0 {
...
Connector {
compatible = "gpio-usb-b-connector";
label = "micro-USB";
type = "micro";
vbus-gpio = <&tegra_main_gpio
TEGRA194_MAIN_GPIO(Z, 1) GPIO_ACTIVE_LOW>;
};
...
};
};
...
};
Note
Check the pinmux spreadsheet for the GPIO that corresponds to the ID pin and
VBUS_DETECT
pin.
Check the pinmux spreadsheet for the GPIO that corresponds to the ID pin and
VBUS_DETECT
pin.
Under the xusb_padctl Node ¶
xusb_padctl
settings for an OTG port are the same as for a host-only
port except that the mode must be
"otg"
.
Taking J5, the USB 2.0 Micro B connector, as an example, create a pad/port node and property list:
xusb_padctl: xusb_padctl@3520000 {
...
pads {
usb2 {
lanes {
usb2-0 {
nvidia,function = "xusb";
status = "okay";
};
...
};
};
...
};
ports {
usb2-0 {
mode = "otg";
vbus-supply = <&battery_reg>;
usb-role-switch;
status = "okay";
Connector {
compatible = "gpio-usb-b-connector";
label = "micro-USB";
type = "micro";
vbus-gpio = <&tegra_main_gpio
TEGRA194_MAIN_GPIO(Z, 1) GPIO_ACTIVE_LOW>;
};
};
...
};
};
Under the xHCI Node ¶
The xHCI settings for an OTG port are the same as for a host-only port:
Taking the USB 2.0 Micro B connector, J51, as an example, create an xHCI node and property list based on the device tree structure described in Under the xHCI Node for a host-only port:
tegra_xhci: xhci@3610000 {
. . .
phys = <&{/xusb_padctl@3520000/pads/usb2/lanes/usb2-0}>;
phy-names = "usb2-0";
nvidia,xusb-padctl = <&xusb_padctl>;
status = "okay";
. . .
};
Under the xUDC Node ¶
The Jetson Xavier NX xUDC controller supports both USB 2.0 High Speed / Full Speed and USB 3.1 SuperSpeed protocols.
-
charger-detector: USB charger detection support. Must be thephandleof the USB charger detection driver DT node. -
phys: An array; must contain pointers to the nodes that define each PHY inphy-names. -
phy-names: An array; must contain an entry for each PHY used by the controller. Names must be in the form- "usb2"or"usb3", and -
nvidia,xusb-padctl: A pointer to thexusb-padctlnode.
For detailed information about xUDC, see the documentation at:
kernel/kernel-5.10/Documentation/devicetree/bindings/usb/nvidia,tegra-xudc.txt
Taking J5, the USB 2.0 Micro B connector, as an example, create an xUDC node and property list for J5 based on the device tree structure described above:
tegra_xudc: xudc@3550000 {
phys = <&{/xusb_padctl@3520000/pads/usb2/lanes/usb2-0}>;
phy-names = "usb2-0";
nvidia,xusb-padctl = <&xusb_padctl>;
status = "okay";
};
The Jetson Nx Xavier Devkit Default PCIe Configuration ¶
|
PCIe Controller |
Controller mode |
Supported link width |
Supported link speed |
Slot |
|---|---|---|---|---|
|
PCIe C4 |
Root port |
X1 |
Gen3 |
M.2 Key E |
|
PCIe C5 |
Dual mode |
x4 |
Gen4 |
M.2 Key M |
Enable PCIe in a Customer CVB Design ¶
-
Select the appropriate UPHY configuration from UPHY Lane Assignment , which suits the CVB design and update ODMDATA accordingly.
-
Refer to Setting Optional Environmental Variablest for more information about how to update ODMDATA.
-
Configure the PCIe reset (PEX_L*_RST_N) and clkreq(PEX_L*_CLKREQ_N) as follows. To update the pinmux value, refer to Changing the Pinmux .
Pinmux Settings for the Root Port Mode
Pinmux
Customer Usage
Pin Direction
3.3V Tolerance Enable
PEX_L*_RST_N
SFIO(PE*_RST_L)
Output
Enable
PEX_L*_CLKREQ_N
SFIO(PE*_CLKREQ_L)
Bidirectional
Enable
Pinmux Settings for Endpoint Mode
Pinmux
Customer Usage
Pin Direction
3.3V Tolerance Enable
PEX_L*_RST_N
GPIO(rsvd1)
Input
Enable
PEX_L*_CLKREQ_N
SFIO(PE*_CLKREQ_L)
Bidirectional
Enable
-
Enable the appropriate PCIe node listed under Here are the PCIe controller DT Nodes: :
-
Add the
pipe2uphy phandleentries as aphyproperty in the PCIe controller DT node.pipe2uphy DT nodes are defined in SoC DT in the
/hardware/nvidia/soc/t19x/kernel-dts/tegra194-soc/tegra194-soc-pcie.dtsi file. Each
pipe2uphynode is a 1:1 map to the UPHY lanes that are defined in UPHY Lane Assignment ,. -
If the Tegra PCIe is operated in the endpoint mode, add the “reset-gpios” property with the gpio phandle, the gpio number connected to PERST# and flags(GPIO_ACTIVE_LOW).
-
If there are platform-specific regulators used to power up endpoints, the regulators can be registered as vpcie3v3-supply and/or vpcie12v-supply in PCIe controller device tree node. The Tegra PCIe controller driver enable the regulators before the PCIe LTSSM sequence starts.
-
For information about Jetson AGX Orin PCIe controller device tree configurations, see the documentation file at:
The
$(KERNEL_TOP)/Documentation/devicetree/bindings/pci/nvidia,tegra194-pcie.txtfile covers topics that include configuring maximum link speed, link width, advertisement of different ASPM states, and so on.
Here are the PCIe controller DT Nodes:
|
PCIe Controller and Mode |
PCIe Controller DT Node |
|---|---|
|
PCIe C0 RP |
|
|
PCIe C1 RP |
|
|
PCIe C2 RP |
|
|
PCIe C3 RP |
|
|
PCIe C4 RP |
|
|
PCIe C5 RP |
|
|
PCIe C5 EP |
Example change: PCIe C4 and C5 in Root Port Mode
file: /hardware/nvidia/platform/t19x/jakku/kernel-dts/common/tegra194-p3668-common.dtsi
pcie@14160000 {
status = "okay";
nvidia,pex-wake = <&tegra_main_gpio TEGRA194_MAIN_GPIO(L, 2)
GPIO_ACTIVE_HIGH>;
vddio-pex-ctl-supply = <&p3668_spmic_sd3>;
nvidia,disable-aspm-states = <0xf>;
nvidia,enable-power-down;
nvidia,max-speed = <3>;
max-link-speed = <3>;
num-lanes = <1>;
#if TEGRA_PCIE_VERSION >= DT_VERSION_2
phys = <&p2u_hsio_11>;
phy-names = "p2u-0";
#else
phys = <&p2u_11>;
phy-names = "pcie-p2u-0";
#endif
};
pcie@141a0000 {
status = "okay";
vddio-pex-ctl-supply = <&p3668_spmic_sd3>;
nvidia,disable-aspm-states = <0xf>;
nvidia,enable-power-down;
nvidia,max-speed = <3>;
max-link-speed = <3>;
#if TEGRA_PCIE_VERSION >= DT_VERSION_2
phys = <&p2u_nvhs_0>, <&p2u_nvhs_1>, <&p2u_nvhs_2>, <&p2u_nvhs_3>,
<&p2u_nvhs_4>, <&p2u_nvhs_5>, <&p2u_nvhs_6>, <&p2u_nvhs_7>;
phy-names = "p2u-0", "p2u-1", "p2u-2", "p2u-3",
"p2u-4", "p2u-5", "p2u-6", "p2u-7";
#else
phys = <&p2u_12>,
<&p2u_13>,
<&p2u_14>,
<&p2u_15>,
<&p2u_16>,
<&p2u_17>,
<&p2u_18>,
<&p2u_19>;
phy-names = "pcie-p2u-0", "pcie-p2u-1", "pcie-p2u-2", "pcie-p2u-3",
"pcie-p2u-4", "pcie-p2u-5", "pcie-p2u-6", "pcie-p2u-7";
#endif
};
pcie_ep@141a0000 {
status = "disabled";
nvidia,disable-aspm-states = <0xf>;
vddio-pex-ctl-supply = <&p3668_spmic_sd3>;
#if TEGRA_PCIE_VERSION >= DT_VERSION_2
reset-gpios = <&tegra_main_gpio TEGRA194_MAIN_GPIO(GG, 1) GPIO_ACTIVE_LOW>;
nvidia,refclk-select-gpios = <&tegra_aon_gpio TEGRA194_AON_GPIO(AA, 5)
GPIO_ACTIVE_HIGH>;
phys = <&p2u_nvhs_0>, <&p2u_nvhs_1>, <&p2u_nvhs_2>, <&p2u_nvhs_3>,
<&p2u_nvhs_4>, <&p2u_nvhs_5>, <&p2u_nvhs_6>, <&p2u_nvhs_7>;
phy-names = "p2u-0", "p2u-1", "p2u-2", "p2u-3",
"p2u-4", "p2u-5", "p2u-6", "p2u-7";
#else
phys = <&p2u_12>, <&p2u_13>, <&p2u_14>, <&p2u_15>,
<&p2u_16>, <&p2u_17>, <&p2u_18>, <&p2u_19>;
phy-names = "pcie-p2u-0", "pcie-p2u-1", "pcie-p2u-2", "pcie-p2u-3",
" pcie-p2u-4", "pcie-p2u-5", "pcie-p2u-6", "pcie-p2u-7";
nvidia,pex-rst-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(GG, 1)
GPIO_ACTIVE_LOW>;
#endif};
file: /hardware/nvidia/platform/t19x/jakku/bct/pinmux/tegra19x-mb1-pinmux-p3668-a01.cfg
pinmux.0x02437048 = 0x00000560; # pex_l4_clkreq_n_pl0: pe4, tristate-disable, input-enable, io_high_voltage-enable, lpdr-enable
pinmux.0x02437050 = 0x00000520; # pex_l4_rst_n_pl1: pe4, tristate-disable, input-disable, io_high_voltage-enable, lpdr-enable
Debug PCIe Link-Up Failure ¶
After you make the required device tree changes, as mentioned in Enable PCIe in a customer CVB design , if the PCIe link fails to come up, complete the following debug steps to triage the issue:
Add the
nvidia,disable-power-down
device tree property in PCIe controller node and complete the following debug steps. After you add this device tree property, lspci should list the Root Port device as follows:
$ lspci 00:01.0 PCI bridge: NVIDIA Corporation Device 10e5 (rev a1)
Triaging from the Platform Side:
Make sure that signal routing is there from Root port to Endpoint that include:
-
PERST# routing: Make sure it goes high when Root port attempts the link up.
-
CLKREQ# routing: By default ASPM is disabled, so this will not have any role in link up failure. If you enabled ASPM L1-CPM or L1SS by completing the steps in Enabling the PCIe ASPM , verify whether this pulled low when Root port attempts the link up.
-
REFCLK routing: Make sure that REFCLK flows from Root port to Endpoint. Connect the scope and observe spread enabled 100 MHz clock.
-
Tx and Rx routing: Verify Tx and Rx lanes routing are fine.
-
If any of the signals above go through board level muxes, make sure those muxes are configured correctly.
-
PCIe slot regulators or GPIOs: If PCIe slot regulators are present, make sure they are powered up. If endpoint excepts some GPIO signals to be toggled as part of power up sequence, make sure that it is happening, Example: Some Wi-Fi cards expects WDISABLE_1 signal to be asserted before PCIe link up.
Triaging from the Software Side:
-
Verify
DLActivestatus in Root portLnkStaoflspci -vvvoutput. This is to check whether the link comes up by the time kernel boots to shell (for example, to confirm whether the link is taking more time to come up). -
Dump PADCTL_PEX_CTL_PEX_L*_CLKREQ_N_0 and PADCTL_PEX_CTL_PEX_L*_RST_N_0 pinmux values and check if settings are correct.
-
Dump PCIE_RP_APPL_DEBUG_0 register, refer to TRM for register address of each controller. Accessing the controller’s address, which is not enabled, will cause a CBB power down error. When you share this information in NVIDIA developer forum, it will help us determine the LTSSM state.
-
Reduce the link speed to Gen-1 and link width to x1 using device tree properties.
Enabling the PCIe ASPM ¶
-
If the card supports L1-CPM or L1SS add “supports-clkreq” in the respective controller device tree node.
Enabling this property for an endpoint that does not support L1-CPM or L1SS will cause a PCIe link-up failure.
-
To enable desired ASPM state, clear the respective bit in the nvidia,disable-aspm-states property .
“nvidia,disable-aspm-states” value
ASPM status
0xf
All states disabled
0xe
ASPM L0s enabled, L1, L1.1 & L1.2 disabled
0xd
ASPM L1 enabled, L0s, L1.1 & L1.2 disabled
0xC
ASPM L0s & L1 enabled, L1.1 & L1.2 disabled
0x8
ASPM L0s, L1 & L1.1 enabled, L1.2 disabled
0x1
ASPM L1, L1.1 & L1.2 enabled, L0s disabled
0x0
All ASPM states enabled
-
Confirm the status of the different ASPM states from the “lspci -vvv” output and aspm_state_count in the PCIe RP controller’s debugfs.
Flashing the Build Image ¶
When you flash the build image, use your specific board name. The
flashing script uses the configuration in the
file during
the flashing process.
Setting Optional Environment Variables ¶
flash.sh
updates the following environment variables based on
board EEPROM and other parameters passed. If you want to give specific
values to these variables, define them in the board-specific file
to override the default values:
# Optional Environment Variables:
# BCTFILE ---------------- Boot control table configuration file to be used.
# BOARDID ---------------- Pass boardid to override EEPROM value
# BOARDREV --------------- Pass board_revision to override EEPROM value
# BOARDSKU --------------- Pass board_sku to override EEPROM value
# BOOTLOADER ------------- Bootloader binary to be flashed
# BOOTPARTLIMIT ---------- GPT data limit. (== Max BCT size + PPT size)
# BOOTPARTSIZE ----------- Total eMMC HW boot partition size.
# CFGFILE ---------------- Partition table configuration file to be used.
# CMDLINE ---------------- Target cmdline. See help for more information.
# DEVSECTSIZE ------------ Device Sector size. (default = 512Byte).
# DTBFILE ---------------- Device Tree file to be used.
# EMMCSIZE --------------- Size of target device eMMC (boot0+boot1+user).
# FLASHAPP --------------- Flash application running in host machine.
# FLASHER ---------------- Flash server running in target machine.
# INITRD ----------------- Initrd image file to be flashed.
# KERNEL_IMAGE ----------- Linux kernel zImage file to be flashed.
# MTS -------------------- MTS file name such as mts_si.
# MTSPREBOOT ------------- MTS preboot file name such as mts_preboot_si.
# NFSARGS ---------------- Static Network assignments; client, server,
# & gateway IP addresses & netmask in format:
# :::
# NFSROOT ---------------- NFSROOT, an IP address, colon, and path,
# e.g. 127.0.0.1:/exported/rootfs_dir.
# ODMDATA ---------------- Odmdata to be used.
# PKCKEY ----------------- RSA key file to use to sign bootloader images.
# ROOTFSSIZE ------------- Linux RootFS size (internal emmc/nand only).
# ROOTFS_DIR ------------- Linux RootFS directory name.
# SBKKEY ----------------- SBK key file to use to encrypt bootloader images.
# SCEFILE ---------------- SCE firmware file such as camera-rtcpu-sce.img.
# SPEFILE ---------------- SPE firmware file path such as bootloader/spe.bin.
# FAB -------------------- Target board's FAB ID.
# TEGRABOOT -------------- lowerlayer bootloader such as nvtboot.bin.
# WB0BOOT ---------------- Warmboot code such as nvtbootwb0.bin
Note
All of the parameters must be added below the reference to the file
p3668.conf.common
to be reflected in the flashed image.
Here is an example of environment variable settings for the Jetson Xavier NX Developer Kit
.conf
file,
p3509-0000+p3668-0000-qspi-sd.conf
:
source "${LDK_DIR}/p3668.conf.common";
BLBlockSize=1048576;
EMMC_CFG=flash_l4t_t194_spi_sd_p3668.xml;
RECROOTFSSIZE=100MiB
To flash the build image ¶
-
Enter the command:
$ sudo ./flash.sh
mmcblk0p1