.. _HR.JetsonModuleAdaptationAndBringUp.JetsonXavierNxSeries: .. include:: /content/swdocs.rsts .. spelling:: dts extcon fansink Fi Nano padctl proc Realtek subnode Wi xHCI Xorg xUDC xusb Designware usb gpio VBus HSIO NVHS UPH PEX RST GPIO PEX_L CLKREQ SFIO PE* CLKREQ vpcie12v ASPM DT CVB REFCLK PADCTL LTSSM CBB L1 CPM L1SS aspm debugfs RP vvv 0xf 0xe 0xd 0xC 0x8 0x1 0x0 PCIE RP APPL DLActive lspci WDISABLE Tx Rx Devkit clkreq uphy GPIO tegra pcie gpios phandle vpcie muxes xf xe xC xd Nx Jetson Module Adaptation and Bring-Up: Jetson Xavier NX Series !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! This topic is for engineers who are developing production software for an NVIDIA Jetson Xavier NX series module. It describes how to port NVIDIA Jetson Linux and Bootloader from a Jetson Xavier NX Developer Kit to a custom hardware platform and bring up a working system on the new platform. .. todo:: This paragraph originally referred to U-Boot. I don't know what wording you'll consider appropriate now that no supported platforms use U-Boot. The examples described include code for the Jetson Xavier NX Developer Kit (P3518). For information on customizing the configuration files, see the section :ref:`MB1 Platform Configuration `. Two other topics are closely related to this one: .. todo:: Need to add back bring-up framework when it supports K5.10 - :ref:`Jetson Module Adaptation and Bring-Up\: Checklists ` contains lists of steps to perform to complete the parts of the adaptation task. Board Configuration @@@@@@@@@@@@@@@@@@@ Jetson Xavier NX series has the following variants: - Jetson Xavier NX (development, P3668-0000), with 8\ |GB| of RAM and an on-module microSD card socket. Is intended only for software development, and is sold only as a component of Jetson Xavier NX Developer Kit. You can program it by flashing a microSD card in place, or by flashing it on a host computer and then installing it on the module. - Jetson Xavier NX (production, P3668-0001), with 8\ |GB| of RAM but no microSD card socket. This module is sold as a separate unit. It is intended for production use but may also be used in a Jetson Xavier NX Developer Kit. You program it in the conventional way, by flashing software to on-board eMMC memory. - Jetson Xavier NX 16GB (P3668-0003), with 16\ |GB| of RAM. This variant is identical to P3668-0001 in all respects except RAM size. Jetson Xavier NX Developer Kit, P3518, includes a P3509-0000 reference carrier board as well as the P3668-0000 SOM. The SOM and carrier board each have an EEPROM where the board ID is saved. The SOM can be used without any software configuration modifications. Jetson Xavier NX series modules have a provision for attaching a heatsink or fansink. The module provided with a Jetson Xavier NX Developer Kit has a fansink already attached. See the `Jetson Xavier NX Thermal Design Guide `__ for guidance on designing or procuring a thermal solution for production Jetson Xavier NX series modules. Before using the SOM with a carrier board other than P3509, you must change the kernel device tree, MB1 configuration, ODM data, and flashing configuration to accommodate the new carrier board instead of P3509. An EEPROM ID for your custom board is not required. You can also use the reference carrier board P3449-0000 (included in the Jetson Nano Developer Kit) with a Jetson Xavier NX series module. The Jetson Linux installer detects the type of module and carrier board and automatically installs the appropriate drivers and other device-specific components. .. todo:: Is the P3668-0000+P3449-0000 combination still supported in r34? Board Naming @@@@@@@@@@@@ To support a Jetson Xavier NX series module with your custom carrier board, you must assign the module/carrier board combination a **board name**. This name must consist of lower case letters, numerals, hyphens, and underscores. You must set the board name in the environment variable ``BOARD`` on your host system. Some examples of valid board names are ``p3509-0000+p3668-0000``, ``p3518-0000-devkit``, and ``devboard``. The installation process uses the board name in filenames and pathnames, including names in the device tree and proc file system. The placeholder ```` is used to represent the board name throughout this topic. Where a command contains ````, substitute the name of your custom board. .. note:: Do not modify files whose filenames and pathnames contain your system’s default board name. The default board name refers to the carrier board provided with your Jetson developer kit. If it appears in a file’s name, that file contains code or data that is specific to that board. Make copies of such files that are named with your custom board’s board name, and modify them to support your custom board. Root File System Configuration @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Jetson Linux can use any standard or customized Linux root file system (rootfs) that is appropriate for its targeted embedded applications. .. todo:: The original said "for their targeted embedded applications," which made no sense, since there's no plural subject for "their" to refer to. It still doesn't make a lot of sense. Is it intended to mean something more than "for embedded applications," which would be perfectly clear? However, certain settings must be configured in the rootfs’s boot-up framework to set default configuration after boot, or some of the core functionalities will not run as expected. For example: 1. The ``nv.sh`` and ``nvfb.sh`` boot-up scripts do some platform-specific configuration in the kernel. 2. The Xorg and X libraries must be correctly configured for the target device. 3. The nvpmodel clock and frequency must be configured for the target device. Jetson Linux provides these rootfs configurations and customizations in this directory and its subdirectories:: Linux_for_Tegra/nv_tegra/ You must incorporate the relevant customization for your target rootfs from this location. .. note:: For the sample Ubuntu root file system provided by NVIDIA, this customization is applied using the script ``Linux_for_Tegra/apply_binaries.sh``. MB1 Configuration Changes @@@@@@@@@@@@@@@@@@@@@@@@@ Multiple ``.cfg`` files define boot time configuration of the hardware. They are applied by Bootloader. The MB1 boot configuration tables are available at:: /bootloader/t186ref/BCT Where ```` is the top-level BSP flashing directory. .. todo:: As I've noted elsewhere, we variously designate the top-level "BSP flashing directory" as ````, ````, ``Linux_for_Tegra``, and probably other terms. We should adopt a single term and use it consistently. If the location is actually fixed at ``/Linux_for_Tegra``, I see no reason to call it anything else. We should also adopt a consistent English description of this directory. "BSP flashing directory" is inappropriate, since the directory doesn't have anything to do with flashing, except in the sense that the whole BSP does. I suggest "BSP home directory." Pinmux Changes ############## If your carrier board schematic differs from that of the reference carrier board in Jetson Xavier NX Developer Kit, you must change the pinmux configuration applied by the software. To define your board’s pinmux configuration, download the `Jetson Xavier NX pinmux table `__ from the Jetson Download Center. Be sure to get the right version of the table for your SOM. The pinmux table is a spreadsheet that: - Shows the locations and default pinmux settings - Acts as a data source for software that defines the pinmux settings in the source code or device tree You must customize the spreadsheet for the configuration of your board, then convert the ``.dtsi`` file generated by Excel to a ``.cfg`` file. For instructions, see the README file at:: Linux_for_Tegra/kernel/pinmux/t19x/ You must perform the same conversion for ``gpio.dtsi`` and ``padvoltage.dtsi``. GPIO Changes ############ If you designed your own carrier board, to translate from SOM connector pins to actual GPIO numbers you must understand the GPIO mapping formula below. The translated GPIO numbers can be controlled by the driver. To check a GPIO number $$$$$$$$$$$$$$$$$$$$$$ This example procedure checks the GPIO number of signal name ``SPI0_CS1``. #. Open the Jetson Xavier NX pinmux table. (See `Pinmux Changes <#pinmux-changes>`__, above.) #. Search the table for ``SPI0_CS1``. #. Confirm that the Customer Usage field contains the GPIO name ``GPIO3_PZ.07``. #. Search PZ.07 using the following command:: cat /sys/kernel/debug/gpio | grep PZ.07 5. For example Output:: gpio-xxx (PZ.07 ) 6. GPIO number of ``SPI0_CS1`` is xxx. Note: To use a pin as GPIO, make sure that E_IO_HV field is disabled in corresponding pinmux register of the GPIO pin. You can **disable** the field **3.3V Tolerance Enable** in pinmux spreadsheet and reflash the board with the updated pinmux file. PMIC Changes ############ The PMIC configuration file configures the initial PMIC in the P3668 SOM. It also defines some GPIO expander-based GPIO regulator settings in the P3509 carrier board configurations. Review this configuration file to replace any references to the P3509 carrier board with references to your custom board. If required, include any regulator information to enable this file. For example, the following settings configure PMIC to turn on SD4 on I2C address ``0x3c``. The example is from the configuration file ``tegra194-mb1-bct-pmic-p3668-0001-a00.cfg``:: ######################## System Configurations #### # PMIC FPS to turn SD4 (VDD_DDR_1V1) on in time slot 0 # PMIC FPS to set GPIO2 (EN_DDR_VDDQ) high in time slot 1 # Set SLPEN = 1 and CLRSE on POR reset pmic.system.block[0].type = -1; # I2C pmic.system.block[0].controller-id = 4; pmic.system.block[0].slave-add = 0x78; # 7BIt:0x3c pmic.system.block[0].reg-data-size = 8; pmic.system.block[0].reg-add-size = 8; pmic.system.block[0].block-delay = 10; pmic.system.block[0].commands[0].0x53.0x38 = 0x00; #_SD4 FPS UP slot 0 pmic.system.block[0].commands[1].0x55.0x38 = 0x10; #_GPIO2 FPS UP slot 2 pmic.system.block[0].commands[2].0x41.0x1C = 0x1C; #_SLPEN=1, CLRSE = 11 Porting the Linux Kernel @@@@@@@@@@@@@@@@@@@@@@@@ It is assumed that you are using a P3668 SOM connected to an unmodified P3509 carrier board; the QSPI, PMIC, DDR, and routing of lines are all unchanged. The modifications you are making are for the SOM and the carrier board. Consequently, based on the peripherals present on your carrier board, you can modify the ``.dts`` files by disabling/enabling the controllers and changing the supplies. To port the kernel configuration code (the device tree) to your platform, modify one of the distributed configuration files to describe the design of your platform. The configuration files are available at:: /hardware/nvidia/platform/t19x/ /hardware/nvidia/soc/t19x The final DTB file used is:: tegra194-p3668-0000-p3509-0000.dtb By reading the above file, you see which other ``.dtsi`` files are referenced by include statements. The following table describes common ``.dtsi`` files that may be modified to reflect hardware design changes. +----------------------------------+------------------------------------------------+ | Types of Changes | DTSI Filename or location | +==================================+================================================+ | Power supply changes | ``tegra194-power-tree-p3668.dtsi`` | +----------------------------------+------------------------------------------------+ | Regulator parameter changes | ``tegra194-spmic-p3668.dtsi`` | +----------------------------------+------------------------------------------------+ | Display panel and node changes | For details, see the topic | | | :ref:`Display Configuration and Bringup | | | `. | +----------------------------------+------------------------------------------------+ | ODM data based feature | ``tegra194-plugin-manager-p3668.dtsi`` | | configuration | | +----------------------------------+------------------------------------------------+ | NVIDIA SoC controller state to | ``soc/t19x/kernel-dts/tegra194-soc/`` | | enable/disable a controller | | +----------------------------------+------------------------------------------------+ | Panels related .dts files | ``platform/tegra/common/kernel-dts/panels/`` | +----------------------------------+------------------------------------------------+ Verify that no other ``.dts`` or ``.dtsi`` file, including these ``.dts`` files, overrides any changes you make. .. todo:: What does "these .dts files" refer to? The obvious meaning is the files in the preceding table, but those are all .dtsi files. As a best practice, create your own set of ``.dts`` files based on the |NVIDIA(r)| |Jetson AGX Xavier(tm)| files already present. Rename your newly created files to the name of your board. .. note:: Use ``fdtdump`` or ``dtc`` to generate a ``.dts`` file from the final ``.dtb`` file and check whether your changes have taken effect. The command usage is:: $ dtc -I dtb -O dts tegra194-p3668-all-p3509-0000.dtb > tegra194-p3668-all-p3509-0000.dts $ fdtdump dts tegra194-p3668-all-p3509-0000.dtb >tegra194-p3668-all-p3509-0000.dts PCIe Controller Configuration @@@@@@@@@@@@@@@@@@@@@@@@@@@@@ This section provides information about PCIe controller configurations. The PCIe host controller is based on Synopsis Designware PCIe intellectual property, and thus inherits all the common properties defined in the information file at:: $(KERNEL_TOP)/nvidia/Documentation/devicetree/bindings/pci/nvidia,tegra19x-pcie.txt PCIe Controller Features ######################## Xavier SoC has six PCIe controllers that are supported by the HSIO and NVHS UPHY bricks. All of these controllers cannot be enabled at the same time, refer to :ref:`UPHY Lane Assignment ` for a list of the supported PCIe configurations. Here is a list of the PCIe controllers that are supported by Jetson Xavier NX series and their specifications: .. list-table:: :widths: auto :header-rows: 1 * - PCIe Controller - Controller mode - Supported link width - Supported link speed * - PCIe C0 - Root port - x4 - Gen4 * - PCIe C1 - Root port - x1 - Gen4 * - PCIe C3 - Root port - x1 - Gen4 * - PCIe C5 - Dual mode - up to x8 - Gen4 - **Dual Mode PCIe Controllers**: The C5 Controller supports dual mode, and it can be configured as a Root Port or an Endpoint. Mode selection is fixed during flashing and dynamic switching of the mode during runtime is not possible. - **ASPM**: All controllers support ASPM L0s, L1, L1.1, and L1.2. Porting Universal Serial Bus @@@@@@@@@@@@@@@@@@@@@@@@@@@@ .. todo:: Note dated 6/17/2020: Brian wrote, “According to marketing, there is no flexibility for customers doing their own designs as far as USB 3.0 & PCIe mapping. A single map exists with one USB 3.0 interface on Tegra X1 PEX6 and a 4-lane PCIe interface on Tegra X1 PEX[4:1].” I gather that substantial parts of this section must be deleted because of that. Please advise me which parts. The Jetson Xavier NX series modules can support one enhanced SuperSpeed Universal Serial Bus (USB) port and up to three High Speed USB ports. The Jetson Xavier NX P3509 carrier board is designed and verified for one USB3.1 port. If you design your own carrier board, verify the compatibility between P3509 and your own board by consulting the NVIDIA team. USB Structure ############# An enhanced SuperSpeed USB port has nine pins: - VBUS. - GND. - D+. - D−. - Two differential signal pairs for SuperSpeed data transfer. - One ground (``GND_DRAIN``) for drain wire termination and managing EMI, RFI, and signal integrity. .. figure:: JetsonXavierNxSeries/UsbSuperspeedPortPinout.svg :alt: USB SuperSpeed port pinout The ``D+``/\ ``D−`` signal pins connect to UTMI pads. The ``SSTX``/\ ``SSRX`` signal pins connect to UPHY and are handled by a single UPHY lane. As UPHY lanes are shared between PCIE, SATA, UFS, and XUSB, UPHY lanes must be assigned according to the custom carrier board’s requirements. Required Device Tree Changes ############################ This section gives step-by-step guidance for checking schematics and configuring USB ports in the device tree. All of the examples are based on the design of the Jetson Xavier NX P3509 carrier board. For a Host-Only Port $$$$$$$$$$$$$$$$$$$$ This section uses U7, a USB 3.1 Gen2 Realtek Enhanced SuperSpeed on-board hub, model number RTS5489-GR (hereafter “\ **Realtek hub**\ ”) as an example of a host-only port. Go Through the Schematics %%%%%%%%%%%%%%%%%%%%%%%%% .. note:: The P3509 carrier board’s schematic file is included in the `Jetson Xavier NX Developer Kit ​Carrier Board ​Design ​Files `__, available from the Jetson Download Center. Check the Realtek hub on the P3509 carrier board and find the socket location wired to the P3668 SOM. - USB2 signal pins ``D+/D-`` (``USP_D_P/USP_D_N``) wire out from U7 and lead to module socket pins 117 (``USB1_D``) and 115 (``USB1_D``). .. figure:: JetsonXavierNxSeries/Usb2SignalPinsFromU7.svg :alt: USB2 signal pins from U7 - USB3 differential pairs (``USP_TX_*`` and ``USP_RX_*``) wire out from U7 and lead to module socket pins 161 (SBSS_RX) and 168 (USBSS_TX). .. figure:: JetsonXavierNxSeries/Usb3SignalPinsFromU7.svg :alt: USB3 signal pins from U7 Through the schematic, you can conclude that for U7: - The USB2 signal pair is wired to UTMI pad 1 (USB2 port 1). - The USB3 signal pairs are wired to UPHY lane 1 (USB 3.1 port 2). The xusb_padctl Node %%%%%%%%%%%%%%%%%%%% The device tree’s ``xusb_padctl`` node follows the conventions of ``pinctrl-bindings.txt``. It contains two groups, named ``pads`` and ``ports``, which describe USB2 and USB3 signals along with parameters and port numbers. The name of each parameter description subnode in ``pads`` and ``ports`` must be in the form ``-``, where ```` is ``"usb2"`` or ``"usb3"``, and ```` is the associated port number. The properties of the ``pads`` subnode are: - ``nvidia,function``: A string containing the name of the function to mux to the pin or group. Must be ``"xusb"``. The properties of the ``ports`` subnode are: - ``mode``: A string that describes USB port capability. A port for USB2 must have this property. It must be one of these values: - ``host`` - ``peripheral`` - ``otg`` - ``nvidia,usb2-companion``: USB2 port (0, 1, or 2) to which the port is mapped. A port for USB3 must have this property. - ``nvidia,oc-pin``: The overcurrent ``VBUS`` pin the port is using. The value must be positive or zero. .. note:: Overcurrent detection and handling for U7, a Realtek hub on the P3509 carrier board, are controlled by the hub itself. Therefore, you need not set this property. - ``vbus-supply``: ``VBUS`` regulator for the corresponding UTMI pad. Set to ``&battery_reg`` for a dummy regulator. .. note:: The Realtek hub is always connected to the root hub port on a P3509, so you need not control hub power. You just need enable it with ``VDD_3V3_SYS``. Therefore, you must set dummy regulators for U7 on the P3509 carrier board. For the detailed information about ``xusb_padctl``, see the documentation at:: kernel/kernel-5.10/Documentation/devicetree/bindings/phy/nvidia,xusb-padctl.txt As an example, consider U7, the Realtek hub, which is always connected to USB2 port 1 and USB3 port 2 on the root hub. Create a pad/port node and property list for U7 based on the device tree structure described above:: xusb_padctl: xusb_padctl@3520000 { ... pads { usb2 { lanes { usb2-1 { nvidia,function = "xusb"; status = "okay"; }; ... }; }; usb3 { lanes { usb3-2 { nvidia,function = "xusb"; status = "okay"; }; }; }; }; ports { ... usb2-1 { mode = "host"; vbus-supply = <&battery_reg>; status = "okay"; }; ... usb3-0 { nvidia,usb2-companion = <1>; status = "okay"; }; }; }; .. _HR.JetsonModuleAdapationAndBringUp.JetsonXavierNx-PortingUsbUniversalSerialBus.RequiredDeviceTreeChanges.ForAHostOnlyPort.UnderTheXhciNode: Under the xHCI Node %%%%%%%%%%%%%%%%%%% The Jetson Xavier NX xHCI controller complies with xHCI specifications, which support both USB 2.0 High Speed / Full Speed / Low Speed and USB 3.1 SuperSpeed protocols. The properties of the node are: - ``phys``: Must contain an entry for each entry in ``phy-names``. - ``phy-names``: Must include an entry for each PHY used by the controller. Names must be in the form ``-``, where ```` is ``"usb2"`` or ``"usb3"``, and ```` is the associated port number. - ``nvidia,xusb-padctl``: A pointer to the ``xusb-padctl`` node. For detailed information about xHCI, see the documentation at:: kernel/kernel-5.10/Documentation/devicetree/bindings/usb/nvidia,tegra-xhci.txt Consider U7, the Realtek hub, as an example. Create an xHCI node and property list for U7 based on the device tree structure described above:: tegra_xhci: xhci@3610000 { ... phys = <&{/xusb_padctl@3520000/pads/usb2/lanes/usb2-1}>, <&{/xusb_padctl@3520000/pads/usb3/lanes/usb3-2}>; phy-names = "usb2-1", "usb3-2"; nvidia,xusb-padctl = <&xusb_padctl>; status = "okay"; ... }; For an On-The-Go Port $$$$$$$$$$$$$$$$$$$$$$$$$$$ .. todo:: 6/17/2020: Brian wrote: “I was under the impression we weren't supposed to claim OTG. We often use that just to mean a port can be selected to be a host or device under SW control. The USB meaning is somewhat different and we have been told not to use this. Maybe something changed?” Later, “I have heard “Dual-role (Host/Device) or similar” used in place of OTG, but would be good to hear from marketing since they have directed those of us working on the HW docs to remove mention of OTG.” Should we avoid "On-The-Go" here? If so, what should we say? USB On-The-Go, which is often abbreviated **USB OTG** or just **OTG**, is a specification that allows USB to act as a host or a device in the same port. A USB OTG port can switch back and forth between the roles of host and device. This section takes J5, USB2.0 Micro B connector, as an example of an OTG port. An OTG port adds a fifth pin to the standard USB connector, called the **ID pin**. An OTG cable has an A-plug on one end and a B-plug on the other end. The A-plug’s ID pin is grounded, while the B-plug’s ID pin is floating. A device with an A-plug inserted becomes and OTG A-device (a host), and a device with a B-plug inserted becomes a B-device (a device). .. figure:: JetsonXavierNxSeries/OtgPortPinout.svg :alt: OTG port pinout .. note:: Because its ID pin is floating, J5 is fixed in the device role in the Jetson Xavier NX Developer Kit. It cannot function as a host, for example, to connect a flash drive, a keyboard, or a mouse. Go Through the Schematics %%%%%%%%%%%%%%%%%%%%%%%%% .. note:: The P3509 carrier board’s schematic file is included in `Jetson Xavier NX Developer Kit Carrier Board Design Files `__, available from the Jetson Download Center. .. todo:: I could not find this document in the J.D.C. I found three other matches; perhaps one of them is the right thing, or contains the right thing, but if so the reader has no way of knowing which one. (The same issue occurs with a later reference to the same document.) Check the USB connectors on the P3509 carrier board and find the socket location wired to the P3668 SOM. - USB2 signal pins D+/D− (``DP`` and ``DN``) wire out from J5 and lead to pin 111 (``USB0_D``) and pin 109 (``USB0_D``) on the SOM socket. .. figure:: JetsonXavierNxSeries/Usb2SignalPinsFromJ5.svg :alt: USB2 signal pins from J5 The USB 2.0 Micro B connector, J5, supports only HighSpeed mode, and does not have USB3 signal pairs. From the schematic, you can see that for J5: - The USB2 signal pair is wired to UTMI pad 0 (USB2 port 0). The USB Connector Class %%%%%%%%%%%%%%%%%%%%%%% A USB connector class represents a physical USB connector. It should be a child of a USB interface controller or a separate node when it is attached to the MUX and USB interface controllers. Generally, port switching between the roles of an OTG port is controlled by the host driver (xHCI) and device driver (xUDC), and can be defined by the state of the ID pin and the ``VBUS_DETECT`` pin. Taking ``GPIO_M3`` as the ``VBUS_DETECT`` pin and ``GPIO_Q0`` as the ID pin, for example: #. Find the corresponding GPIO states on the ``VBUS_DETECT`` pin and the ID pin. Generally, the ID pin is designed as internal pull high (logical high). With an A-plug connected the ID pin is pulled to ground (logical low), while with a B-plug connected or no cable connected it remains logical high. The operation of the ``VBUS_DETECT`` pin depends on the device’s design. Consider the schematic in the following diagram, for example: .. figure:: JetsonXavierNxSeries/OtgRoleSwitching.svg :alt: OTG Role Switching With a B-plug connected, ``VBUS_DETECT`` is logical low, because VBUS is provided from an external power supply. When no cable is connected, it is logical high. .. note:: ``VBUS_DETECT`` is initially logical high, then logical low because ``VBUS`` is provided by the host controller. Therefore, the state of the ``VBUS_DETECT`` pin does not matter when the OTG port is operating in host mode. #. Create the table of GPIO states and their corresponding output cable states: ============ ===================== ========================= GPIO_Q0 (ID) GPIO_M3 (VBUS_DETECT) Data Role ============ ===================== ========================= 1 1 Not Connected 0 0 HOST 0 1 HOST 1 0 DEVICE ============ ===================== ========================= Under the Connector Node %%%%%%%%%%%%%%%%%%%%%%%% Port switching between the roles of an OTG port is defined by the state of the ID pin and the ``VBUS_DETECT`` pin and the settings of the USB connector class. The properties of the ``extcon`` node are: - ``compatible``: Value must be ``"Value must be "gpio-usb-b-connector"``. - ``Label``:Symbolic name for the connector. - ``type``: Size of the connector, should be specified in case of non-full size 'usb-a-connector' or 'usb-b-connector' compatible connectors. - ``id-gpios``:An input gpio for USB ID pin. - ``vbus-gpios``: An input gpio for the USB VBus pin, used to detect the presence of VBUS 5V. - ``cable-connected-on-boot``: Name of the output cable connected on boot. It should be ``USB_ROLE_NONE``, ``USB_ROLE_HOST``, and ``USB_ROLE_DEVICE``. If not specified, the system assumes that no cable is to be connected. - ``wakeup-source``: A Boolean; true if the device can wake up the system. For detailed information about ``USBConnector``, see the documentation at:: ``kernel/kernel-5.10/Documentation/devicetree/bindings/connector/usb-connector.yaml`` .. todo:: It's not clear what the following list item has to do with the rest of the list. It's a process step; the other entries are node properties. In the original document the preceding paragraph was unindented, making the following a separate one-entry list, but that didn't make sense either. - Create a ``USBConnector`` device node and property list based on the device tree structure described above and the table of GPIO states and corresponding output cable states for ``GPIO_Q0`` and ``GPIO_M3``:: vxusb_padctl: xusb_padctl@3520000 { ... ports { usb2-0 { ... Connector { compatible = "gpio-usb-b-connector"; label = "micro-USB"; type = "micro"; vbus-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(M, 3) GPIO_ACTIVE_LOW>; id-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(Q, 0) GPIO_ACTIVE_HIGH>; }; ... }; }; ... The USB 2.0 Micro B connector, J5, has the connector’s ID pin floating and the ``VBUS_DETECT`` pin of the connector wired out to ``GPIO00``, which corresponds to ``GPIO_PZ1``. As a result, J5 can only function in the device role. .. figure:: JetsonXavierNxSeries/Usb20MicroBConnectorJ5.svg :alt: USB 2.0 Micro B connector J5 has the connector's ID pin floating This table describes the GPIO states on J5 and the corresponding output: cable states: ====================== ===================== GPIO_PZ1 (VBUS_DETECT) Data Role ====================== ===================== 1 Not Connected 0 Device ====================== ===================== .. todo:: The original referred to the table in the Adaptation and Bring-Up topic for Jetson Nano, and said "states for GPIO_PZ1, customized for Jetson Xavier NX..." Presumably customizing the table of states for Xavier NX was left as an exercise for the reader. Now that Nano is unsupported the table must be moved to this topic and customized appropriately. This is the ``USBConnectorClass`` device node and property list based on the device tree structure and the table of GPIO states and corresponding output cable states for GPIO_PZI that is customized for Jetson Xavier NX, where the ID pin that is floating the port is fixed in the device role:: xusb_padctl: xusb_padctl@3520000 { ... ports { usb2-0 { ... Connector { compatible = "gpio-usb-b-connector"; label = "micro-USB"; type = "micro"; vbus-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(Z, 1) GPIO_ACTIVE_LOW>; }; ... }; }; ... }; .. todo:: The original added, "...where the ID pin is floating the port is fixed in the device role." I'm not sure what this means, so it's not clear how to render "the ID pin is floating the port is fixed" into grammatical English. .. note:: Check the pinmux spreadsheet for the GPIO that corresponds to the ID pin and ``VBUS_DETECT`` pin. Check the pinmux spreadsheet for the GPIO that corresponds to the ID pin and ``VBUS_DETECT`` pin. Under the xusb_padctl Node %%%%%%%%%%%%%%%%%%%%%%%%%% ``xusb_padctl`` settings for an OTG port are the same as for a host-only port except that the mode must be ``"otg"``. Taking J5, the USB 2.0 Micro B connector, as an example, create a pad/port node and property list:: xusb_padctl: xusb_padctl@3520000 { ... pads { usb2 { lanes { usb2-0 { nvidia,function = "xusb"; status = "okay"; }; ... }; }; ... }; ports { usb2-0 { mode = "otg"; vbus-supply = <&battery_reg>; usb-role-switch; status = "okay"; Connector { compatible = "gpio-usb-b-connector"; label = "micro-USB"; type = "micro"; vbus-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(Z, 1) GPIO_ACTIVE_LOW>; }; }; ... }; }; Under the xHCI Node %%%%%%%%%%%%%%%%%%% The xHCI settings for an OTG port are the same as for a host-only port: Taking the USB 2.0 Micro B connector, J51, as an example, create an xHCI node and property list based on the device tree structure described in :ref:`Under the xHCI Node ` for a host-only port:: tegra_xhci: xhci@3610000 { . . . phys = <&{/xusb_padctl@3520000/pads/usb2/lanes/usb2-0}>; phy-names = "usb2-0"; nvidia,xusb-padctl = <&xusb_padctl>; status = "okay"; . . . }; Under the xUDC Node %%%%%%%%%%%%%%%%%%% The Jetson Xavier NX xUDC controller supports both USB 2.0 High Speed\ |nbsp|\ / Full Speed and USB 3.1 SuperSpeed protocols. - ``charger-detector``: USB charger detection support. Must be the ``phandle`` of the USB charger detection driver DT node. - ``phys``: An array; must contain pointers to the nodes that define each PHY in ``phy-names``. - ``phy-names``: An array; must contain an entry for each PHY used by the controller. Names must be in the form ``-``, where ```` is one of ``"usb2"`` or ``"usb3"``, and ```` is the associated port number. - ``nvidia,xusb-padctl``: A pointer to the ``xusb-padctl`` node. For detailed information about xUDC, see the documentation at:: kernel/kernel-5.10/Documentation/devicetree/bindings/usb/nvidia,tegra-xudc.txt Taking J5, the USB 2.0 Micro B connector, as an example, create an xUDC node and property list for J5 based on the device tree structure described above:: tegra_xudc: xudc@3550000 { phys = <&{/xusb_padctl@3520000/pads/usb2/lanes/usb2-0}>; phy-names = "usb2-0"; nvidia,xusb-padctl = <&xusb_padctl>; status = "okay"; }; The Jetson Nx Xavier Devkit Default PCIe Configuration ###################################################### .. list-table:: :widths: auto :header-rows: 1 * - PCIe Controller - Controller mode - Supported link width - Supported link speed - Slot * - PCIe C4 - Root port - X1 - Gen3 - M.2 Key E * - PCIe C5 - Dual mode - x4 - Gen4 - M.2 Key M Enable PCIe in a Customer CVB Design #################################### #. Select the appropriate UPHY configuration from :ref:`UPHY Lane Assignment `, which suits the CVB design and update ODMDATA accordingly. #. Refer to :ref:`Setting Optional Environmental Variablest ` for more information about how to update ODMDATA. #. Configure the PCIe reset (PEX_L*_RST_N) and clkreq(PEX_L*_CLKREQ_N) as follows. To update the pinmux value, refer to :ref:`Changing the Pinmux `. **Pinmux Settings for the Root Port Mode** .. list-table:: :widths: auto :header-rows: 1 * - Pinmux - Customer Usage - Pin Direction - 3.3V Tolerance Enable * - PEX_L*_RST_N - SFIO(PE*_RST_L) - Output - Enable * - PEX_L*_CLKREQ_N - SFIO(PE*_CLKREQ_L) - Bidirectional - Enable **Pinmux Settings for Endpoint Mode** .. list-table:: :widths: auto :header-rows: 1 * - Pinmux - Customer Usage - Pin Direction - 3.3V Tolerance Enable * - PEX_L*_RST_N - GPIO(rsvd1) - Input - Enable * - PEX_L*_CLKREQ_N - SFIO(PE*_CLKREQ_L) - Bidirectional - Enable #. Enable the appropriate PCIe node listed under *Here are the PCIe controller DT Nodes:*: #. Add the ``pipe2uphy phandle`` entries as a ``phy`` property in the PCIe controller DT node. pipe2uphy DT nodes are defined in SoC DT in the /hardware/nvidia/soc/t19x/kernel-dts/tegra194-soc/tegra194-soc-pcie.dtsi file. Each ``pipe2uphy`` node is a 1:1 map to the UPHY lanes that are defined in :ref:`UPHY Lane Assignment `,. #. If the Tegra PCIe is operated in the endpoint mode, add the "reset-gpios" property with the gpio phandle, the gpio number connected to PERST# and flags(GPIO_ACTIVE_LOW). #. If there are platform-specific regulators used to power up endpoints, the regulators can be registered as vpcie3v3-supply and/or vpcie12v-supply in PCIe controller device tree node. The Tegra PCIe controller driver enable the regulators before the PCIe LTSSM sequence starts. #. For information about Jetson AGX Orin PCIe controller device tree configurations, see the documentation file at: The ``$(KERNEL_TOP)/Documentation/devicetree/bindings/pci/nvidia,tegra194-pcie.txt`` file covers topics that include configuring maximum link speed, link width, advertisement of different ASPM states, and so on. **Here are the PCIe controller DT Nodes:** .. list-table:: :widths: auto :header-rows: 1 * - PCIe Controller and Mode - PCIe Controller DT Node * - PCIe C0 RP - pcie@14180000 * - PCIe C1 RP - pcie@14100000 * - PCIe C2 RP - pcie@14120000 * - PCIe C3 RP - pcie@14140000 * - PCIe C4 RP - pcie@14160000 * - PCIe C5 RP - pcie@141a0000 * - PCIe C5 EP - pcie_ep@141a0000 **Example change: PCIe C4 and C5 in Root Port Mode** .. code-block:: none file: /hardware/nvidia/platform/t19x/jakku/kernel-dts/common/tegra194-p3668-common.dtsi pcie@14160000 { status = "okay"; nvidia,pex-wake = <&tegra_main_gpio TEGRA194_MAIN_GPIO(L, 2) GPIO_ACTIVE_HIGH>; vddio-pex-ctl-supply = <&p3668_spmic_sd3>; nvidia,disable-aspm-states = <0xf>; nvidia,enable-power-down; nvidia,max-speed = <3>; max-link-speed = <3>; num-lanes = <1>; #if TEGRA_PCIE_VERSION >= DT_VERSION_2 phys = <&p2u_hsio_11>; phy-names = "p2u-0"; #else phys = <&p2u_11>; phy-names = "pcie-p2u-0"; #endif }; pcie@141a0000 { status = "okay"; vddio-pex-ctl-supply = <&p3668_spmic_sd3>; nvidia,disable-aspm-states = <0xf>; nvidia,enable-power-down; nvidia,max-speed = <3>; max-link-speed = <3>; #if TEGRA_PCIE_VERSION >= DT_VERSION_2 phys = <&p2u_nvhs_0>, <&p2u_nvhs_1>, <&p2u_nvhs_2>, <&p2u_nvhs_3>, <&p2u_nvhs_4>, <&p2u_nvhs_5>, <&p2u_nvhs_6>, <&p2u_nvhs_7>; phy-names = "p2u-0", "p2u-1", "p2u-2", "p2u-3", "p2u-4", "p2u-5", "p2u-6", "p2u-7"; #else phys = <&p2u_12>, <&p2u_13>, <&p2u_14>, <&p2u_15>, <&p2u_16>, <&p2u_17>, <&p2u_18>, <&p2u_19>; phy-names = "pcie-p2u-0", "pcie-p2u-1", "pcie-p2u-2", "pcie-p2u-3", "pcie-p2u-4", "pcie-p2u-5", "pcie-p2u-6", "pcie-p2u-7"; #endif }; pcie_ep@141a0000 { status = "disabled"; nvidia,disable-aspm-states = <0xf>; vddio-pex-ctl-supply = <&p3668_spmic_sd3>; #if TEGRA_PCIE_VERSION >= DT_VERSION_2 reset-gpios = <&tegra_main_gpio TEGRA194_MAIN_GPIO(GG, 1) GPIO_ACTIVE_LOW>; nvidia,refclk-select-gpios = <&tegra_aon_gpio TEGRA194_AON_GPIO(AA, 5) GPIO_ACTIVE_HIGH>; phys = <&p2u_nvhs_0>, <&p2u_nvhs_1>, <&p2u_nvhs_2>, <&p2u_nvhs_3>, <&p2u_nvhs_4>, <&p2u_nvhs_5>, <&p2u_nvhs_6>, <&p2u_nvhs_7>; phy-names = "p2u-0", "p2u-1", "p2u-2", "p2u-3", "p2u-4", "p2u-5", "p2u-6", "p2u-7"; #else phys = <&p2u_12>, <&p2u_13>, <&p2u_14>, <&p2u_15>, <&p2u_16>, <&p2u_17>, <&p2u_18>, <&p2u_19>; phy-names = "pcie-p2u-0", "pcie-p2u-1", "pcie-p2u-2", "pcie-p2u-3", " pcie-p2u-4", "pcie-p2u-5", "pcie-p2u-6", "pcie-p2u-7"; nvidia,pex-rst-gpio = <&tegra_main_gpio TEGRA194_MAIN_GPIO(GG, 1) GPIO_ACTIVE_LOW>; #endif}; file: /hardware/nvidia/platform/t19x/jakku/bct/pinmux/tegra19x-mb1-pinmux-p3668-a01.cfg pinmux.0x02437048 = 0x00000560; # pex_l4_clkreq_n_pl0: pe4, tristate-disable, input-enable, io_high_voltage-enable, lpdr-enable pinmux.0x02437050 = 0x00000520; # pex_l4_rst_n_pl1: pe4, tristate-disable, input-disable, io_high_voltage-enable, lpdr-enable Debug PCIe Link-Up Failure ########################## After you make the required device tree changes, as mentioned in `Enable PCIe in a customer CVB design`_, if the PCIe link fails to come up, complete the following debug steps to triage the issue: Add the ``nvidia,disable-power-down`` device tree property in PCIe controller node and complete the following debug steps. After you add this device tree property, lspci should list the Root Port device as follows: .. code-block:: none $ lspci 00:01.0 PCI bridge: NVIDIA Corporation Device 10e5 (rev a1) **Triaging from the Platform Side:** Make sure that signal routing is there from Root port to Endpoint that include: #. PERST# routing: Make sure it goes high when Root port attempts the link up. #. CLKREQ# routing: By default ASPM is disabled, so this will not have any role in link up failure. If you enabled ASPM L1-CPM or L1SS by completing the steps in `Enabling the PCIe ASPM`_, verify whether this pulled low when Root port attempts the link up. #. REFCLK routing: Make sure that REFCLK flows from Root port to Endpoint. Connect the scope and observe spread enabled 100 MHz clock. #. Tx and Rx routing: Verify Tx and Rx lanes routing are fine. #. If any of the signals above go through board level muxes, make sure those muxes are configured correctly. #. PCIe slot regulators or GPIOs: If PCIe slot regulators are present, make sure they are powered up. If endpoint excepts some GPIO signals to be toggled as part of power up sequence, make sure that it is happening, Example: Some Wi-Fi cards expects WDISABLE_1 signal to be asserted before PCIe link up. **Triaging from the Software Side:** #. Verify ``DLActive`` status in Root port ``LnkSta`` of ``lspci -vvv`` output. This is to check whether the link comes up by the time kernel boots to shell (for example, to confirm whether the link is taking more time to come up). #. Dump PADCTL_PEX_CTL_PEX_L*_CLKREQ_N_0 and PADCTL_PEX_CTL_PEX_L*_RST_N_0 pinmux values and check if settings are correct. #. Dump PCIE_RP_APPL_DEBUG_0 register, refer to TRM for register address of each controller. Accessing the controller's address, which is not enabled, will cause a CBB power down error. When you share this information in NVIDIA developer forum, it will help us determine the LTSSM state. #. Reduce the link speed to Gen-1 and link width to x1 using device tree properties. Enabling the PCIe ASPM ###################### #. If the card supports L1-CPM or L1SS add "supports-clkreq" in the respective controller device tree node. **Enabling this property for an endpoint that does not support L1-CPM or L1SS will cause a PCIe link-up failure.** #. To enable desired ASPM state, clear the respective bit in the nvidia,disable-aspm-states property . .. list-table:: :widths: auto :header-rows: 1 * - "nvidia,disable-aspm-states" value - ASPM status * - 0xf - All states disabled * - 0xe - ASPM L0s enabled, L1, L1.1 & L1.2 disabled * - 0xd - ASPM L1 enabled, L0s, L1.1 & L1.2 disabled * - 0xC - ASPM L0s & L1 enabled, L1.1 & L1.2 disabled * - 0x8 - ASPM L0s, L1 & L1.1 enabled, L1.2 disabled * - 0x1 - ASPM L1, L1.1 & L1.2 enabled, L0s disabled * - 0x0 - All ASPM states enabled #. Confirm the status of the different ASPM states from the "lspci -vvv" output and aspm_state_count in the PCIe RP controller's debugfs. Flashing the Build Image @@@@@@@@@@@@@@@@@@@@@@@@ When you flash the build image, use your specific board name. The flashing script uses the configuration in the ``.conf`` file during the flashing process. Setting Optional Environment Variables ###################################### ``flash.sh`` updates the following environment variables based on board EEPROM and other parameters passed. If you want to give specific values to these variables, define them in the board-specific file ``.conf`` to override the default values:: # Optional Environment Variables: # BCTFILE ---------------- Boot control table configuration file to be used. # BOARDID ---------------- Pass boardid to override EEPROM value # BOARDREV --------------- Pass board_revision to override EEPROM value # BOARDSKU --------------- Pass board_sku to override EEPROM value # BOOTLOADER ------------- Bootloader binary to be flashed # BOOTPARTLIMIT ---------- GPT data limit. (== Max BCT size + PPT size) # BOOTPARTSIZE ----------- Total eMMC HW boot partition size. # CFGFILE ---------------- Partition table configuration file to be used. # CMDLINE ---------------- Target cmdline. See help for more information. # DEVSECTSIZE ------------ Device Sector size. (default = 512Byte). # DTBFILE ---------------- Device Tree file to be used. # EMMCSIZE --------------- Size of target device eMMC (boot0+boot1+user). # FLASHAPP --------------- Flash application running in host machine. # FLASHER ---------------- Flash server running in target machine. # INITRD ----------------- Initrd image file to be flashed. # KERNEL_IMAGE ----------- Linux kernel zImage file to be flashed. # MTS -------------------- MTS file name such as mts_si. # MTSPREBOOT ------------- MTS preboot file name such as mts_preboot_si. # NFSARGS ---------------- Static Network assignments; client, server, # & gateway IP addresses & netmask in format: # ::: # NFSROOT ---------------- NFSROOT, an IP address, colon, and path, # e.g. 127.0.0.1:/exported/rootfs_dir. # ODMDATA ---------------- Odmdata to be used. # PKCKEY ----------------- RSA key file to use to sign bootloader images. # ROOTFSSIZE ------------- Linux RootFS size (internal emmc/nand only). # ROOTFS_DIR ------------- Linux RootFS directory name. # SBKKEY ----------------- SBK key file to use to encrypt bootloader images. # SCEFILE ---------------- SCE firmware file such as camera-rtcpu-sce.img. # SPEFILE ---------------- SPE firmware file path such as bootloader/spe.bin. # FAB -------------------- Target board's FAB ID. # TEGRABOOT -------------- lowerlayer bootloader such as nvtboot.bin. # WB0BOOT ---------------- Warmboot code such as nvtbootwb0.bin .. note:: All of the parameters must be added below the reference to the file ``p3668.conf.common`` to be reflected in the flashed image. Here is an example of environment variable settings for the Jetson Xavier NX Developer Kit ``.conf`` file, ``p3509-0000+p3668-0000-qspi-sd.conf``:: source "${LDK_DIR}/p3668.conf.common"; BLBlockSize=1048576; EMMC_CFG=flash_l4t_t194_spi_sd_p3668.xml; RECROOTFSSIZE=100MiB To flash the build image ######################## - Enter the command:: $ sudo ./flash.sh mmcblk0p1