.. _AR.BootArchitecture.JetsonXavierNxAndJetsonAgxXavierBootFlow: .. include:: /content/swdocs.rsts .. spelling:: CBoot clib eDP Microboot mSATA Zlib BootROM SysRAM TBoot NVDisp init framebuffer Jetson Xavier NX and Jetson AGX Xavier Boot Flow !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! .. todo:: All of the diagrams in this section have been redrawn to improve readability and produce SVG files. Please check for accuracy. **Boot flow** is the sequence of operations that the Bootloader performs initialize the SoC and boot |NVIDIA(r)| |Jetson(tm)| Linux. The major operations the bootloader performs are: - Initializing the memory controller (MC), external memory controller (EMC), and CPU - Setting up security parameters - Loading firmware components - Maintaining the chain of trust - Setting memory carveouts for various firmware components - Flashing the device .. todo:: Flashing the device is considered part of boot flow??? Why? In the document outline it's a separate topic under "Software Features in Depth," not even part of the "Boot Architecture" topic. - Booting to the operating system Additionally, the Jetson boot software may perform other operations defined by product requirements, including but not limited to: - Initializing |HDMI(r)| or DisplayPort - Displaying a boot logo The following diagram shows the flow of control in the boot software. .. todo:: These are OK for T194, IIRC, but T234/Orin has a completely different boot flow - PSC added, etc. Check w/either the Orin HW architects or the BL team for the correct diagram for T234, I can't find it right now. .. figure:: JetsonXavierNxAndJetsonAgxXavierBootFlow/FlowOfControl.svg :figwidth: 650px BootROM @@@@@@@ **BootROM** (BR) is hard-wired into the processor. It initializes the boot media and loads Microboot1 (MB1) from it. Multiple copies of the **BootROM Boot Configuration Table*** (BR-BCT) may be stored at the start of the boot media. The BR-BCT contains configuration parameters used by the BootROM for hardware initialization. The BCT also contains information about the Bootloader (BL), including: - Size - Entry point - Load address - Hash The BootROM uses this information to verify and load the Bootloader. The boot flow is shown in the following diagram. .. figure:: JetsonXavierNxAndJetsonAgxXavierBootFlow/Bootrom.svg :figwidth: 650px Bootloader Components @@@@@@@@@@@@@@@@@@@@@ The Bootloader and flash components are: - Microboot1 (MB1) - TBoot-BPMP - TBoot-CPU (used in flashing) - UEFI (kernel/OS loader used in cold boot) Common Driver Framework ####################### TBoot-BPMP and TBoot-CPU use a common set of drivers. Some libraries are common across the Bootloader components. Instead of using separate sets of drivers and libraries, the binaries share a common pool of drivers and libraries called the **Common Driver Framework** (CDF). The following diagram shows the software architecture of the Common Driver Framework. .. figure:: JetsonXavierNxAndJetsonAgxXavierBootFlow/CommonDriverFramework.svg :figwidth: 650px .. todo:: I've done my best to reorganize this diagram so that the text is readable, but there's just too much detail. Ideally, an engineer will help me redesign it to present the detail in a different way. Failing that, the reader will just have to enlarge the screen. As a separate issue, the original diagram's key defined a fourth color, medium green, for "NVIDIA Component." I don't know what this means, since all of the components are NVIDIA components. Is dropping this from the color key correct and sufficient? As another separate issue, can we replace "SE" with a more specific abbreviation? Even if the reader knows what "SE" means in context, it's inherently ambiguous because it's so short. The Common Driver Framework consists of: - Storage drivers like eMMC, QSPI-NOR, mSATA, and UFS - Host interface drivers like UART, USB - Display drivers (HDMI, DP, and eDP) - Debug and Console library - Libraries of other modules such as: - Clock - SE - PMIC - Timer - Fuse - GPIO - PWM - EEPROM - Keyboard - PSCI - USB host mode (2.0 mode only) - USB host class drivers (only mass storage devices, no hub or HID support) - Software libraries such as: - Cryptographic - Transport - Partition manager - Sparse - Transport libraries .. todo:: Some entries are adjectives (cryptographic). Others are descriptive phrases (Transport, Partition manager). Let's be consistent. As a related issue, what does it mean to have software libraries named "Transport" and "Transport libraries"? If these are not duplicate entries, they need to be distinguished more clearly. - A decompression library that supports LZF, Zlib, and LZ4 - ``clib``, a dynamic memory allocation library, and a cache library .. todo:: From the source, we have Zlib, capitalized in text font, and ``clib``, lower case in code font. I suspect these are both library names or both filenames, and they should be composed the same way. As a separate issue, it's confusing for a single list entry to name one library and describe two others. This reads as if "a dynamic memory allocation library" is an explanation of "clib," then "a cache library" is... what? The CDF is provided at these locations:: /vendor/nvidia/tegra/bootloader/partner/common/ /vendor/nvidia/tegra/bootloader/partner/t18x/common/ .. todo:: I suspect, but am not sure, that these two libraries are alternatives for different processors, one of which is no longer supported. Microboot1 ########## Microboot1 (MB1) is the first boot software component loaded by BR in SysRAM, and runs on BPMP. This component implements certain platform initialization (including CPU) and security configuration. .. todo:: Is it important to use abbreviations like BR? They add to the reader's cognitive load by introducing a whole group of technical terms for which the reader must learn and remember not only meanings, but abbreviations. "BootROM," for example, would be only a little longer and a lot clearer. .. todo:: "SysRAM" is introduced here without ever being explained. Please correct this, and watch for and correct similar cases. (The reader can infer that it means "system RAM," but the very fact that a special term is used implies that it has a more specific technical meaning. If it doesn't, we should say "system RAM," which again is only a little longer and doesn't raise that issue.) MB1 is signed and encrypted by an NVIDIA-owned key. The following diagram shows the flow of control in MB1. .. figure:: JetsonXavierNxAndJetsonAgxXavierBootFlow/Microboot1.svg :figwidth: 650px .. todo:: In this diagram and all of the following ones, the colors are unexplained. (I've changed them to make them more readily distinguishable. I assume that will not interfere with their meaning.) MB1 is responsible for: - Platform configuration, including pinmux, GPIO , pad voltage, SCRs, and firewalls - Initializing the SDRAM based on the MB1 boot configuration table (MB1-BCT) - Loading firmware, including the components that initialize the CPU complex (CCplex) - Programming the PMIC for enabling the VDD_CPU rail - Creating memory carveouts - Loading the next stage Bootloader, TBoot-BPMP MB1 is owned by NVIDIA, and so is provided as a binary in the Jetson BSP package. Although it is provided as a binary, you can configure its behavior for a specific platform using its Boot Configuration Table, called MB1-BCT. For information about the Boot Configuration Table, see MB1 BCT. .. todo:: I think "MB1 BCT" is supposed to be a link, but a section with that title doesn't appear anywhere in the document. There a section titled "About MB1 BCT" in "MB1 Platform Configuration." TegraBoot @@@@@@@@@ TegraBoot is the Bootloader component that executes after MB1. It is divided into two components: - TBoot-BPMP - TBoot-CPU The processor on which these parts run determines which component runs. .. todo:: This appears to mean that these two components are alternative paths to completing the boot process that run on different processors, one or the other being selected in an unspecified way. I suspect that not what's intended; execution probably forks and runs both of these components on their respective processors, then proceeds when both have finished. Please clarify. TegraBoot-BPMP ############## TegraBoot-BMP This portion of TegraBoot runs on BPMP (TBoot-BPMP). There are two variants of TegraBoot-BPMP: .. todo:: It's not clear whether "Tegra-BPMP" and "TBoot-BPMP" are related but distinct components, or two names for the same thing. Please clarify. - One for cold boot - One for flashing and RCM (recovery) boot TBoot-BPMP is responsible for: - Loading and initializing firmware (FW) components. - Creating memory carveouts - Restarting the CPU after it is halted - Loading the next stage Bootloader - Supporting flashing - Supporting RCM boot The following diagram shows the components of TBoot-BPMP. .. figure:: JetsonXavierNxAndJetsonAgxXavierBootFlow/TegrabootBpmp.svg :figwidth: 650px .. todo:: I notice that several of the labels in the green elements end with "-FW," which I assume means "firmware." Is this necessary? The enclosing gray container is labeled "Load and authorize firmware," so we're labeling these elements as firmware twice. And what about the ones that *don't* end with "-FW"? Are they *not* firmware? If not, why are they in a "firmware" container, or why is the container labeled as it is? TegraBoot-CPU ############# TegraBoot-CPU runs on the CPU, and is responsible for: - Flashing - RCM boot The RCM boot flow is similar to a cold boot except that the binaries are transferred from the host over USB, and are loaded directly into SDRAM. A trusted operating system (TOS) and BPMP-FW are not loaded in this path. .. note:: The CPU starts execution in EL3 mode, and executes the TOS monitor. The TOS monitor completes its initialization and gives control to TegraBoot-CPU in EL2 mode. It then initializes the USB and starts the PPP (Point-to-Point Protocol) protocol to flash the device. - For normal flashing, TegraBoot-CPU takes binaries from the host one by one and flashes them to the device. - For RCM boot, TegraBoot-CPU takes binaries downloaded over USB. The binaries are not flashed to the device. The following diagram shows the components of TBoot-CPU. .. figure:: JetsonXavierNxAndJetsonAgxXavierBootFlow/TegrabootCpu.svg :figwidth: 650px NVDisp-init ########### NVDisp-init runs on the CPU/CCPLEX, is derived from public CBoot source, and is only responsible for display HW init. NVDisp-init initializes the T194 display HW and chains/hands off to UEFI so that UEFI can copy a BMP file to the display framebuffer for the boot logo display. The NVDisp-init binary is bundled with the UEFI binary during flash, and MB2 loads it as the CPU-BL. UEFI @@@@ **Unified Extensible Firmware Interface** (UEFI) is an industry specification that describes standard interfaces between platform firmware and the operating system. It replaces CBoot in the Jetson boot flow as the CPUBL for Jetson Linux devices. .. todo:: CPUBL is mentioned several times in the MB1 topic, but is never explained. It is apparently not a generally recognized term. We can't use it this way without explaining what it is -- what is UEFI replacing? Features of UEFI include: - Along with other specifications (SMBIOS, ACPI), it can load a generic OS without requiring any platform customization of the operating system. - It defines a standardized secure boot mechanism for authenticating third-party software (i.e. operating systems and PCIe option ROMs). - It supports add-in card drivers via option ROMs, and integrates them with the system configuration user interface. - It defines standard methods for updating firmware. The following diagram describes the major aspects of UEFI boot flow. .. figure:: JetsonXavierNxAndJetsonAgxXavierBootFlow/UefiBootFlow.png :figwidth: 650px .. todo:: The diagram is a PNG copied from Winnie's email of 3/30/2022 @22:38. At this point (three working days before release) there is no time to redraw it in Visio. Note, the colors used in the diagram clearly have some meaning, but it has not been explained. UEFI sources and compilation details for this release are available at: https://github.com/NVIDIA/edk2-nvidia/wiki