NVIDIA® Tegra® system-on-a-chip and NVIDIA® Tegra® Board Support Package (BSP) provides many features related to power management, thermal management, and electrical management. These features deliver the best user experience possible given the constraints of a particular platform. The target user experience ensures the perception that the device provides:

This topic describes the power, thermal, and electrical management features visible to software, as well as some tools and related techniques.

Power, thermal, and electrical management features place dynamic constraints on many operational settings (“knobs”), such as:

Some of these knobs are constrained by more than one feature. For example, cpufreq implements load based scaling based on the how busy the CPU is, and adjusts the CPU frequency accordingly. CPU thermal management, however, can override the target frequency of cpufreq. Therefore, before you attempt to debug power, performance, thermal, or electrical problems, familiarize yourself with all of the power, thermal, and electrical management features in the BSP.

This topic describes the features that the SoC implements to save power and extend battery life. Many of these features are implemented by the Linux kernel, with support from firmware and hardware, and without significant involvement from the user space.

The supported power states are listed in order of increasing flexibility or configurability:

The following table describes the supported power states.

Tegra BSP maps these hardware power states onto Linux power states as follows.

Because frequency is proportional to voltage, dynamic voltage scaling is closely related to frequency scaling. For example, higher frequencies require higher voltages and vice versa.

Most clock register manipulation on Tegra Xavier is handled by the Boot and Power Management (BPMP) firmware: power management firmware running on the BPMP. A Linux kernel driver on the CPU exposes a somewhat simplified view of the physical clock tree to software on the main CPU via the Linux Common Clock Framework.

Each of the significant clock domains on the chip has its own dedicated clock source known as a Noise Aware Frequency Lock Loop (NAFLL).

The Linux regulator framework provides an abstraction that allows regulator consumer drivers to dynamically adjust voltage or current regulators at runtime, without knowledge of the underlying hardware power tree.

The framework provides a mechanism that platform initialization code can use to declare a power tree topology and assign a driver that provides regulators for each node in the hardware power tree. Such a driver is called a regulator provider driver.

Tegra BSP configures the platform power tree appropriately for Tegra devices. Additionally, drivers within Tegra BSP act as regulator consumers, where appropriate.

When porting Tegra BSP to a new platform, ensure that:

The SoC core power rails (VDD_CORE, VDD_CPU, VDD_GPU, VDD_CV) are under the direct control of the BPMP firmware. They are configured via the BPMP device tree blob (which is distinct from the Linux device tree blob)

Tegra CPU power management strategy includes dynamic frequency scaling with dynamic voltage scaling, idle power states, and core management tuned for the Tegra Xavier architecture.

Tegra BSP implements CPU Dynamic Frequency Scaling (DFS) with the Linux cpufreq subsystem. The cpufreq subsystem comprises:

The policy for frequency scaling depends on which cpufreq governor is selected at runtime.

For details consult the information available at:

For each SoC hardware reference design, a cpufreq governor is selected and tuned to achieve a balance between power and performance.

When a governor requests a CPU frequency change, the SoC-specific cpufreq platform driver reconciles that request with limits imposed by thermal or electrical constraints. The driver updates the clock speed of the CPU.

Tegra Xavier uses an NAFLL to clock each CPU. The NAFLLs are configured for AVFS. Hardware, with the assistance of the BPMP, ensures that the CPU voltage is appropriate for the NAFLL to deliver requested CPU frequencies.

The Linux cpuidle infrastructure supports the implementation of SoC-specific idle states for each CPU core. cpuidle lacks direct support for idle states applicable to an entire CPU cluster and for idle states extending beyond a CPU cluster.

For more information on the Linux cpuidle infrastructure, see:

NVIDIA provides an SoC-specific cpuidle driver that plugs into the cpuidle framework to enable CPU idle power management.

The SoC cpuidle driver exposes two per-core CPU idle states as follows:

When the final core within a cluster transitions to idle, the SoC-specific cpuidle driver can transition the CPU cluster to a cluster idle state.

Additionally, as the final core within a cluster transitions to idle, the SoC’s cpuidle driver optionally disables any SoC resources if the CPU was the last active user.

For example, the final CPU core transitioning to idle can optionally do one or more of the following:

The idle task is scheduled when there are no runnable tasks left in the run queue for a particular core. This task, through the cpuidle driver and cpuidle governor, selects the core and puts it into a low-power state, where it stays until an interrupt wakes it up to process more work.

When the last active core in a CPU cluster goes into an idle or offline state, the idle task puts the entire CPU cluster in a low-power state.

Core states are denoted as Cx states, cluster states are denoted as CCx states, and CCPlex states are denoted as CCPx states. The table below summarizes the different states available onXavier.

Use KConfig and the device tree node to enable cpuidle.

The pathnames of the nodes that represent core power states are:

Where state<y> is mapped to a specific CPU core power state.

The value returned is:

For example, to get the number of times that Denver core 2 has entered state C6, run the following command:

To get the total time in microseconds that Denver core 2 has spent in state C6, run the following command:

NVIDIA SoC chipsets include power saving features whose operation is largely invisible to software at runtime. Most of those features are statically enabled at boot, according to settings in the boot configuration table (BCT).

Additionally, Tegra BSP implements EMC frequency scaling, which is dynamic frequency scaling for the memory controller (EMC/MC) and DRAM. This is a critical power saving feature that requires tuning and characterization for each new printed circuit board design.

The calibration results include a BCT and an EMC DVFS table specific to the board design. The EMC DVFS table must be included in the platform BPMP device tree file.

The following factors affect EMC frequency scaling policy at runtime:

The Jetson AGX Xavier is designed with high efficiency PMIC, voltage regulators, and power tree to optimize power efficiency. It supports three different power modes, such as 10W, 15W, and 30W. For each mode, several configurations with various CPU frequencies and number of cores online are possible.

Capping the memory, CPU, and GPU frequencies and number of online CPU, GPU TPC, DLA and PVA cores at a pre-qualified level confines the module to the target mode. The configurations pre-defined by NVIDIA are as follows.

Where x is the power mode ID.

Once you set a power mode, the module stays in that mode until you change it. The mode persists across power cycles and SC7.

You can define your own custom mode by adding a mode definition to the following file:

Following is an example entry for mode 2:

The frequency unit of measure for the CPU is kilohertz. The unit for GPU and EMMC is hertz. You must assign a unique number in the ID field. Test your use case to:

The frequencies you select are subject to the MaxN limit defined in mode 0.

Thermal management is essential for system stability and quality of user experience. Tegra Xavier thermal management provides the following capabilities:

Thermal management in Tegra Xavier is performed by:

The following table identifies each thermal management action and the associated module for the SoC.

The Linux thermal framework provides generic user space and kernel space interfaces for working with devices that measure temperature and devices that control temperature. The central component of the framework is the Tthermal Zzone.

More information about the Linux thermal framework is available at:

A thermal zone is a virtual object that represents an area on the die whose temperature is monitored and controlled. A thermal zone acts as an object with the following components:

Tegra BSP includes drivers that provide interfaces to these components.

This topic introduces these components and demonstrates how they form a thermal zone on a Tegra-based device.

A thermal zone provides knobs to tune the thermal response of the zone. Tegra BSP provides several thermal zones tuned to provide optimum thermal performance. These provided thermal zones can be modified by editing the entries in the device tree. Users can define sensors to use temperature limits and cooling actions on those limits. When a device becomes too hot, in most cases, it can be resolved by tuning the thermal zone.

The following code snippet provides an example of a thermal zone for Tegra Xavier. This thermal zone monitors the temperature of the THERMAL_ZONE_GPU sensor.

Clock throttling is performed using the CPU-balanced cooling device when the passive trip point, trip_bthrot, is crossed at 88° Celsius.

More information about thermal knobs is available at:

A temperature sensor in a thermal zone is responsible for reporting the temperature in millidegrees Celsius. Tegra has several types of temperature sensors spread across on the die and board.

For more information see Thermal Sensing in Linux.

A trip point is used to communicate with a thermal zone. A trip point specifies the temperature (the trip) at which to perform a thermal action. Trip points are classified as active or passive, based on the type of cooling they trigger. A trip point is classified as critical if it triggers a thermal shutdown. A cooling map specifies how a cooling device is associated with certain trip points. Tegra BSP supports fan and clock throttling.

A cooling device does not actually remove heat; only a fan cooling device can do that. The Linux thermal framework makes this distinction by classifying a fan cooling device as an “active” cooling device. Clock throttling, the other type of cooling device, is classified as a “passive” cooling device.

For more information, see Cooling Devices.

Thermal management requires some form of feedback control system that keeps the device within a safe operating temperature. A governor implements this feedback control loop. While the Linux thermal framework provides many different governors, Tegra BSP provides a simple Proportional Integral Derivative (PID) controller for all passive throttling needs.

The Tegra BSP defines platform specific thermal zones. They are tuned to provide the best performance within the thermal constraints of the device. Each thermal zone uses a temperature sensor that is controlled by either the Linux kernel or the BPMP firmware, as described in the following table.

For more information, see Thermal Management in BPMP.

All gains achieved by tuning are limited by the Thermal Design Power (TDP) of the system. Tuning cannot remedy a faulty TDP. Removing all of the thermal zones does not guarantee maximum performance, and can cause resets and/or irreversible damage to the device.

Tegra BSP includes several drivers for temperature sensing.

Tegra BSP includes a driver for devices such as:

These devices can sense their own temperature as well as the temperature of a remote diode. Tegra platforms have these sensors set up as follows:

Tegra BSP configures these sensors to operate in an extended mode to increase the temperature range to −64° Celsius to 191° Celsius.

On many platforms, the voltage rail that powers the sensor is gated when Tegra enters SC7 state. Consequently, the sensor is stopped when Tegra enters SC7 and turned back on when Tegra exits SC7 state.

The NCT sensors generate thermal events for:

The NCT sensors allow software to program a static offset temperature for the remote sensor. This accounts for any inaccuracy that may be present in the sensor hardware. Tegra BSP reads the offset from the Device Tree and programs it into the offset register on boot. The offset is calculated and validated via oil bath experiments.

Tegra Xavier replaces the soctherm and aotag drivers in the Linux kernel with the tegra_bpmp_thermal sensor driver. This module registers itself as the sensor device driver with the Linux thermal framework for all the thermal sensors except the NCT sensors.

Each BPMP sensor is exposed using the Application Binary Interface (ABI), and has an ABI name as shown in the table in Tegra BSP Specific Thermal Zones. BPMP sensors, without the thermal_zone prefix, work as described in the following paragraphs. All BPMP sensors have one programmable temperature threshold (one trip), allocated for a thermal zone trip point.

The thermal framework walks through the list of thermal trip points in a thermal zone based on the current temperature. It then comes up with trip to program the BPMP sensor that is specified in the thermal zone. The tegra_bpmp_thermal driver then uses the following thermal message requests (MRQs) to communicate with the BPMP thermal framework.

The driver receives a CMD_THERMAL_HOST_TRIP_REACHED MRQ message when a particular sensor crosses a trip. The message is then relayed back to the Linux thermal framework.

For more information on these thermal management features provided as part of the Tegra BSP, see Thermal Management in BPMP.

Tegra BSP provides thermal cooling via fan control and throttling of various clocks in the system.

Tegra BSP provides active cooling by fan management through the cooling device pwm-fan, which provides:

The PWM-RPM mapping, and the various ramp rates, are stored as part of the device tree binary. The pwm-fan cooling device maps these PWM values to a cooling state. The fan cooling device can be attached to monitor the temperature of any of the Tegra BSP sensors. As the temperature increases, the governor progressively picks a deeper cooling state for the fan. This results in a higher RPM for the fan which then results in more cooling.

SoC thermal management uses the fan as the first line of defense to delay clock throttling until a much higher temperature is reached.

Tegra BSP provides thermal cooling by throttling various clocks in the system. When a rising temperature crosses a trip point, clock throttling relies on the DVFS capabilities of the clocks to reduce their operating frequency, and thereby the voltage of the rail that powers the clock. This lowered frequency and voltage reduces the power consumption which then helps in controlling the temperature.

Because cooling is achieved by reducing the clock frequency, it has a direct impact on performance and user experience. If a device feels warm and seems sluggish, it may be due to thermal throttling on the clocks. This can be remedied by tuning thermal zones that are mapped to the following balanced cooling devices:

Each of these balanced cooling devices provides several cooling states that translate to a maximum allowable operating frequency for the CPU, GPU, and EMC clocks. These frequencies are optimized to provide the best possible performance at a given temperature. The frequency tables for these clocks are part of the device tree binary.

The governor uses the current temperature of the sensor as an input to the feedback control loop. Similarly, the governor uses the control loop’s output as the new cooling state for the operation of the cooling device. As the device heats up the governor progressively picks a higher cooling setting, which then results in a higher frequency cap for all the clocks, and potentially higher cooling. Tegra BSP performs this thermal throttling of the clocks to maintain the junction temperature of the die within the recommended safe limits.

The thermal zones also define a special type of trip point called a critical trip point that triggers a software shutdown. This special trip point allows the operating system to save its state and perform an orderly shutdown before a hardware reset due to high temperature rates. Tegra BSP defines one critical trip point per thermal zone. Users can set the lower limit for the orderly shutdown. A thermal shutdown occurs after all the other cooling strategies have failed. It is considered a rare event. The shutdown limits are as follows.

The Tegra BSP thermal management features are part of the firmware running on BPMP for Tegra platforms running any host operating system (host OS) on the CPU.

The BPMP firmware hosts the soctherm and aotag drivers for the on-die thermal sensors as follows:

SOC_THERM is the collection of on-chip ring oscillators whose frequency changes are based on the temperature. To convert a measured frequency to a temperature, the oscillating frequency of the sensor, at a fixed temperature, must be known in advance and stored in the on-chip fuses.

The soctherm driver uses these fuses during boot and calibrates the sensor. Once the calibration is complete, the temperature sensor reports the temperature, in degrees Celsius, with a precision margin of 0.5° Celsius.

The temperature sensors on the chip are logically grouped into sensor groups, based on their proximity to certain hardware blocks. The sensor groups are represented as a single sensor to the host OS and the BPMP firmware.

For example, Tegra Xavier has two temperature sensors in the SOC cluster and one near the CV cluster. These are grouped as AUX sensors that are represented as THERMAL_ZONE_AUX to the operating system running on the CPUs. SOC_THERM reports the temperature of a given group by taking the maximum of all the sensors in the group.

Thermal sensors can report the temperature when the current temperature crosses a software programmed trip point. The sensors are capable of monitoring several of these software trip points to perform the following thermal actions:

To provide accurate temperature sensing, the sensors require a minimum voltage. Additionally, the sensors cannot operate when the rail is power-gated.

When the system is in a low-power state, the firmware provides the following modes of operation:

As a side effect of PLLX fallback, the programmable offset compensates for the fact that the PLLX sensor’s oscillator is farther away from the oscillator that it is replacing. The host OS continues to use all the thermal zones without side effects. The offset ensures that the CPU sensor reports more accurate temperatures than the PLLX sensor’s sensor. The host OS must therefore continue to use the right sensors for measuring the CPU temperatures.

The Always-On Thermal Alert Generator (AOTAG) is a ring oscillator based temperature sensor. It is in the always-on power domain and can monitor temperatures even when the device is in the SC7 state. Apart from this distinction, the AOTAG sensor operates the same as any of the SOC_THERM sensors.

Just like the SOC_THERM sensor, the AOTAG sensor can generate interrupts. Additionally, it can monitor two software programmed levels that the BSP uses as:

The BPMP firmware hosts a thermal framework to:

The thermal framework maintains a list of trips per sensor that includes the current trip from the host OS and various BPMP modules. As temperatures change, the framework examines the list of current trips and notifies the owner of the trip of the temperature change. The notification is sent using a callback for the BPMP owned trips and the thermal MRQ command CMD_THERMAL_HOST_TRIP_REACHED for trips that are owned by the host OS.

The primary thermal MRQ requests handled by the framework are:

For details on these MRQ requests, see the comments in this header file in the release package:

Since there can be several trips on a given sensor, the thermal framework must ensure that a notification is generated whenever a given trip is crossed. For example, if THERMAL_ZONE_CPU has a trip at 55°, 60°, 65°, and 70° Celsius, the thermal framework sends a single notification when the temperature crosses 55°, 60°, 65 ° and 70°.

Additionally, the framework implements hysteresis to prevent sending too many notifications. So, for the above example, the framework:

To perform the above notifications, the thermal framework sets low trips on the sensors to receive events that the temperature has dropped below the limit.

Each element in a power delivery system includes limitations such as:

These limitations can result in fast transient electrical and thermal events such as:

The firmware refers to these as OC alarms and triggers hardware throttling of the clocks to handle them.

Similar to software throttling, hardware throttling may cause lower performance. However, since the triggering events are rare and transient in nature, the user experience is minimally impacted.

The host OS is not notified of these events, but can detect the drop in the clocks using some performance measuring tools that sample the CPU cycle counters. While the thermal management in the host OS works to keep temperature under control, the hardware throttling performs a clampdown of the clocks to handle events.

The BPMP device tree binary holds the various throttle points and the throttle settings that govern when and how throttling is performed. The soctherm driver in the firmware programs the hardware and handles any interrupts resulting from these events. The throttle points can be modified by changing the BPMP device tree.

The throttle temperatures are as follows:

The hardware throttling levels are as follows:

Throttle vectors are optimized for limiting peak current consumption while maximizing performance. To manage peak current consumption, the firmware supports capping the CPU and GPU clocks at three levels (light, medium, and heavy), as described in the device tree bindings. This capping prevents the CPU and GPU from drawing more current than their voltage regulators can supply.

Designing failsafe measures into Power Management Integrated Circuits (PMIC), or the battery controller to shut down the device when these events occur, results in a bad user experience. Similarly, designing power delivery hardware for worst-case loads results in large and costly components. Consequently, NVIDIA SoCs are designed for power delivery systems that are adequate for common loads. Additionally, NVIDIA SoCs actively manage their components to avoid exceeding the design limits. When these events are transient in nature, the need for this design management system becomes more compelling.

The final failsafe for firmware thermal management is a hardware thermal reset or thermtrip. If software and hardware throttling are unable to control heat generation in the system, and the software becomes unresponsive, the SoC asserts the reset pin on the PMIC as the hardware shutdown mechanism.

The following are the thermtrip temperatures in Tegra Xavier:

The Jetson AGX Xavier module has 3-channel INA3221 power monitors at I2C addresses 0x40 and 0x41.

The information from the INA3221 power monitors can be read using sysfs nodes. The naming convention for sysfs nodes is as follows:

The Jetson TX2 module has 3-channel INA3221 power monitors at I2C address 0x40 and 0x41. The sysfs nodes to read for rail names, voltage, current, and power are at:

The rail names for I2C address 0x40 are:

 

The rail names for I2C address 0x41 are:

This section describes the tools and techniques to manage power.

3D frequency scaling is enabled by default.

Use the following procedures to set frequencies and report current frequency settings.

In all of these procedures, <x> is a CPU core number. For example, to apply a command to CPU core 1, replace cpu<x> with cpu1.

Tegra BSP provides the jetson_clocks.sh script to maximize Jetson AGX Xavier performance by setting static max frequency to CPU, GPU and EMC clocks. The script can also be used to show current clock settings, store current clock settings into a file, and restore clock settings from a file. The jetson_clocks.sh script is available at:

Basic usage is as follows:

 

Manage CPU hot plug with the following procedures.

Where <x> is a CPU core number.