Is this page helpful?

Optimization for DGX Spark#

This section discusses software optimization practices that are relevant for DGX Spark.

General Practices#

  • Profile your application to find hotspots that would benefit most from optimization. Making assumptions about hotspots can lead to incorrect conclusions. Some appropriate tools are discussed in a section below.

  • Create reproducible microbenchmarks when optimizing hotspots. Measuring the runtime of short functions can be tricky; consider using a framework like google benchmark or nanobench. These frameworks generally handle the most notorious microbenchmarking challenges such as data variance, outliers, warmups, and subtracting overhead. Setting up microbenchmarks will help you track your progress while optimizing a hotspot.

  • If possible, take advantage of vectorization, either by using your compiler’s auto-vectorization or doing it by hand. SIMD is generally faster and more power-efficient with respect to work done than scalar code. If you are migrating vectorized x86_64 code, also see CPU Instruction Extensions.

  • Pay attention to memory access patterns. If appropriate, use tiling methods to maintain data locality.

    Note that DGX Spark is a hybrid architecture with varying cache sizes across its cores.

  • The cores in DGX Spark are organized as two clusters with up to 5 P-cores and 5 E-cores each. Minimize concurrent writes to the same cache lines by different cores, especially ones in different clusters. Cache lines are 64 bytes long. Keep thread synchronization to a necessary minimum.

  • When parallelizing large workloads using OpenMP, also consider guided or dynamic scheduling to better balance tasks in a hybrid architecture. If tasks complete quickly, try increasing the chunk size to keep synchronization overhead under control.

Compiler Configuration#

DGX Spark is based on ARMv9.2-A architecture. Developers may choose to target an older architecture to maintain compatibility; it is recommended to target at least ARMv8.2 with the rcpc extension.

gcc and clang#

A specific architecture can be targeted by adding the -march switch, for example -march=armv9.0-a.

Since LLVM version 21 and gcc version 15, it is also possible to specifically target DGX Spark with -mcpu=gb10. This enables optimizations for the Cortex-X925 and Cortex-A725 cores as well as the optional crypto extension and is thus preferred over the more generic -march switch.

When compiling on the DGX Spark itself, -march=native will have the compiler target the host machine’s architecture.

When targeting an architecture older than ARMv8.2-A, it is advisable to also enable -moutline-atomics. This option will replace inline atomics with calls to the best atomics available for the platform, as determined at runtime.

.NET Applications#

ARM Advanced FP & SIMD (also known as NEON) data types are available in modern .NET runtimes through the Vector<T>, Vector128<T>, and similar classes. Operations and ISA extensions on these vectors are accessible in the System.Runtime.Intrinsics.Arm namespace. When targeting multiple architectures, the availability of a particular ISA extension can be determined at runtime, for example System.Runtime.Intrinsics.Arm.Dp.IsSupported.

Access to SVE instructions is currently experimental and may have implications when using NativeAOT because of the runtime-determined vector width. See dotnet issue 93095 and Engineering the Scalable Vector Extension in .NET for more information.

Programming CUDA on UMA systems#

To efficiently use the DGX Spark consider reading the CUDA Programming Guide. The guide provides detailed information regarding platform-specific considerations and best practices for optimizing CUDA workflows on UMA-enabled hardware.

Memory reporting on UMA systems (including DGX Spark)#

DGX Spark systems use a unified memory architecture (UMA), where the GPU shares system memory (DRAM) with the CPU and other compute engines. This design reduces latency and allows larger amounts of memory to be used for GPU workloads. On UMA systems, the CPU can dynamically manage DRAM contents, including freeing up memory from the system�s SWAP area and page caches. However, the cudaMemGetInfo API does not account for memory that could potentially be reclaimed. As a result, the memory size reported by cudaMemGetInfo may be smaller than the actual allocatable memory, since the CPU may be able to release additional DRAM pages.

To more accurately estimate the amount of allocatable device memory on DGX Spark platforms, CUDA application developers should consider the memory that can be reclaimed from the OS and not rely solely on the values returned by cudaMemGetInfo. For a reference implementation using C standard libraries, see the link below. This snippet returns the maximum memory that can be allocated without swapping (availableMemory) and the maximum memory that can be allocated with swapping (availableMemory + freeSwap).

See Guidance for reporting memory resources with unified memory architecture.

As a workaround for debugging purposes, you can flush the buffer cache manually with the following command:

sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'

After flushing the cache, restart your application.

Display Reserved Memory#

On DGX Spark, UEFI reserves part of system memory for the display. This memory is not available to the OS, CUDA, or applications. Starting with OTA2610, the size of this reservation is configurable. The default is 2 GB. Supported sizes are Auto, 0.5 GB, 1 GB, 2 GB, and 4 GB. For headless or memory-intensive workloads, select Auto or 0.5 GB to make up to 1.5 GB more memory available to the OS. For configuration steps, refer to Display Reserved Memory in the DGX Spark UEFI Manual.

Set the reservation from the OS#

You can also request a new size from the OS without entering UEFI setup. Write the UEFI variable NvidiaDispFrmSizeRequest. UEFI reads the request on the next boot, checks it, applies it if it is valid, and then deletes the request. An invalid value is ignored.

Item

Value

Variable name

NvidiaDispFrmSizeRequest

Vendor GUID

284e19b9-235f-4aab-ae4c-190456d1bd76

Attributes

Non-volatile, boot service, runtime (0x00000007)

Data

1 byte: 0 = Auto, 1 = 0.5 GB, 2 = 1 GB, 3 = 2 GB, 5 = 4 GB

When applied

On the next boot

Example (Linux, set Auto, then restart):

sudo chattr -i /sys/firmware/efi/efivars/NvidiaDispFrmSizeRequest-284e19b9-235f-4aab-ae4c-190456d1bd76 2>/dev/null
printf '\x07\x00\x00\x00\x00' | sudo tee \
/sys/firmware/efi/efivars/NvidiaDispFrmSizeRequest-284e19b9-235f-4aab-ae4c-190456d1bd76 > /dev/null
sudo reboot

Read the current reservation from the OS#

Each boot, UEFI publishes the current setting and the reserved size in the variable NvidiaDispFrmStatus. This variable is not stored in flash, and UEFI writes it again on every boot.

Item

Value

Variable name

NvidiaDispFrmStatus

Vendor GUID

284e19b9-235f-4aab-ae4c-190456d1bd76

Data

5 bytes, packed: byte 0 = configured option (same codes as above); bytes 1–4 = reserved size in MiB (UINT32, little-endian)

Example (Linux):

sudo od -An -tx1 /sys/firmware/efi/efivars/NvidiaDispFrmStatus-284e19b9-235f-4aab-ae4c-190456d1bd76
# The first 4 bytes are attributes. Then: mode (1 byte), size in MiB (4 bytes).
# Example: 06 00 00 00  00  00 02 00 00  ->  Auto, 0x200 = 512 MiB

Profiling#

Profiling your application is highly advisable before undertaking any optimization efforts. The tools of choice on Linux platforms are Nsight tools and perf.

Nsight Systems#

NVIDIA Nsight Systems is a powerful performance analysis tool that presents the captured traces in a top-down graphical manner. Profiles are presented as a timeline of evolving metrics, such as per-thread CPU usage, GPU metrics and other data.

It is possible to profile an application through an SSH connection using Nsight systems. When creating an Nsight project, select an existing SSH connection or add a new one using the Configure targets… item.

Additionally NVIDIA Nsight Graphics and NVIDIA Nsight Compute are available to analyze 3D rendering and compute workloads.

Recording perf traces has security implications; you may need to unlock access to performance counters by adding a file to /etc/sysctl.d with the following content:

kernel.perf_event_paranoid=-1
kernel.kptr_restrict=0

It is also possible to enable these settings only until the next reboot.

To install the perf tools, execute sudo apt install linux-tools-$(uname -r).

To record a trace for your application and launch the analyzer:

sudo perf record -e cycles,(...) ./your-application
perf report

In perf report, a top-down view of hot functions is presented. A mixed C++ and disassembler view with proportion of time spent can be generated by selecting a function (shared object) and using the Annotate function. If the C++ code is not shown in the disassembler view, ensure the source code can be found by the profiler and that the application was compiled with the -g switch, which includes debug information in the binary.

For a listing of available performance monitoring counters, see Arm Cortex-X925 Performance monitoring events and Arm Cortex-A725 Performance monitoring events.