The NVIDIA Vera CPU and NVIDIA Vera Rubin#
The NVIDIA® Vera CPU is the second-generation data center CPU designed by NVIDIA, and is the successor to the NVIDIA Grace™ CPU.
The Vera CPU has 88 high-performance and power efficient NVIDIA custom designed Olympus cores that are connected by the second generation NVIDIA Scalable Coherency Fabric and server-class LPDDR5X memory.
The Vera CPU is codesigned in various NVIDIA solutions:
The NVIDIA Vera Rubin that pairs the power efficient, high-bandwidth NVIDIA Vera CPU with two NVIDIA Rubin GPUs to maximize the capabilities for accelerated computing and generative AI workloads.
NVLink-C2C enables the GPU to have direct access to host memory.
Vera Rubin runs the full NVIDIA software stack and can be easily deployed in standard servers to run a variety of inference, data analytics, and other compute and memory-intensive workloads.
The NVIDIA Vera CPU, in dual socket and single socket combinations, with the dual socket combining two Vera CPUs in a no-compromise CPU for agentic, cloud, data analytics, storage, enterprise and HPC workloads.
The following graphic illustrates a 2S Vera CPU configuration:
NVIDIA Grace CPU Single Socket |
NVIDIA Vera CPU Single Socket |
Gen on Gen Improvement |
|
|---|---|---|---|
Cores |
72 Neoverse V2 Cores |
88 NVIDIA-Designed Arm-Compatible Cores |
1.2x |
Threads |
72 |
176 with Spatial Multithreading |
2.4x |
Cache |
1 MB per core L2 114 MB SCF Cache (L3) |
2 MB per core L2 164 MB SCF Cache (L3) |
2x 1.4x |
SIMD |
4x128b SVE2 per core |
6x128b SVE2 per core |
1.8x |
Fabric |
NVIDIA Scaled Coherency Fabric (SCF), 3.2 TB/s |
Second generation NVIDIA SCF, 3.4 TB/s |
1.1x |
Per core Fabric BW |
Up to 60 GB/s (streaming write BW) |
Up to 100 GB/s (streaming write BW) |
1.7x |
Memory Capacity |
LPDDR5X: 120 GB, 240 GB, 480 GB |
LPDDR5X: Up to 1.5 TB SOCAMM |
LPDDR5X: 3x |
Memory BW (Peak) |
Up to 512 GB/s (120 GB, 240 GB LPDDR5X) Up to 384 GB/s (480 GB LPDDR5X) |
Up to 1.2 TB/s (256 GB to 768 GB LPDDR5X) Up to 1 TB/s (1 TB, 1.5 TB LPDDR5X) |
LPDDR5X: 2.4x |
All core Memory BW |
456 GB/s |
Up to ~1.1 TB/s |
2.4x |
Raw, bi- directional NVLink-C2C |
1st Generation 900GB/s |
2nd Generation to ~1.8 TB/s |
Up 2.0x |
PCIe |
64 lanes, 5x PCIe Gen 5 x16, x8 lanes bifurcation |
Up to 96 lanes (VR) and 88 (CPU only), PCIe Gen 6 x16, x8, x4, x2 bifurcation |
3.0x |
CPU to DRAM Latency |
170 ns |
120 ns |
1.4x |
CPU to SLC Latency |
38 ns |
34 ns |
1.1x |
Memory RAS |
Base LPDDR5X Support |
Enhanced LPDDR5X |
|
CXL Support |
N/A |
Yes, 3.1 |
|
Confidential Computing |
N/A |
Yes |
|
CPU Power |
250W (OEM configurable 140W-265W) |
350W (OEM configurable 250W-450W) |
1.4x |
Performance |
1x |
2x |
2x+ |
Vera CPU Core Architecture#
Olympus, which is an NVIDIA custom designed, high-performance Arm® v9.2 CPU with 48 bits of PA and 48 bits of VA, is the NVIDIA core that drives the Vera CPU architecture.
The key optimization to the programming model is the Spatial Multithreading (SMT) feature, where each Olympus core can be programmed at runtime to execute as two independent threads.
The following graphic illustrates the block-level diagram of the core:
The frontend of the machine is designed to target applications with large code footprint. To achieve this, it features very large branch target buffers, significantly improved branch prediction units, and the ability to fetch beyond any two taken branches per cycle. Some instructions are fused before being sent to the decoders such that the 10-wide decode and rename units can process more than 10 Arm instructions per cycle. The fetch unit is capable of maintaining a constant rate of instructions such that instruction level parallelism can be extracted in the back-end of the machine.
The system’s back-end is a deep and wide out-of-order engine, and supports the following:
Six simple ALUs
Two complex ALUs
Four branches
Six vector operations
Four loads
Two stores per cycle
Here is some additional information:
To better use the width, the core breaks serial dependency chains using memory renaming and value prediction.
To sustain the execution, the memory sub-system relies on a 112-entry fully associative data TLB, and a 96KB six-way data cache.
The data and instruction TLBs are backed by a 3k entry second level TLB, which can also perform up to four concurrent page walks when needed.
The instruction and data caches are backed by a 2 MB second level private cache.
A large number of specialized prefetch engines facilitate data movement across the cache and TLB hierarchies, substantially reducing the average memory latency observed by each operation.
The following table summarizes the key differences between the CPU on the Grace systems and Olympus.
NVIDIA Grace CPU |
NVIDIA Olympus CPU |
|
|---|---|---|
Architecture |
Arm v9.0 |
Arm v9.2 |
L1 iTLB |
48 entry, FA |
64 entry, FA |
I-Cache |
64KB, 4-way |
64KB, 4-way |
Uop Cache |
1.5k entries |
N/A |
Decode Width |
6-wide / 8-wide (Uop Cache) |
10-wide w/ fused instructions |
L1 dTLB |
48 entry, FA |
112 entry, FA |
D-Cache |
64KB, 4-way |
96KB, 6-way |
L2 Cache |
1 MB, 8-way |
2 MB, 8-way |
L2 TLB |
2k entry, 8-way |
3k entry, 6-way |
ALU |
4 Simple, 2 Complex |
6 Simple, 2 Complex |
Branch |
2 |
4 |
Load/Store |
1 Load, 2 Load or Store |
4 Loads and 2 Stores |
Vector |
4x128b SVE2 |
6x128b SVE2 (adds FP8 support) |
Spatial Multithreading Mode#
Olympus introduces Spatial Multithreading, a new threading model that is different from existing Simultaneous Multithreading implementations in the following ways:
Most of the resources are statically split.
Most of the pipelines are dedicated per thread.
In more traditional Simultaneous Multithreading implementations, most resources are dynamically shared, which is the source of performance glassjaws due to interference across threads, and security issues. In the Spatial Multithreading mode, all critical resources are statically split when two threads are active which results in performance predictability and a much smaller surface for security vulnerabilities. Additionally, Simultaneous Multithreading systems typically have heuristics that pick which thread will occupy a given pipeline on a cycle by cycle basis. In the Spatial Multithreading mode, each thread has its own pipelines and can concurrently execute and stay agnostic to the other thread every cycle. An exception to the static partitioning is the vector execution units, wherein if only one thread is executing vector code, it will opportunistically leverage the vector unit of the second thread. When both threads start executing vector code, the core will detect this and statically split the vector units. This behavior is off by default, but users can enable the mode by setting the appropriate bit in the NV_CPUACTLR1_EL1 control register.
With the core being so deep and wide, static partitioning of the threads allow to fully exploit all the hardware resources. As in existing Simultaneous Multithreading systems, when only one thread is active in the core, that thread can use all the CPU resources. When Olympus is configured with Spatial Multithreading enabled, but one thread is in Wait For Interrupt (WFI) and the other thread gets all the machine’s resources, this is referred to as the Single Thread (ST) mode.
When Olympus is running with two threads in the Spatial Multithreading enabled configuration, and one of the threads has issued a WFI, the remaining threads will be able to use the full resources of the core as if it was the only thread mapped to this core. Towards this end, a hardware mechanism observes that a thread has been in WFI for enough cycles and will reconfigure Olympus to give all hardware resources to the remaining thread.
Note
The primary hardware-based mechanism only requires that the software uses WFI, so the sleeping thread’s architectural state must be retained by the hardware (for example, a WFI without a context save) so that it is available when the thread wakes up.
The reconfiguration requires a flush and saving/restoring the architectural state that has latency implications. When the system is being reconfigured to Single Thread-equivalent mode, and the thread that was in WFI wakes up, the reconfiguration is aborted and both threads operate in the Spatial Multithreading mode.
Hardware supports the following cases where two threads are active, but the system’s resources (or a subset) are fully devoted to one thread:
Switch from Two Threads Active mode to ST-equivalent mode, where one thread remains idle as long as indicated by a WFI. The overall throughput will benefit from the other thread using the system’s entire resources (including the time to switch into ST-equivalent mode and back).
The hardware will save the sleeping thread’s architectural state.
VX full-bandwidth mode allows the vector execution bandwidth to be allocated entirely to one thread in the Two Threads Active mode when the other thread has no vector micro-ops. This is configurable, and, to comply with the strict partitioning between the threads, is disabled by default.
SIMD Vectorization#
Olympus implements the following single instruction multiple data (SIMD) vector instruction sets in a 6x128-bit configuration:
The Scalable Vector Extension version 2 (SVE2)
Advanced SIMD (NEON)
Each of the six 128-bit functional units can retire SVE2 or NEON instructions, and this design allows more code to take advantage of the SIMD performance. The core has two crypto engines. In the Spatial Multithreading mode, each thread can simultaneously perform crypto operations.
In addition to the FP64, FP32, BF16 and INT8 formats supported in Grace, Olympus adds support for FP8 operations.
Many applications and libraries already take advantage of Advanced SIMD (also known as NEON). SVE is a length-agnostic, next generation SIMD instruction set architecture and is a successor to Advanced SIMD, which provides features such as prediction, first-faulting loads, gather, scatter. It also has the ability to scale to large vector lengths without requiring recompilation or manual porting to new vector lengths. SVE2 builds on VLA (Vector Length Agnostic) and adds Bitwise Permute, Multi-precision Arithmetic, String Processing, and Histogram Acceleration, which benefit general-purpose consumer and enterprise applications.
SVE is implemented in many flagship Arm implementations, and by using length-agnostic instructions for the Olympus CPU, it ensures compatibility with SVE optimizations and contributes to portable binaries across the Arm ecosystem. SVE2 also extends the SVE ISA with advanced instructions that can accelerate key HPC applications like machine learning, genomics, and cryptography.
Atomic Operations#
NVIDIA Olympus CPU supports the Large System Extension (LSE), which was introduced in Armv8.1. LSE provides the following low-cost atomic operations, which can improve system throughput for CPU-to-CPU communication, locks, and mutexes:
CAS and CASP, the Compare and Swap instructions.
LD<OP> and ST<OP>, the Atomic memory operation instructions, where <OP> is one of the following options:
ADD
CLR
EOR
SET
SMAX
SMIN
UMAX
UMIN
SWP, the Swap procedure.
These instructions can operate on integer data. All compilers that support the Olympus CPU automatically use these instructions in synchronization functions like GCC’s __atomic built-ins.
Note
In a CAS loop, you can read the initial value by using a CAS instruction
that specifies the same register for its compare and swap operands, instead
of by using a load. This form, referred to in this guide as an identity CAS,
or identity CASA, allows the memory system to take actions that are expected
to increase the likelihood of a subsequent CAS, CASA, CASAL, or CASL
succeeding. An identity CASL or CASAL does not have this effect, but the
plain and acquire forms does have this effect. The idiom is as follows, where X0 holds the
address of the location:
CAS X1, X1, [X0] // Read the current value, instead of LDR X1, [X0]
... // Compute X2, the new value
CAS X1, X2, [X0] // Attempt the update
On Vera, the idiom usually improves performance when the location is contended but can cost slightly when it is not, so measure the effect on your workload. Refer to “C6.2.51 CAS, CASA, CASAL, CASL” in the Arm Architecture Reference Manual for more information.
Note
Avoid using Load/Store Exclusives because they do not guarantee forward progress in large systems with atomics. Refer to Arm Architecture Reference Manual - Known Issues section “2.43 R22882” for more information.
Atomics to DEV memory space will raise a fault.
Additional Arm v9 Features#
The following table provides a list of the features that are supported in the Grace CPU and in Olympus.
Feature name |
Arch rev |
Grace CPU |
Olympus |
|---|---|---|---|
Arch support |
Misc |
Armv9.0 |
Armv9.2 |
AArch32 support |
Misc |
Yes |
No |
PA size |
Misc |
48 |
48 |
Mixed endian support |
Misc |
Yes |
Yes |
MPAM PARTID size (number of bits) |
Misc |
9 |
8 |
MPAM PMG size (number of bits) |
Misc |
1 |
1 |
MECID bits (for RME-MEC) |
Misc |
n/a |
10 |
SVE vector length |
Misc |
128 |
128 |
Coherent I-cache |
Misc |
Y |
Y |
FEAT_BF16 |
Armv8.6 |
Y |
Y |
FEAT_ECV |
Armv8.6 |
N |
Y |
FEAT_FGT |
Armv8.6 |
N |
Y |
FEAT_DGH |
Armv8.6 |
Y |
Y |
FEAT_I8MM |
Armv8.6 |
Y |
Y |
FEAT_MPAMv1p1 |
Armv8.6 |
N |
Y |
FEAT_PAuth2 |
Armv8.6 |
Y |
Y |
FEAT_FPAC |
Armv8.6 |
Y |
Y |
FEAT_FPACCOMBINE |
N |
Y |
|
FEAT_ETS |
Armv8.6 |
N |
Y |
FEAT_AFP |
Armv8.7 |
N |
Y |
FEAT_HCX |
Armv8.7 |
N |
Y |
FEAT_LS64 |
Armv8.7 |
N |
Y |
FEAT_LS64_V |
Armv8.7 |
N |
Y |
FEAT_LS64_ACCDATA |
Armv8.7 |
N |
Y |
FEAT_PAN3 |
Armv8.7 |
Y |
Y |
FEAT_PMUv3p7 |
Armv8.7 |
N |
Y |
FEAT_SPEv1p2 |
Armv8.7 |
N |
Y |
FEAT_WFxT |
Armv8.7 |
N |
Y |
FEAT_XS |
Armv8.7 |
N |
Y |
FEAT_nTLBPA |
Armv8.7 |
N |
Y |
FEAT_HPMN0 |
Armv8.8 |
N |
Y |
FEAT_PACQARMA3 |
Armv8.8 |
N |
Y |
FEAT_CONSTPACFIELD |
Armv8.8 |
N |
Y |
FEAT_RNG_TRAP |
Armv8.8 |
N |
Y |
FEAT_RASSA_RV |
Armv8.9 (2022 RAS Extensions) |
N |
Y |
FEAT_TCR2 |
Armv8.9 |
N |
Y |
FEAT_ECBHB |
Armv8.9 |
N |
Y |
FEAT_SCTLR2 |
Armv8.9 |
N |
Y |
FEAT_PMUv3_EXT |
Armv8.9 |
Y |
Y |
FEAT_PMUv3_EXT32 |
Armv8.9 |
Y |
Y |
FEAT_ETE |
Armv9.0 |
Y |
Y |
FEAT_SVE2 |
Armv9.0 |
Y |
Y |
FEAT_SVE_AES |
Armv9.0 |
Y |
Y |
FEAT_SVE_PMULL128 |
Armv9.0 |
Y |
Y |
FEAT_SVE_SHA3 |
Armv9.0 |
Y |
Y |
FEAT_SVE_SM4 |
Armv9.0 |
Y |
Y |
FEAT_SVE_BitPerm |
Armv9.0 |
Y |
Y |
FEAT_TRBE |
Armv9.0 |
Y |
Y |
CSV |
Armv9.1 |
N |
Y |
FEAT_BRBE |
Armv9.2 |
N |
Y |
FEAT_RME |
Armv9.2 |
N |
Y |
FEAT_ETEv1p2 |
Armv9.2 |
N |
Y |
FEAT_MEC |
Armv9.3 |
N |
Y |
Arm PMU Events#
Arm® Performance Monitor Units (PMUs) provide a standard performance tool interface for Arm-based SoCs. These PMUs measure key CPU performance metrics in CPU clusters and system level cache (SLC). Arm PMUs follow the Arm standards.
The functionality of the Arm Core PMU that is implemented in the Olympus core is exposed through performance monitoring hardware infrastructures available in Vera. Refer to the Arm Architecture Reference Manual for more information about Arm PMUs.
The following table summarizes the set of Arm PMU features or extensions that are implemented in Vera.
Feature ID |
Feature name |
Demeter (Grace) |
Olympus (Vera) |
|---|---|---|---|
FEAT_PMUv3 |
PMU extension version 3 (PMUv3) |
Yes |
Yes |
FEAT_PMUv3p1 |
PMU extensions version 3.1 (Armv8.1-PMU) |
Yes |
Yes |
FEAT_PMUv3p4 |
PMU extension version 3.4 (Armv8.4-PMU) |
Yes |
Yes |
FEAT_PMUv3p5 |
PMU Extension version 3.5 (Armv8.5-PMU) |
Yes |
Yes |
FEAT_MTPMU |
Multi-threaded PMU Extensions (Armv8.6-MTPMU) |
No |
No |
FEAT_PMUv3p7 |
Armv8.7 PMU Extensions |
Yes |
Yes |
FEAT_HPMN0 |
Setting of MDCR_EL2.HPMN to zero |
Yes |
Yes |
FEAT_PMUv3_TH |
Event counting threshold |
No |
No |
FEAT_PMUv3p8 |
Armv8.8 PMU Extensions |
No |
No |
FEAT_PMUv3_EXT64 |
Optional 64-bit external interface to the Performance Monitors |
No |
No |
FEAT_PMUv3_EXT32 |
Represents the original mostly 32-bit external interface to the Performance Monitors |
Yes |
Yes |
Feature ID |
Feature name |
Demeter (Grace) |
Olympus (Vera) |
|---|---|---|---|
FEAT_PCSRv8 |
PC Sample-base Profiling extension (not EL3 and EL2) |
No |
No |
FEAT_PCSRv8p2 |
PC Sample-based profiling version 8.2 |
Yes |
Yes |
FEAT_DoubleLock |
Double Lock |
No |
No |
FEAT_Debugv8p4 |
Debug relaxations and extensions version 8.4 |
Yes |
Yes |
FEAT_FGT |
Fine Grain Traps |
Yes |
Yes |
FEAT_ExS |
Context synchronization and exception handling |
No |
No |
FEAT_SEL2 |
Secure EL2 |
Yes |
Yes |
FEAT_DoPD |
Debug over Power Down |
Yes |
Yes |
FEAT_SPEv1p1 |
Statistical Profiling Extensions version 1.1 (Arm v8.3) |
Yes |
Yes |
FEAT_SPEv1p2 |
Statistical Profiling Extensions version 1.2 (Armv8.7) |
Yes |
Yes |
FEAT_HPMN0 |
Setting of MDCR_EL2.HPMN to zero |
Yes |
Yes |
FEAT_PCSRv8p9 |
PCSR disable control |
No |
No |
FEAT_FGT2 |
Fine-grained traps 2 |
No |
No |
FEAT_RME |
Realm Management Extension |
Yes |
Yes |
Software Lock (Obsolete when FEAT_Debugv8p4 is implemented) |
No |
No |
Vera includes core PMUs, SPE, and System PMUs, including CoreSight System PMUs
and NVIDIA proprietary System PMUs. Detailed setup, perf command templates,
event tables, top-down and derived metrics, SPE usage, uncore traffic coverage,
filters, and interpretation guidance are in Measure Workload Performance
with Hardware Performance Counters.
Vera supports Arm Confidential Computing Architecture (CCA), which enables end-to-end encryption and isolation within a trusted execution environment (TEE) that spans the CPU and GPU, extends to the rack. The Olympus core and the uncore PMUs support CCA.
Confidential Compute#
Vera implements Arm CCA, which includes the following feature IDs:
FEAT_RME
FEAT_RME_DA
FEAT_RME_MEC
FEAT_RME_CDA
Arm References:
Power and Thermal Management#
The following table provides information about managing the power and thermal issues.
Feature |
Grace |
Vera |
Description |
|---|---|---|---|
Per Core Power Gating |
All cores |
Individual cores |
Reduces leakage when a subset of cores is being used and improves the idle and runtime power. |
Per Core Voltage Regulation |
All cores |
Individual cores |
Voltage can be adjusted on a per-core basis for greater power savings at package level. |
Autonomous (HW) DVFS |
Managed by the software |
Managed by software and hardware |
Rapid, dynamic adjustment of frequency and voltage to cores done at runtime. This feature is OS agnostic with no software overhead. |
Thermal Management |
Managed by the software |
Managed by the hardware |
Faster response time, lower margin, localized control, and has no software overhead versus software management. |
Supported Memory Configurations#
The following table provides information about the supported memory configurations.
LPDDR5X |
||||
|---|---|---|---|---|
Density |
16 Gigabit Die |
24 Gigabit Die |
||
CPU Part |
Standard Vera CPU Package |
|||
Channel Config |
32x 32b channels |
|||
Rank |
2-Rank (8 DP) |
2-Rank (4 DP) |
2-Rank (8 DP) |
4-Rank (16 DP) |
Max Bitrate* |
8533 MT/s |
9600 MT/s |
7500 MT/s |
|
Peak Bandwidth** |
1092 GB/s |
1229 GB/s |
960 GB/s |
|
BW/sec/Core*** |
12.4 GB/s |
14 GB/s |
10.9 GB/s |
|
Capacity**** |
512 GB |
384 GB |
768 GB |
1536 GB |
Capacity/Core*** |
5.8 GB/core |
4.4 GB/core |
8.7 GB/core |
17.5 GB/core |
LPDDR5X |
|
|---|---|
Density |
24 Gigabit Die |
CPU Part |
Standard Vera CPU Package |
Channel Config |
30x 32b channels |
Rank |
2-Rank (8 DP) |
Max Bitrate* |
7500 MT/s |
Peak Bandwidth** |
900 GB/s |
BW/sec/Core*** |
10.2 GB/s |
Capacity**** |
768 GB raw, 720 GB usable |
Capacity/Core*** |
8.7 GB/core raw, 8.2 GB/core usable |
* Target Bit-Rate: Vendor Roadmap for 2025/2026
** Bandwidth: GB = 10^9 Bytes
*** Assumes 88 physical cores
**** Capacity: GB = 2^30 Bytes