The NVIDIA Vera CPU and NVIDIA Vera Rubin#

The NVIDIA® Vera CPU is the second-generation data center CPU designed by NVIDIA, and is the successor to the NVIDIA Grace™ CPU.

The Vera CPU has 88 high-performance and power efficient NVIDIA custom designed Olympus cores that are connected by the second generation NVIDIA Scalable Coherency Fabric and server-class LPDDR5X memory.

The Vera CPU is codesigned in various NVIDIA solutions:

  • The NVIDIA Vera Rubin that pairs the power efficient, high-bandwidth NVIDIA Vera CPU with two NVIDIA Rubin GPUs to maximize the capabilities for accelerated computing and generative AI workloads.

    • NVLink-C2C enables the GPU to have direct access to host memory.

    • Vera Rubin runs the full NVIDIA software stack and can be easily deployed in standard servers to run a variety of inference, data analytics, and other compute and memory-intensive workloads.

  • The NVIDIA Vera CPU, in dual socket and single socket combinations, with the dual socket combining two Vera CPUs in a no-compromise CPU for agentic, cloud, data analytics, storage, enterprise and HPC workloads.

The following graphic illustrates a 2S Vera CPU configuration:

_images/vera-cpu-superchip.png
NVIDIA Grace/Vera Comparison#

NVIDIA Grace CPU Single Socket

NVIDIA Vera CPU Single Socket

Gen on Gen Improvement

Cores

72 Neoverse V2 Cores

88 NVIDIA-Designed Arm-Compatible Cores

1.2x

Threads

72

176 with Spatial Multithreading

2.4x

Cache

1 MB per core L2

114 MB SCF Cache (L3)

2 MB per core L2

164 MB SCF Cache (L3)

2x

1.4x

SIMD

4x128b SVE2 per core

6x128b SVE2 per core

1.8x

Fabric

NVIDIA Scaled Coherency Fabric (SCF), 3.2 TB/s

Second generation NVIDIA SCF, 3.4 TB/s

1.1x

Per core Fabric BW

Up to 60 GB/s

(streaming write BW)

Up to 100 GB/s

(streaming write BW)

1.7x

Memory Capacity

LPDDR5X: 120 GB, 240 GB, 480 GB

LPDDR5X: Up to 1.5 TB SOCAMM

LPDDR5X: 3x

Memory BW (Peak)

Up to 512 GB/s (120 GB, 240 GB LPDDR5X)

Up to 384 GB/s (480 GB LPDDR5X)

Up to 1.2 TB/s (256 GB to 768 GB LPDDR5X)

Up to 1 TB/s (1 TB, 1.5 TB LPDDR5X)

LPDDR5X: 2.4x

All core Memory BW

456 GB/s

Up to ~1.1 TB/s

2.4x

Raw, bi- directional NVLink-C2C

1st Generation 900GB/s

2nd Generation to ~1.8 TB/s

Up 2.0x

PCIe

64 lanes, 5x PCIe Gen 5

x16, x8 lanes bifurcation

Up to 96 lanes (VR) and 88 (CPU only), PCIe Gen 6

x16, x8, x4, x2 bifurcation

3.0x

CPU to DRAM Latency

170 ns

120 ns

1.4x

CPU to SLC Latency

38 ns

34 ns

1.1x

Memory RAS

Base LPDDR5X Support

Enhanced LPDDR5X

CXL Support

N/A

Yes, 3.1

Confidential Computing

N/A

Yes

CPU Power

250W (OEM configurable 140W-265W)

350W (OEM configurable 250W-450W)

1.4x

Performance

1x

2x

2x+

Vera CPU Core Architecture#

Olympus, which is an NVIDIA custom designed, high-performance Arm® v9.2 CPU with 48 bits of PA and 48 bits of VA, is the NVIDIA core that drives the Vera CPU architecture.

The key optimization to the programming model is the Spatial Multithreading (SMT) feature, where each Olympus core can be programmed at runtime to execute as two independent threads.

The following graphic illustrates the block-level diagram of the core:

_images/olympus-core.png

The frontend of the machine is designed to target applications with large code footprint. To achieve this, it features very large branch target buffers, significantly improved branch prediction units, and the ability to fetch beyond any two taken branches per cycle. Some instructions are fused before being sent to the decoders such that the 10-wide decode and rename units can process more than 10 Arm instructions per cycle. The fetch unit is capable of maintaining a constant rate of instructions such that instruction level parallelism can be extracted in the back-end of the machine.

The system’s back-end is a deep and wide out-of-order engine, and supports the following:

  • Six simple ALUs

  • Two complex ALUs

  • Four branches

  • Six vector operations

  • Four loads

  • Two stores per cycle

Here is some additional information:

  • To better use the width, the core breaks serial dependency chains using memory renaming and value prediction.

  • To sustain the execution, the memory sub-system relies on a 112-entry fully associative data TLB, and a 96KB six-way data cache.

  • The data and instruction TLBs are backed by a 3k entry second level TLB, which can also perform up to four concurrent page walks when needed.

  • The instruction and data caches are backed by a 2 MB second level private cache.

  • A large number of specialized prefetch engines facilitate data movement across the cache and TLB hierarchies, substantially reducing the average memory latency observed by each operation.

The following table summarizes the key differences between the CPU on the Grace systems and Olympus.

Key Differences Between the Grace CPU and the Olympus CPU#

NVIDIA Grace CPU

NVIDIA Olympus CPU

Architecture

Arm v9.0

Arm v9.2

L1 iTLB

48 entry, FA

64 entry, FA

I-Cache

64KB, 4-way

64KB, 4-way

Uop Cache

1.5k entries

N/A

Decode Width

6-wide / 8-wide (Uop Cache)

10-wide w/ fused instructions

L1 dTLB

48 entry, FA

112 entry, FA

D-Cache

64KB, 4-way

96KB, 6-way

L2 Cache

1 MB, 8-way

2 MB, 8-way

L2 TLB

2k entry, 8-way

3k entry, 6-way

ALU

4 Simple, 2 Complex

6 Simple, 2 Complex

Branch

2

4

Load/Store

1 Load, 2 Load or Store

4 Loads and 2 Stores

Vector

4x128b SVE2

6x128b SVE2 (adds FP8 support)

Spatial Multithreading Mode#

Olympus introduces Spatial Multithreading, a new threading model that is different from existing Simultaneous Multithreading implementations in the following ways:

  • Most of the resources are statically split.

  • Most of the pipelines are dedicated per thread.

In more traditional Simultaneous Multithreading implementations, most resources are dynamically shared, which is the source of performance glassjaws due to interference across threads, and security issues. In the Spatial Multithreading mode, all critical resources are statically split when two threads are active which results in performance predictability and a much smaller surface for security vulnerabilities. Additionally, Simultaneous Multithreading systems typically have heuristics that pick which thread will occupy a given pipeline on a cycle by cycle basis. In the Spatial Multithreading mode, each thread has its own pipelines and can concurrently execute and stay agnostic to the other thread every cycle. An exception to the static partitioning is the vector execution units, wherein if only one thread is executing vector code, it will opportunistically leverage the vector unit of the second thread. When both threads start executing vector code, the core will detect this and statically split the vector units. This behavior is off by default, but users can enable the mode by setting the appropriate bit in the NV_CPUACTLR1_EL1 control register.

With the core being so deep and wide, static partitioning of the threads allow to fully exploit all the hardware resources. As in existing Simultaneous Multithreading systems, when only one thread is active in the core, that thread can use all the CPU resources. When Olympus is configured with Spatial Multithreading enabled, but one thread is in Wait For Interrupt (WFI) and the other thread gets all the machine’s resources, this is referred to as the Single Thread (ST) mode.

When Olympus is running with two threads in the Spatial Multithreading enabled configuration, and one of the threads has issued a WFI, the remaining threads will be able to use the full resources of the core as if it was the only thread mapped to this core. Towards this end, a hardware mechanism observes that a thread has been in WFI for enough cycles and will reconfigure Olympus to give all hardware resources to the remaining thread.

Note

The primary hardware-based mechanism only requires that the software uses WFI, so the sleeping thread’s architectural state must be retained by the hardware (for example, a WFI without a context save) so that it is available when the thread wakes up.

The reconfiguration requires a flush and saving/restoring the architectural state that has latency implications. When the system is being reconfigured to Single Thread-equivalent mode, and the thread that was in WFI wakes up, the reconfiguration is aborted and both threads operate in the Spatial Multithreading mode.

Hardware supports the following cases where two threads are active, but the system’s resources (or a subset) are fully devoted to one thread:

  • Switch from Two Threads Active mode to ST-equivalent mode, where one thread remains idle as long as indicated by a WFI. The overall throughput will benefit from the other thread using the system’s entire resources (including the time to switch into ST-equivalent mode and back).

    The hardware will save the sleeping thread’s architectural state.

  • VX full-bandwidth mode allows the vector execution bandwidth to be allocated entirely to one thread in the Two Threads Active mode when the other thread has no vector micro-ops. This is configurable, and, to comply with the strict partitioning between the threads, is disabled by default.

SIMD Vectorization#

Olympus implements the following single instruction multiple data (SIMD) vector instruction sets in a 6x128-bit configuration:

  • The Scalable Vector Extension version 2 (SVE2)

  • Advanced SIMD (NEON)

Each of the six 128-bit functional units can retire SVE2 or NEON instructions, and this design allows more code to take advantage of the SIMD performance. The core has two crypto engines. In the Spatial Multithreading mode, each thread can simultaneously perform crypto operations.

In addition to the FP64, FP32, BF16 and INT8 formats supported in Grace, Olympus adds support for FP8 operations.

Many applications and libraries already take advantage of Advanced SIMD (also known as NEON). SVE is a length-agnostic, next generation SIMD instruction set architecture and is a successor to Advanced SIMD, which provides features such as prediction, first-faulting loads, gather, scatter. It also has the ability to scale to large vector lengths without requiring recompilation or manual porting to new vector lengths. SVE2 builds on VLA (Vector Length Agnostic) and adds Bitwise Permute, Multi-precision Arithmetic, String Processing, and Histogram Acceleration, which benefit general-purpose consumer and enterprise applications.

SVE is implemented in many flagship Arm implementations, and by using length-agnostic instructions for the Olympus CPU, it ensures compatibility with SVE optimizations and contributes to portable binaries across the Arm ecosystem. SVE2 also extends the SVE ISA with advanced instructions that can accelerate key HPC applications like machine learning, genomics, and cryptography.

Atomic Operations#

NVIDIA Olympus CPU supports the Large System Extension (LSE), which was introduced in Armv8.1. LSE provides the following low-cost atomic operations, which can improve system throughput for CPU-to-CPU communication, locks, and mutexes:

  • CAS and CASP, the Compare and Swap instructions.

  • LD<OP> and ST<OP>, the Atomic memory operation instructions, where <OP> is one of the following options:

    • ADD

    • CLR

    • EOR

    • SET

    • SMAX

    • SMIN

    • UMAX

    • UMIN

  • SWP, the Swap procedure.

These instructions can operate on integer data. All compilers that support the Olympus CPU automatically use these instructions in synchronization functions like GCC’s __atomic built-ins.

Note

In a CAS loop, you can read the initial value by using a CAS instruction that specifies the same register for its compare and swap operands, instead of by using a load. This form, referred to in this guide as an identity CAS, or identity CASA, allows the memory system to take actions that are expected to increase the likelihood of a subsequent CAS, CASA, CASAL, or CASL succeeding. An identity CASL or CASAL does not have this effect, but the plain and acquire forms does have this effect. The idiom is as follows, where X0 holds the address of the location:

CAS X1, X1, [X0]    // Read the current value, instead of LDR X1, [X0]
...                 // Compute X2, the new value
CAS X1, X2, [X0]    // Attempt the update

On Vera, the idiom usually improves performance when the location is contended but can cost slightly when it is not, so measure the effect on your workload. Refer to “C6.2.51 CAS, CASA, CASAL, CASL” in the Arm Architecture Reference Manual for more information.

Note

Avoid using Load/Store Exclusives because they do not guarantee forward progress in large systems with atomics. Refer to Arm Architecture Reference Manual - Known Issues section “2.43 R22882” for more information.

Atomics to DEV memory space will raise a fault.

Additional Arm v9 Features#

The following table provides a list of the features that are supported in the Grace CPU and in Olympus.

Features Supported in the Grace CPU and in Olympus#

Feature name

Arch rev

Grace CPU

Olympus

Arch support

Misc

Armv9.0

Armv9.2

AArch32 support

Misc

Yes

No

PA size

Misc

48

48

Mixed endian support

Misc

Yes

Yes

MPAM PARTID size (number of bits)

Misc

9

8

MPAM PMG size (number of bits)

Misc

1

1

MECID bits (for RME-MEC)

Misc

n/a

10

SVE vector length

Misc

128

128

Coherent I-cache

Misc

Y

Y

FEAT_BF16

Armv8.6

Y

Y

FEAT_ECV

Armv8.6

N

Y

FEAT_FGT

Armv8.6

N

Y

FEAT_DGH

Armv8.6

Y

Y

FEAT_I8MM

Armv8.6

Y

Y

FEAT_MPAMv1p1

Armv8.6

N

Y

FEAT_PAuth2

Armv8.6

Y

Y

FEAT_FPAC

Armv8.6

Y

Y

FEAT_FPACCOMBINE

N

Y

FEAT_ETS

Armv8.6

N

Y

FEAT_AFP

Armv8.7

N

Y

FEAT_HCX

Armv8.7

N

Y

FEAT_LS64

Armv8.7

N

Y

FEAT_LS64_V

Armv8.7

N

Y

FEAT_LS64_ACCDATA

Armv8.7

N

Y

FEAT_PAN3

Armv8.7

Y

Y

FEAT_PMUv3p7

Armv8.7

N

Y

FEAT_SPEv1p2

Armv8.7

N

Y

FEAT_WFxT

Armv8.7

N

Y

FEAT_XS

Armv8.7

N

Y

FEAT_nTLBPA

Armv8.7

N

Y

FEAT_HPMN0

Armv8.8

N

Y

FEAT_PACQARMA3

Armv8.8

N

Y

FEAT_CONSTPACFIELD

Armv8.8

N

Y

FEAT_RNG_TRAP

Armv8.8

N

Y

FEAT_RASSA_RV

Armv8.9 (2022 RAS Extensions)

N

Y

FEAT_TCR2

Armv8.9

N

Y

FEAT_ECBHB

Armv8.9

N

Y

FEAT_SCTLR2

Armv8.9

N

Y

FEAT_PMUv3_EXT

Armv8.9

Y

Y

FEAT_PMUv3_EXT32

Armv8.9

Y

Y

FEAT_ETE

Armv9.0

Y

Y

FEAT_SVE2

Armv9.0

Y

Y

FEAT_SVE_AES

Armv9.0

Y

Y

FEAT_SVE_PMULL128

Armv9.0

Y

Y

FEAT_SVE_SHA3

Armv9.0

Y

Y

FEAT_SVE_SM4

Armv9.0

Y

Y

FEAT_SVE_BitPerm

Armv9.0

Y

Y

FEAT_TRBE

Armv9.0

Y

Y

CSV

Armv9.1

N

Y

FEAT_BRBE

Armv9.2

N

Y

FEAT_RME

Armv9.2

N

Y

FEAT_ETEv1p2

Armv9.2

N

Y

FEAT_MEC

Armv9.3

N

Y

Arm PMU Events#

Arm® Performance Monitor Units (PMUs) provide a standard performance tool interface for Arm-based SoCs. These PMUs measure key CPU performance metrics in CPU clusters and system level cache (SLC). Arm PMUs follow the Arm standards.

The functionality of the Arm Core PMU that is implemented in the Olympus core is exposed through performance monitoring hardware infrastructures available in Vera. Refer to the Arm Architecture Reference Manual for more information about Arm PMUs.

The following table summarizes the set of Arm PMU features or extensions that are implemented in Vera.

Arm PMU Features for the Olympus Core#

Feature ID

Feature name

Demeter (Grace)

Olympus (Vera)

FEAT_PMUv3

PMU extension version 3 (PMUv3)

Yes

Yes

FEAT_PMUv3p1

PMU extensions version 3.1 (Armv8.1-PMU)

Yes

Yes

FEAT_PMUv3p4

PMU extension version 3.4 (Armv8.4-PMU)

Yes

Yes

FEAT_PMUv3p5

PMU Extension version 3.5 (Armv8.5-PMU)

Yes

Yes

FEAT_MTPMU

Multi-threaded PMU Extensions (Armv8.6-MTPMU)

No

No

FEAT_PMUv3p7

Armv8.7 PMU Extensions

Yes

Yes

FEAT_HPMN0

Setting of MDCR_EL2.HPMN to zero

Yes

Yes

FEAT_PMUv3_TH

Event counting threshold

No

No

FEAT_PMUv3p8

Armv8.8 PMU Extensions

No

No

FEAT_PMUv3_EXT64

Optional 64-bit external interface to the Performance Monitors

No

No

FEAT_PMUv3_EXT32

Represents the original mostly 32-bit external interface to the Performance Monitors

Yes

Yes

PMU-Dependent Arm-core Features for the Olympus Core#

Feature ID

Feature name

Demeter (Grace)

Olympus (Vera)

FEAT_PCSRv8

PC Sample-base Profiling extension (not EL3 and EL2)

No

No

FEAT_PCSRv8p2

PC Sample-based profiling version 8.2

Yes

Yes

FEAT_DoubleLock

Double Lock

No

No

FEAT_Debugv8p4

Debug relaxations and extensions version 8.4

Yes

Yes

FEAT_FGT

Fine Grain Traps

Yes

Yes

FEAT_ExS

Context synchronization and exception handling

No

No

FEAT_SEL2

Secure EL2

Yes

Yes

FEAT_DoPD

Debug over Power Down

Yes

Yes

FEAT_SPEv1p1

Statistical Profiling Extensions version 1.1 (Arm v8.3)

Yes

Yes

FEAT_SPEv1p2

Statistical Profiling Extensions version 1.2 (Armv8.7)

Yes

Yes

FEAT_HPMN0

Setting of MDCR_EL2.HPMN to zero

Yes

Yes

FEAT_PCSRv8p9

PCSR disable control

No

No

FEAT_FGT2

Fine-grained traps 2

No

No

FEAT_RME

Realm Management Extension

Yes

Yes

Software Lock (Obsolete when FEAT_Debugv8p4 is implemented)

No

No

Vera includes core PMUs, SPE, and System PMUs, including CoreSight System PMUs and NVIDIA proprietary System PMUs. Detailed setup, perf command templates, event tables, top-down and derived metrics, SPE usage, uncore traffic coverage, filters, and interpretation guidance are in Measure Workload Performance with Hardware Performance Counters.

Vera supports Arm Confidential Computing Architecture (CCA), which enables end-to-end encryption and isolation within a trusted execution environment (TEE) that spans the CPU and GPU, extends to the rack. The Olympus core and the uncore PMUs support CCA.

Confidential Compute#

Vera implements Arm CCA, which includes the following feature IDs:

  • FEAT_RME

  • FEAT_RME_DA

  • FEAT_RME_MEC

  • FEAT_RME_CDA

Arm References:

Power and Thermal Management#

The following table provides information about managing the power and thermal issues.

Power and Thermal Management Features#

Feature

Grace

Vera

Description

Per Core Power Gating

All cores

Individual cores

Reduces leakage when a subset of cores is being used and improves the idle and runtime power.

Per Core Voltage Regulation

All cores

Individual cores

Voltage can be adjusted on a per-core basis for greater power savings at package level.

Autonomous (HW) DVFS

Managed by the software

Managed by software and hardware

Rapid, dynamic adjustment of frequency and voltage to cores done at runtime. This feature is OS agnostic with no software overhead.

Thermal Management

Managed by the software

Managed by the hardware

Faster response time, lower margin, localized control, and has no software overhead versus software management.

Supported Memory Configurations#

The following table provides information about the supported memory configurations.

Supported Memory Configurations for NVIDIA Vera CPU#

LPDDR5X

Density

16 Gigabit Die

24 Gigabit Die

CPU Part

Standard Vera CPU Package

Channel Config

32x 32b channels

Rank

2-Rank (8 DP)

2-Rank (4 DP)

2-Rank (8 DP)

4-Rank (16 DP)

Max Bitrate*

8533 MT/s

9600 MT/s

7500 MT/s

Peak Bandwidth**

1092 GB/s

1229 GB/s

960 GB/s

BW/sec/Core***

12.4 GB/s

14 GB/s

10.9 GB/s

Capacity****

512 GB

384 GB

768 GB

1536 GB

Capacity/Core***

5.8 GB/core

4.4 GB/core

8.7 GB/core

17.5 GB/core

Supported Memory Configurations for NVIDIA Vera Rubin#

LPDDR5X

Density

24 Gigabit Die

CPU Part

Standard Vera CPU Package

Channel Config

30x 32b channels

Rank

2-Rank (8 DP)

Max Bitrate*

7500 MT/s

Peak Bandwidth**

900 GB/s

BW/sec/Core***

10.2 GB/s

Capacity****

768 GB raw, 720 GB usable

Capacity/Core***

8.7 GB/core raw, 8.2 GB/core usable

* Target Bit-Rate: Vendor Roadmap for 2025/2026

** Bandwidth: GB = 10^9 Bytes

*** Assumes 88 physical cores

**** Capacity: GB = 2^30 Bytes