Most of the coverage of Nvidia’s data center push focuses on the GPU — Blackwell’s transistor count, its tensor core throughput, its NVLink fabric. But sitting next to that GPU in Nvidia’s highest-end systems is a CPU most of the conversation skips past: Grace, Nvidia’s own Arm-based processor. Grace isn’t a side project — it’s the piece that lets Nvidia build “superchips” where a CPU and one or more GPUs share a single coherent memory space at bandwidths ordinary PCIe-connected x86 servers can’t match. This piece focuses specifically on Grace and the Arm-based CPU+GPU architecture it enables, distinct from Blackwell’s own GPU architecture, which Nvidia documents separately.
At a Glance
| Product | Composition | CPU Cores | CPU Memory | CPU-GPU Interconnect |
|---|---|---|---|---|
| Grace CPU Superchip | 2x Grace CPU chips (CPU-only) | 144 Arm Neoverse V2 cores total (72 per chip) | Up to 960GB LPDDR5X | 900 GB/s NVLink-C2C (chip-to-chip) |
| Grace Hopper Superchip | 1x Grace CPU + 1x Hopper GPU | 72 Arm Neoverse V2 cores | Up to 480GB LPDDR5X | 900 GB/s NVLink-C2C, coherent CPU-GPU memory |
| GB200 Grace Blackwell Superchip | 1x Grace CPU + 2x Blackwell GPUs | 72 Arm Neoverse V2 cores | Up to 480GB LPDDR5X at up to 512 GB/s | NVLink-C2C per GPU; 3.6 TB/s total NVLink bandwidth |
Short Verdict
Grace matters because of what it enables rather than because of raw CPU benchmarks: a CPU-GPU link fast enough and coherent enough that the GPU can treat CPU memory almost like an extension of its own, instead of waiting on comparatively slow PCIe transfers. For workloads that move enormous amounts of data between CPU and GPU — large language model training and inference chief among them — that coherent link is often a bigger practical win than any single spec. For general-purpose, GPU-free workloads, the standalone Grace CPU Superchip is Nvidia’s bet that Arm-based energy efficiency can compete with established x86 server CPUs on performance-per-watt, which is Nvidia’s own claim and worth validating against independent benchmarks for any specific workload before committing.
What a “Superchip” Actually Is
Nvidia’s “superchip” naming refers to multiple dies — two CPUs, or a CPU and GPUs — packaged and connected via Nvidia’s NVLink Chip-to-Chip (NVLink-C2C) interconnect, rather than communicating over a standard motherboard bus like PCIe. The Grace CPU Superchip connects two Grace CPU dies over NVLink-C2C at 900 GB/s, for a combined 144 Arm Neoverse V2 cores and up to 960GB of LPDDR5X memory — a CPU-only product aimed at traditional server workloads that still benefit from Arm’s power efficiency, with no GPU involved. The Grace Hopper Superchip instead pairs one Grace CPU with one Hopper GPU, and the GB200 Grace Blackwell Superchip pairs one Grace CPU with two Blackwell GPUs — in both cases using the same NVLink-C2C link, but now joining a CPU to GPUs rather than CPU to CPU.
Why Arm Instead of x86
Grace is built on Arm’s Neoverse V2 core design rather than x86, the architecture nearly all traditional data center CPUs (Intel Xeon, AMD EPYC) use. Nvidia’s rationale, laid out in its own architecture documentation, centers on power efficiency and memory design choices that are easier to make from a clean-sheet Arm design: Grace uses server-class LPDDR5X memory with ECC — which Nvidia describes as the first use of this memory class in a data center server CPU — instead of the DDR5 or HBM found in most server CPUs. Nvidia states LPDDR5X delivers up to 500 GB/s of bandwidth at around 16W, roughly one-fifth the power of comparable DDR5 memory, and claims the Grace CPU Superchip offers roughly 2x the performance-per-watt of traditional x86 server CPUs using DDR5, plus up to 3x better energy efficiency on graph analytics workloads versus leading x86 systems in Nvidia’s own benchmark configurations. Grace is far from the only Arm server chip in this market — AWS’s Graviton and Google’s Axion take a similar power-efficiency bet on Arm — but Grace is distinguished by being built specifically to pair with Nvidia’s own GPUs via NVLink-C2C, which neither Graviton nor Axion is designed to do.
The Real Differentiator: Coherent CPU-GPU Memory
In a conventional x86 AI server, the CPU and GPU are separate chips connected over PCIe, with each maintaining its own distinct memory pool; moving data between them means explicit transfers across a comparatively narrow, higher-latency link, and the CPU and GPU do not share a unified view of memory. Grace-based superchips change that by connecting CPU and GPU dies with NVLink-C2C at 900 GB/s — roughly 7x the bandwidth of PCIe Gen5 x16 — and, in the Grace Hopper and Grace Blackwell designs, maintaining memory coherency between the CPU’s LPDDR5X and the GPU’s HBM. In practice, this means the GPU can address CPU-attached memory (and vice versa) without the application explicitly managing separate memory pools and copies, which matters most for workloads with large memory footprints that exceed what fits in GPU HBM alone, such as large-scale LLM training and serving. The GB200 superchip, for instance, pairs 480GB of Grace’s LPDDR5X with 372GB of Blackwell HBM3E running at 16 TB/s, giving a combined system memory pool the GPU can draw on far beyond its own onboard HBM.
From Superchip to Rack: GB200 NVL72
Nvidia scales the Grace Blackwell architecture up from an individual superchip to a full rack in its GB200 NVL72 system, which combines 36 Grace CPUs and 72 Blackwell GPUs (18 GB200 superchips) in a single liquid-cooled rack. Nvidia’s NVLink Switch System connects all 72 GPUs into one NVLink domain that Nvidia describes as functioning like a “single massive GPU,” with 130 TB/s of aggregate NVLink bandwidth. At the rack level, the combined Grace CPU memory reaches 17 TB of LPDDR5X alongside 13.4 TB of HBM3E GPU memory. This is the Grace architecture’s clearest demonstration: Nvidia isn’t just building a fast chip, it’s building an entire rack-scale system where CPU, GPU, memory, and interconnect are co-designed, something that’s much harder to replicate by bolting Nvidia GPUs onto a generic x86 server platform.
Trade-offs
- Pros: NVLink-C2C’s 900 GB/s CPU-GPU link and coherent memory substantially reduce data-movement bottlenecks for memory-intensive AI workloads; LPDDR5X’s power efficiency reduces overall system power draw; Arm Neoverse V2 cores bring strong performance-per-watt for general compute running alongside GPU work.
- Trade-offs: Arm-based Grace requires software compiled or recompiled for Arm (aarch64), which can complicate migration for organizations with deep x86-specific toolchains, legacy binaries, or Arm-incompatible dependencies; Grace-based systems are tightly coupled to Nvidia’s own GPU and interconnect roadmap, which is a form of platform lock-in distinct from a generic x86 server that can pair with GPUs from multiple vendors.
Who This Matters For
Infrastructure teams designing or procuring large-scale AI training and inference clusters are the primary audience, especially where model or dataset sizes push against the memory limits of GPU HBM alone. It also matters to organizations evaluating Arm as a general-purpose server CPU architecture for efficiency reasons, independent of any GPU attachment, via the standalone Grace CPU Superchip. It’s less relevant to teams running smaller, GPU-memory-bound workloads that don’t benefit meaningfully from CPU-GPU memory coherence.
Bottom Line
Grace is Nvidia’s answer to a specific bottleneck: moving enormous volumes of data between CPU and GPU fast enough, and coherently enough, that it stops being the limiting factor in large AI workloads. The Arm-based Neoverse V2 design gives Nvidia the power-efficiency and clean-sheet flexibility to build NVLink-C2C directly into the chip, something retrofitting onto an existing x86 design would be far harder to do. Whether evaluating a standalone Grace CPU Superchip for general compute or a GB200/GB300 Grace Blackwell system for AI training and inference, the architectural story is the same: CPU and GPU designed together as one coherent memory system, not bolted together after the fact.
FAQ
Is Grace a GPU or a CPU?
Grace is a CPU — specifically an Arm-based processor built on Arm Neoverse V2 cores. It’s a separate product from Nvidia’s GPUs (Hopper, Blackwell), though it’s frequently paired with them in “superchip” designs.
What’s the difference between the Grace CPU Superchip and Grace Hopper/Grace Blackwell?
The Grace CPU Superchip connects two Grace CPU dies together with no GPU involved, aimed at general-purpose server workloads. Grace Hopper and Grace Blackwell instead connect one Grace CPU to one or more GPUs (Hopper or Blackwell, respectively) over the same NVLink-C2C interconnect, aimed at AI training and inference.
Why does CPU-GPU memory coherence matter?
It lets a GPU effectively draw on CPU-attached memory without explicit, slower data copies across a narrower bus, which matters most for workloads — like large language models — whose memory footprint exceeds what fits in the GPU’s own onboard HBM memory.
How is Grace different from AWS Graviton or Google Axion?
All three are Arm-based server CPUs built for power efficiency, but Graviton and Axion are general-purpose cloud CPUs not designed around a coherent GPU interconnect. Grace is built specifically to pair with Nvidia’s own GPUs via NVLink-C2C, which is its core differentiator.
Do I need to recompile software to run on Grace?
Generally yes. Because Grace uses the Arm (aarch64) instruction set rather than x86, software needs to be compiled for Arm, or already have Arm-compatible builds available, to run natively.
