AWS has spent close to a decade building its own AI silicon, and by late 2025 that bet reached a new generation: Trainium3, paired with Trainium3 UltraServers that AWS says scale to hundreds of thousands of chips. The pitch is not that Trainium beats Nvidia’s GPUs on raw throughput — AWS doesn’t make that claim on its own product pages — but that it beats them on cost per token, delivered as part of a vertically integrated stack AWS fully controls, from the Graviton host CPUs and Nitro hypervisor to the Elastic Fabric Adapter (EFA) networking and the Neuron software stack. Here’s what’s verifiably true about Trainium3’s architecture, how it stacks up against Nvidia’s current flagship data center GPU platform, and where the comparison gets murkier.
At a Glance
| Spec (per chip/GPU) | AWS Trainium3 | Nvidia Blackwell (GB200 NVL72, per GPU) |
|---|---|---|
| Peak compute | ~2.52 PFLOPS MXFP8 (derived from UltraServer total) | ~10 PFLOPS FP8 sparse (derived from rack total) |
| HBM capacity | 144 GB HBM3e | ~186 GB HBM3e |
| HBM bandwidth | 4.9 TB/s | ~8 TB/s |
| Chip-to-chip interconnect | NeuronLink / NeuronSwitch (fabric-level, UltraServer-wide) | 5th-gen NVLink, 1.8 TB/s GPU-to-GPU |
| Scale-out networking | EFA, up to 28.8 Tbps per UltraServer | InfiniBand/NVLink domain, 130 TB/s rack-wide NVLink |
| Software stack | AWS Neuron SDK; claims unmodified PyTorch, vLLM, HuggingFace, Ray support | CUDA, cuDNN, Transformer Engine, Magnum IO |
Note: AWS publishes Trainium3 figures per chip and per 144-chip UltraServer directly. Nvidia publishes GB200 NVL72 figures at the rack level (72 GPUs); the per-GPU numbers above are derived by dividing rack totals and should be read as approximate, not as Nvidia’s own stated per-chip spec.
What Trainium3 Actually Is
According to AWS’s own Trainium product page, Trainium3 is built around eight large NeuronCores per chip, each combining Tensor, Vector, Scalar, and GPSIMD engines that run concurrently. AWS claims up to 2x MXFP8 compute throughput over Trainium2, plus hardware-level support for Mixture-of-Experts (MoE) routing — relevant given how many frontier models now use MoE architectures. Each chip carries 144 GB of HBM3e at 4.9 TB/s of bandwidth, which AWS describes as 1.7x the bandwidth of Trainium2, and supports hardware-accelerated W4A8 quantization that AWS says doubles effective weight-loading speed. A two-level on-chip SRAM hierarchy sits between the HBM and the compute engines to reduce memory-access latency.
Chips are grouped into Trainium3 UltraServers of up to 144 chips, which AWS rates at 362 PFLOPS of MXFP8 compute, 20.7 TB of pooled HBM3e, and 706 TB/s of aggregate memory bandwidth. Inside the UltraServer, a NeuronSwitch all-to-all fabric doubles interchip bandwidth versus Trainium2 UltraServers, with redundant NeuronLink lanes for chip-level fault tolerance. For scaling beyond a single UltraServer, EFA networking provides up to 28.8 Tbps of aggregate scale-out bandwidth, and AWS’s “UltraClusters 3.0” architecture is designed to connect hundreds of thousands of chips.
What Nvidia’s Blackwell Platform Brings to the Comparison
Nvidia’s GB200 NVL72 — 72 Blackwell GPUs and 36 Grace CPUs in a single rack — is the current reference point for large-scale GPU training, per Nvidia’s own product page. Nvidia publishes the rack as a single 72-GPU NVLink domain: 13.4 TB of pooled HBM3e, 576 TB/s of aggregate memory bandwidth, and a 130 TB/s NVLink domain, with fifth-generation NVLink providing 1.8 TB/s of GPU-to-GPU bandwidth. Nvidia’s own materials claim roughly 4x faster LLM training than the prior H100 generation on a 1.8-trillion-parameter MoE benchmark — a figure the company’s own footnote flags as “projected and subject to change” — plus up to 30x faster real-time trillion-parameter LLM inference and roughly 25x better energy efficiency versus H100 air-cooled infrastructure. Nvidia’s pitch is the inverse of AWS’s: a single, enormous, CUDA-programmable compute domain built for both bleeding-edge training performance and the broadest possible software compatibility.
The Trade-Offs: Cost, Risk, and Ecosystem
AWS’s central argument for Trainium is economics, not peak performance. Amazon has publicly claimed customers see roughly 50% lower training and inference costs compared with “GPU alternatives” — a framing that does not name Nvidia specifically and is based on AWS’s own customer case studies (cited examples include Karakuri, Metagenomi, NetoAI, Ricoh, and Splash Music), not an independently verified benchmark. Anthropic is the most prominent named customer running production workloads on Trainium, and Amazon Bedrock itself serves production traffic from Trainium3 infrastructure.
The catch is ecosystem maturity. Moving a training or inference workload to Trainium means going through AWS’s Neuron SDK, and performance-critical code sometimes needs to be rewritten or re-tuned for it, whereas CUDA remains the default, far more battle-tested target for nearly every published model architecture, research paper, and open-source framework. That’s a real switching cost that AWS’s own page addresses indirectly by emphasizing “unmodified” support for PyTorch, vLLM, HuggingFace, and Ray — a claim about framework-level compatibility, not a guarantee of matching CUDA-level performance out of the box. For teams doing novel model R&D where every detail of kernel behavior matters, that relative novelty is a real risk; for teams running well-understood training or inference workloads at large, predictable scale, it’s a more acceptable trade for lower cost.
It’s also worth noting AWS and Nvidia aren’t purely adversarial here. AWS has signaled that its next generation, Trainium4 (expected late 2026 or early 2027), will add support for Nvidia’s NVLink Fusion interconnect alongside the UALink open standard, which would let AWS build heterogeneous clusters mixing its own silicon with Nvidia GPU resources rather than treating the two as strictly separate options. AWS is also reported to remain one of Nvidia’s largest cloud customers, buying GPU capacity even as it scales up its own chips.
Bottom Line
Trainium3 is a credible, rapidly maturing alternative to GPUs for specific, well-understood training and inference workloads at AWS scale, built on genuinely competitive memory bandwidth and a purpose-built MoE-aware architecture — but it competes primarily on cost and vertical integration, not on raw per-chip compute, where Nvidia’s Blackwell platform still appears to hold a lead based on each company’s own published figures. The practical decision for most teams isn’t “Trainium or Nvidia” in the abstract; it’s whether a given workload’s tolerance for Neuron SDK porting work is outweighed by the cost savings AWS advertises, and that calculus will differ sharply between a stable production inference pipeline and a team pushing the frontier of new model architectures.
FAQ
Is Trainium3 faster than Nvidia’s Blackwell GPUs?
Based on each company’s own published figures, Blackwell’s GB200 NVL72 appears to have higher peak per-GPU compute and memory bandwidth than Trainium3. AWS does not claim to beat Nvidia on raw throughput; its public claims focus on cost per token instead.
What is a Trainium3 UltraServer?
It’s AWS’s term for a tightly interconnected cluster of up to 144 Trainium3 chips, sharing 20.7 TB of pooled HBM3e memory and connected by a NeuronSwitch fabric, rated by AWS at 362 PFLOPS of MXFP8 compute.
Do I need to rewrite my code to use Trainium?
AWS states that PyTorch, vLLM, HuggingFace, and Ray run on Trainium “without modification” at the framework level, but workloads optimized specifically for CUDA kernels may still need re-tuning through the Neuron SDK to get comparable performance.
Which major AI companies use Trainium in production?
Anthropic is the most prominent named customer running production workloads on Trainium, per AWS and third-party reporting; Amazon Bedrock also runs production inference traffic on Trainium3.
Will AWS keep buying Nvidia GPUs even as it builds Trainium?
All available reporting suggests yes — AWS continues to offer Nvidia GPU instances alongside Trainium, and Trainium4 is expected to add support for Nvidia’s NVLink Fusion interconnect, pointing toward coexistence rather than full replacement.
