Nvidia’s GPUs get most of the attention in the AI infrastructure conversation, but Google has been building and running its own AI chips since 2015 — and today they train and serve a meaningful share of the world’s largest AI workloads, including Google’s own Gemini models. Tensor Processing Units (TPUs) are purpose-built silicon that Google rents out through Google Cloud, and in 2026 the lineup looks very different from even a year ago. Here’s what a TPU actually is, what Google currently offers, what it costs, and how the trade-offs against Nvidia GPUs really break down.
At a Glance
| What it is | Google’s custom ASIC accelerators for AI training and inference, available only on Google Cloud |
| Current flagship generations | Ironwood (7th gen) and Trillium (6th gen) — both generally available |
| Coming soon | TPU 8t (large-scale pre-training) and TPU 8i (post-training and inference) |
| Core architecture | Systolic-array Matrix Multiplier Unit (MXU), not a general-purpose GPU core |
| On-demand price | $1.20/chip-hour (TPU v5e) up to $12.00–$13.20/chip-hour (Ironwood), region-dependent |
| Where you can get one | Google Cloud only — not available from other cloud providers or on-prem |
| Software support | JAX and OpenXLA natively; PyTorch via TorchTPU; vLLM and SGLang for inference |
Short Verdict
TPUs are a legitimate, fast-moving alternative to Nvidia GPUs for teams already building on Google Cloud or on JAX/PyTorch-XLA — especially at the largest training scales, where Google’s co-designed networking and software stack pay off. They are not a drop-in GPU replacement: you’re locked into Google Cloud, the software ecosystem is narrower than CUDA’s, and the newest chip (Ironwood) carries a steep per-chip-hour price. For most teams outside Google’s own ecosystem, Nvidia GPUs remain the path of least resistance; for teams running JAX-based research or large-scale inference on Google Cloud already, TPUs are worth serious evaluation.
What a TPU Actually Is
A TPU is an application-specific integrated circuit (ASIC) — silicon designed from the ground up for one job, rather than a general-purpose chip adapted to that job. Google designs TPUs specifically to accelerate the matrix multiplication and tensor operations that dominate neural network training and inference, and it co-designs the chip alongside the software stack that runs on it.
The core of every TPU is the Matrix Multiplier Unit (MXU), built around a systolic array: thousands of arithmetic logic units (ALUs) arranged so that data is read from memory once and then flows continuously through the array, with partial results accumulating as they pass from one ALU to the next. That’s fundamentally different from a GPU’s architecture, where thousands of general-purpose cores repeatedly read and write to memory as they execute many parallel threads (an approach called SIMT, or single-instruction, multiple-thread). The systolic-array design cuts down on memory traffic for the specific pattern of repeated matrix multiplications in a neural network, which is where TPUs get much of their efficiency advantage for that one workload — at the cost of being far less flexible for other kinds of computation.
TPUs also lean heavily on reduced-precision arithmetic (16-bit and 8-bit floating point formats) to push more computations through the same silicon without sacrificing the accuracy modern AI models need. And they’re designed from day one to work in large groups: a pod is a big cluster of TPU chips wired together over Google’s high-speed interconnect (described by Google as Optical Circuit Switching, branded “Virgo”), and a slice is a smaller, rentable chunk of a pod — which is what you actually provision on Google Cloud.
The Current Lineup: Ironwood and Trillium
As of 2026, Google lists two generally-available flagship TPU generations on its Cloud TPU product page:
| Generation | Status | What Google says it’s for | Headline spec Google cites |
|---|---|---|---|
| Ironwood (7th gen) | Generally available | Training, reasoning, and inference | 9,216 liquid-cooled chips per pod; 42.5 ExaFLOPS per pod; up to 4x the performance per chip versus Trillium |
| Trillium (6th gen) | Generally available | Training and inference | 4.7x higher peak compute per chip than TPU v5e; 67% more energy-efficient than v5e |
| TPU v5p / v5e | Still available, lower cost | General training (v5p) and cost-efficient training/inference (v5e) | Used by Google as the baseline Trillium is measured against |
| TPU 8t / TPU 8i | Coming soon (announced, not yet GA) | 8t: large-scale pre-training and embedding-heavy workloads. 8i: post-training and low-latency inference | 8t: 9,600 chips per superpod, ~3x compute per pod vs. the prior generation. 8i: Google claims an 80% performance-per-dollar improvement over previous generations for low-latency inference on large mixture-of-experts (MoE) models |
Google also says its Google Kubernetes Engine (GKE) can scale a single cluster up to 130,000 nodes, and that a cluster can span up to 1 million TPU chips — figures aimed squarely at the handful of organizations training frontier-scale models, not at a typical engineering team.
What It Costs
Cloud TPU pricing is quoted per chip-hour, though the Google Cloud console bills by VM-hour (a single TPU host can bundle multiple chips, so the displayed hourly rate is higher than the per-chip figure). On-demand, region-dependent prices from Google’s own pricing page:
| Generation | Representative on-demand price (per chip-hour) |
|---|---|
| Ironwood (v7) | $12.00 (us-central1) to $13.20 (europe-west2) |
| Trillium (v6e) | $2.70 (us-east1 / us-east5) to $3.24 (asia-northeast1) |
| TPU v5p | $4.20 (us-east1 / us-east5) |
| TPU v5e | $1.20 (most US regions) to $1.56 (Europe/Asia) |
Committed-use discounts, spot pricing, and reservations can all bring these numbers down substantially; the figures above are the standard on-demand rate. Note that these prices exist only on Google Cloud — unlike Nvidia GPUs, which you can rent from dozens of providers (AWS, Azure, Oracle, CoreWeave, Lambda, and many more) or buy outright, a TPU is a Google Cloud exclusive.
TPU vs. Nvidia GPU: The Honest Trade-Offs
Google doesn’t publish head-to-head TPU-vs-GPU benchmarks on its own product pages, and neither does Nvidia for the reverse comparison, so treat any specific “TPU is X% faster/cheaper than GPU Y” claim you see elsewhere with caution unless it cites a reproducible, apples-to-apples benchmark. What can be said reliably, based on how each platform is actually built and sold:
| Dimension | TPU | Nvidia GPU |
|---|---|---|
| Architecture | Purpose-built systolic array (MXU) for matrix multiplication | General-purpose SIMT cores, usable for AI and non-AI parallel workloads alike |
| Software ecosystem | Native JAX/OpenXLA; PyTorch via TorchTPU; narrower third-party tooling | CUDA — the most mature, widely supported accelerator software ecosystem in the industry |
| Where you can run it | Google Cloud only | Virtually every major cloud, specialized GPU clouds, and on-prem |
| Best-documented strength | Large, co-designed pods for frontier-scale training and inference within Google’s own stack | Breadth — the default choice for nearly any framework, model, or deployment target |
| Vendor lock-in | High — tied to Google Cloud and (mostly) to JAX/XLA-friendly workflows | Low to moderate — GPUs and CUDA code are portable across many providers |
Who Should Care
Teams already standardized on Google Cloud, or using JAX for research, are the clearest fit — Google’s own frontier models (including Gemini) are trained on TPUs, which is the strongest existing proof that the stack scales. Teams running PyTorch workloads that need broad framework and library compatibility, multi-cloud flexibility, or on-prem hardware are better served sticking with Nvidia GPUs, at least until TorchTPU support matures further. Cost-sensitive inference workloads on large MoE models are worth watching closely once TPU 8i reaches general availability, given Google’s stated performance-per-dollar target.
Bottom Line
TPUs are not a marketing afterthought — they’re a real, continuously-updated chip line that powers a meaningful share of the industry’s largest AI workloads, and the jump from v5e to Trillium to Ironwood represents genuine architectural progress. But “best chip” depends entirely on where you already live: inside Google Cloud with JAX-friendly code, TPUs are a strong and often cheaper-at-scale option; everywhere else, Nvidia’s ecosystem breadth is still the safer default.
FAQ
Can I buy a TPU and run it myself?
No. TPUs are not sold as hardware; they are only available as a rented Google Cloud service.
Does TPU support mean I have to rewrite my PyTorch code?
Not entirely — Google supports PyTorch on TPUs through TorchTPU, but JAX and OpenXLA remain the most natively supported and best-documented path.
Is Ironwood always the right choice over Trillium?
Not necessarily. Ironwood costs roughly 4–5x more per chip-hour than Trillium, so unless your workload needs Ironwood’s extra headroom, Trillium (or even v5e for lighter jobs) may be the more cost-effective choice.
Are TPU 8t and TPU 8i available now?
No — as of this writing, Google lists both as “coming soon” rather than generally available. Check Google’s Cloud TPU release notes for the current launch status before planning around them.
Sources
- Google Cloud — Cloud TPU product page
- Google Cloud — Cloud TPU pricing
- Google Cloud — Cloud TPU release notes
Discover more from Software Online Review
Subscribe to get the latest posts sent to your email.
