While most of the AI infrastructure conversation is about GPU clusters in hyperscaler data centers, IBM has been building AI acceleration directly into the processor that still runs the world’s core banking, insurance, and airline reservation transactions. The Telum II chip, announced in August 2024 and shipping inside the IBM z17 mainframe launched in April 2025, puts an AI accelerator on the same silicon as the CPU cores — letting a mainframe score a transaction for fraud in the same sub-millisecond window it takes to process it.
At a Glance
| Spec | Telum II |
|---|---|
| Cores | 8 high-performance cores per chip |
| Clock speed | 5.5 GHz |
| Process node | 5nm, manufactured by Samsung Foundry |
| On-chip cache | 36 MB L2 per core; virtual L4 cache of 2.88 GB per processor drawer (40% more than prior generation) |
| On-chip AI accelerator | Integrated AI accelerator for in-transaction inferencing; 4x compute capacity per chip vs. first-generation Telum |
| Companion chip | IBM Spyre Accelerator (PCIe card, tech preview at launch) |
| Ships in | IBM z17 mainframe (announced/launched April 8, 2025) |
Short Verdict
Telum II isn’t trying to compete with an Nvidia GPU cluster on raw throughput — it’s solving a completely different problem: running AI inference inside the same low-latency transaction path as the mainframe’s core workload, with no network hop to a separate inference server. For banks and insurers that need to score every single card swipe or claim for fraud without adding latency to the transaction itself, that architecture is the entire point.
What Changed From Telum I
IBM’s own announcement frames Telum II’s improvements almost entirely as multipliers over the original Telum (which shipped in the z16), rather than absolute specs for the predecessor: 40% more on-chip and virtual L4 cache, and up to 4x the AI compute capacity per chip. The chip keeps the architecture’s signature move — an AI accelerator built into the processor itself rather than bolted on as a separate card — and adds a coherently attached Data Processing Unit (DPU) that IBM says delivers 50% higher I/O density, aimed at keeping data moving fast enough to feed the AI accelerator without the chip’s compute cores becoming the bottleneck.
IBM’s benchmark claim for z17’s AI performance — up to 5 million inference operations per second at under 1 millisecond response time — comes from an internal test using a synthetic LSTM-based credit card fraud detection model at a batch size of 160, run against the Integrated Accelerator for AI. IBM itself notes results may vary, so treat it as a vendor-reported benchmark under specific test conditions, not a universal throughput figure.
The Spyre Accelerator: A Separate Piece
Telum II’s on-chip accelerator handles low-latency, in-transaction inference. For larger or more general AI workloads — like applying an LLM to mainframe transaction data — IBM pairs it with the Spyre Accelerator, a PCIe card that was announced alongside Telum II as a technology preview. Spyre specs: up to 1 TB of memory, 32 compute cores per chip supporting int8 and fp16 datatypes, a 75 W power envelope per card, and support for up to eight cards per I/O drawer. Spyre was expected to become generally available in 2025, separate from the on-chip accelerator built into every Telum II.
Why This Matters for AI Infrastructure
Most “AI infrastructure” coverage is about training ever-larger models on GPU clusters. Telum II represents the opposite end of the stack: inference embedded directly into a transaction-processing system that institutions already run their most latency- and compliance-sensitive workloads on. A bank doesn’t want to ship a credit card authorization request to an external inference service and wait for a round trip — it wants the fraud score computed inline, on the same chip, before the transaction completes. That’s a narrower use case than general-purpose AI acceleration, but for IBM Z’s installed base of banks, insurers, and governments, it’s the more relevant one.
Pros and Cons
- Pros: Sub-millisecond in-transaction inference with no network hop; large cache hierarchy reduces memory-bound bottlenecks; Spyre extends the same platform to larger AI models
- Cons: Only relevant to organizations already running IBM Z mainframes — it’s not a general-purpose AI accelerator you can buy standalone; IBM’s own performance figures are internal, synthetic benchmarks rather than independently verified numbers; Spyre launched as a technology preview, meaning full production availability trailed the chip announcement
Who This Is For
Banks, insurers, payment processors, and governments already operating IBM Z infrastructure who need real-time AI inference (fraud detection, risk scoring) embedded directly in their highest-volume transaction paths, without re-architecting around an external inference service.
Bottom Line
Telum II is a reminder that “AI chip” doesn’t mean one thing — IBM is optimizing for inference latency inside an existing transaction-processing architecture, not for training throughput or raw inference volume. It’s a compelling design for the specific institutions that run IBM Z, and largely irrelevant to everyone else’s AI infrastructure decisions.
FAQ
Is Telum II the same as the Spyre Accelerator?
No. Telum II is the mainframe’s main processor with an AI accelerator built into the same chip. Spyre is a separate PCIe accelerator card designed for larger AI workloads, used alongside Telum II rather than replacing it.
Can you buy a Telum II chip separately from a z17 mainframe?
No — it’s built into the z17 system across all three of its form factors (multi-frame, single-frame, and rack-mount configurations).
How does Telum II compare to a GPU for AI workloads?
They’re not really comparable on the same axis. GPUs like Nvidia’s are built for massively parallel training and batch inference throughput; Telum II is built for extremely low-latency inference embedded in a transaction-processing pipeline, at a much smaller scale per chip.
