Posted in Artificial Intelligence, Cloud & Infrastructure

IBM Telum II Explained: The On-Chip AI Accelerator Inside IBM’s z17 Mainframes

While most of the AI infrastructure conversation is about GPU clusters in hyperscaler data centers, IBM has been building AI acceleration directly into the processor that still runs the world’s core banking, insurance, and airline reservation transactions. The Telum II chip, announced in August 2024 and shipping inside the IBM z17 mainframe launched in April 2025, puts an AI accelerator on the same silicon as the CPU cores — letting a mainframe score a transaction for fraud in the same sub-millisecond window it takes to process it.

At a Glance

SpecTelum II
Cores8 high-performance cores per chip
Clock speed5.5 GHz
Process node5nm, manufactured by Samsung Foundry
On-chip cache36 MB L2 per core; virtual L4 cache of 2.88 GB per processor drawer (40% more than prior generation)
On-chip AI acceleratorIntegrated AI accelerator for in-transaction inferencing; 4x compute capacity per chip vs. first-generation Telum
Companion chipIBM Spyre Accelerator (PCIe card, tech preview at launch)
Ships inIBM z17 mainframe (announced/launched April 8, 2025)

Short Verdict

Telum II isn’t trying to compete with an Nvidia GPU cluster on raw throughput — it’s solving a completely different problem: running AI inference inside the same low-latency transaction path as the mainframe’s core workload, with no network hop to a separate inference server. For banks and insurers that need to score every single card swipe or claim for fraud without adding latency to the transaction itself, that architecture is the entire point.

What Changed From Telum I

IBM’s own announcement frames Telum II’s improvements almost entirely as multipliers over the original Telum (which shipped in the z16), rather than absolute specs for the predecessor: 40% more on-chip and virtual L4 cache, and up to 4x the AI compute capacity per chip. The chip keeps the architecture’s signature move — an AI accelerator built into the processor itself rather than bolted on as a separate card — and adds a coherently attached Data Processing Unit (DPU) that IBM says delivers 50% higher I/O density, aimed at keeping data moving fast enough to feed the AI accelerator without the chip’s compute cores becoming the bottleneck.

IBM’s benchmark claim for z17’s AI performance — up to 5 million inference operations per second at under 1 millisecond response time — comes from an internal test using a synthetic LSTM-based credit card fraud detection model at a batch size of 160, run against the Integrated Accelerator for AI. IBM itself notes results may vary, so treat it as a vendor-reported benchmark under specific test conditions, not a universal throughput figure.

The Spyre Accelerator: A Separate Piece

Telum II’s on-chip accelerator handles low-latency, in-transaction inference. For larger or more general AI workloads — like applying an LLM to mainframe transaction data — IBM pairs it with the Spyre Accelerator, a PCIe card that was announced alongside Telum II as a technology preview. Spyre specs: up to 1 TB of memory, 32 compute cores per chip supporting int8 and fp16 datatypes, a 75 W power envelope per card, and support for up to eight cards per I/O drawer. Spyre was expected to become generally available in 2025, separate from the on-chip accelerator built into every Telum II.

Why This Matters for AI Infrastructure

Most “AI infrastructure” coverage is about training ever-larger models on GPU clusters. Telum II represents the opposite end of the stack: inference embedded directly into a transaction-processing system that institutions already run their most latency- and compliance-sensitive workloads on. A bank doesn’t want to ship a credit card authorization request to an external inference service and wait for a round trip — it wants the fraud score computed inline, on the same chip, before the transaction completes. That’s a narrower use case than general-purpose AI acceleration, but for IBM Z’s installed base of banks, insurers, and governments, it’s the more relevant one.

Pros and Cons

  • Pros: Sub-millisecond in-transaction inference with no network hop; large cache hierarchy reduces memory-bound bottlenecks; Spyre extends the same platform to larger AI models
  • Cons: Only relevant to organizations already running IBM Z mainframes — it’s not a general-purpose AI accelerator you can buy standalone; IBM’s own performance figures are internal, synthetic benchmarks rather than independently verified numbers; Spyre launched as a technology preview, meaning full production availability trailed the chip announcement

Who This Is For

Banks, insurers, payment processors, and governments already operating IBM Z infrastructure who need real-time AI inference (fraud detection, risk scoring) embedded directly in their highest-volume transaction paths, without re-architecting around an external inference service.

Bottom Line

Telum II is a reminder that “AI chip” doesn’t mean one thing — IBM is optimizing for inference latency inside an existing transaction-processing architecture, not for training throughput or raw inference volume. It’s a compelling design for the specific institutions that run IBM Z, and largely irrelevant to everyone else’s AI infrastructure decisions.

FAQ

Is Telum II the same as the Spyre Accelerator?
No. Telum II is the mainframe’s main processor with an AI accelerator built into the same chip. Spyre is a separate PCIe accelerator card designed for larger AI workloads, used alongside Telum II rather than replacing it.

Can you buy a Telum II chip separately from a z17 mainframe?
No — it’s built into the z17 system across all three of its form factors (multi-frame, single-frame, and rack-mount configurations).

How does Telum II compare to a GPU for AI workloads?
They’re not really comparable on the same axis. GPUs like Nvidia’s are built for massively parallel training and batch inference throughput; Telum II is built for extremely low-latency inference embedded in a transaction-processing pipeline, at a much smaller scale per chip.

Sources

Unknown's avatar

Author:

Innovations, logic & design thinking