IBM’s broader Power11 and watsonx push covers enterprise AI generally. But IBM’s mainframe line has its own, much narrower AI story: a processor called Telum II, built into the z17 mainframe, that puts an AI accelerator on the same die as the processor cores running a bank’s core transaction workload. It’s a fundamentally different approach to AI acceleration than the GPU clusters behind most generative-AI infrastructure — and it exists for a specific reason: scoring transactions for fraud in real time, without sending data anywhere else.
At a Glance
| Item | Detail |
|---|---|
| Processor | Telum II — 8 cores, 5.5GHz, built on Samsung 5nm process |
| Mainframe | IBM z17 — announced April 8, 2025; generally available June 18, 2025 |
| On-chip AI accelerator | Up to 24 TOPS per chip; up to 192 TOPS per fully configured 8-chip drawer |
| Companion accelerator | IBM Spyre Accelerator — 32 AI cores per PCIe card, stackable |
| IBM’s claimed throughput | More than 450 billion inference operations/day (IBM-reported, ~50% more than z16) |
| IBM’s claimed latency | 1 millisecond response time (IBM-reported internal/extrapolated benchmark) |
What Telum II Actually Is
Telum II is IBM’s latest mainframe processor, introduced alongside the z17 system. IBM describes it as built on Samsung’s 5-nanometer process with eight high-performance cores running at 5.5GHz. Cache capacity grew roughly 40% over the prior generation, reaching a virtual L3 cache of 360MB and a virtual L4 cache of 2.88GB shared across the system — large on-chip caches that matter because mainframe transaction workloads are extremely cache-sensitive. Telum II also adds a dedicated data processing unit (DPU) to handle I/O acceleration, offloading that work from the main cores.
The On-Chip AI Accelerator
The part that matters here is Telum II’s integrated AI accelerator — a block of matrix-math hardware sitting on the same die as the processor cores, not a separate card reached over a bus. IBM says each accelerator’s compute capacity is roughly 4x higher than the previous Telum generation, reaching 24 trillion operations per second (TOPS), and added INT8 precision support, which IBM says improves efficiency and enables newer model types. In a fully populated processor drawer, any of the system’s cores can dispatch work to any of the drawer’s eight AI accelerators, which IBM says improves load balancing — and across all eight, IBM cites up to 192 TOPS available when fully configured.
Why This Is Different From GPU-Based AI Acceleration
A GPU cluster is a separate pool of compute that data has to travel to — over PCIe, NVLink, or a network fabric — before and after inference. That round trip adds latency and is usually fine for workloads like training large models or running chatbots, where milliseconds of network time don’t matter. It’s a poor fit for scoring a credit card swipe before the transaction is approved, where the system has a few milliseconds, total, to decide. IBM’s answer is to put a comparatively much smaller AI accelerator directly on the processor die, right next to the cores already executing the transaction. There’s no separate card, no PCIe hop, and no network call: the model runs in the same silicon, close to the same memory, as the business logic it’s scoring. IBM frames the payoff this way in its own benchmarking: more than 450 billion inference operations per day on z17, a claimed 50% increase over z16, with a 1 millisecond response time — figures IBM describes as extrapolated from internal tests using a synthetic credit-card fraud detection model, so they should be read as IBM’s own reported benchmark rather than an independently audited figure.
IBM Spyre: A Second, Complementary Accelerator
Not every AI workload fits on Telum II’s on-chip accelerator, which is sized for high-volume, low-latency scoring rather than large generative models. For that, IBM pairs the mainframe with the IBM Spyre Accelerator, a separate PCIe card containing 32 AI accelerator cores built on an architecture similar to Telum II’s on-chip engine. Multiple Spyre cards can be attached to a single z17 or LinuxONE system, and IBM positions it for generative-AI and larger-model workloads that complement, rather than replace, the real-time scoring done on-chip.
The Fraud Detection Use Case
IBM’s flagship example for Telum II is an ensemble approach to fraud and compliance. A traditional, relatively small neural network running on the on-chip accelerator gives an initial risk score for a transaction in real time; that score can then be combined with a large language model (typically run elsewhere, such as on Spyre) to improve accuracy without slowing down the initial decision. IBM describes applying the same ensemble pattern to detecting suspicious financial activity and supporting regulatory compliance work — cases where the first-pass model has to run inline with the transaction, and a heavier model can refine the result afterward.
On-Chip vs Spyre vs a Typical GPU Accelerator
| Accelerator | Location | Connects Via | Best For | Latency Profile |
|---|---|---|---|---|
| Telum II on-chip | Same die as CPU cores | Integrated — no external link | Small models, real-time transaction scoring | Sub-millisecond, in-line with transaction |
| IBM Spyre | Separate card | PCIe | Larger models, generative AI | Low, but higher than on-chip |
| Typical GPU accelerator | Separate card or node | PCIe or network fabric (NVLink/InfiniBand/Ethernet) | Training, large-scale inference | Higher — depends on network hop |
Trade-offs
- The on-chip accelerator’s 24 TOPS per chip is tiny next to a modern training GPU, but it isn’t trying to compete on raw throughput — it’s optimized to never leave the transaction path.
- IBM’s headline performance figures (450 billion inferences/day, 1ms latency) come from IBM’s own internal and extrapolated benchmarks, not third-party testing, and should be treated as vendor-reported.
- Running AI on a mainframe only makes sense for organizations already running core transaction workloads on IBM Z; it isn’t a general-purpose AI platform the way a GPU cloud is.
- Spyre adds generative-AI capacity, but it’s a separate card with different latency characteristics than the on-chip accelerator — the two aren’t interchangeable.
Bottom Line
Telum II and the z17 represent a narrower, more specific bet than GPU clusters: instead of moving data to a big pool of accelerators, IBM moved a small AI accelerator onto the same chip as the workload it needs to score. For something like real-time fraud detection, where the entire decision window is a few milliseconds, that architecture has a real structural advantage over any design that requires a PCIe or network hop — even though the raw compute numbers look small next to a data-center GPU.
FAQ
Is Telum II a GPU?
No. It’s a general-purpose mainframe CPU with an integrated AI accelerator block on the same die, designed for inline inference on structured transaction data rather than the large matrix workloads GPUs handle.
How fast is the z17’s AI accelerator really?
IBM states up to 24 TOPS per Telum II chip and up to 192 TOPS across a fully configured 8-chip processor drawer. For context, that’s a small fraction of a modern data-center GPU’s throughput — it’s built for latency on small models, not raw volume.
What’s the difference between Telum II’s accelerator and IBM Spyre?
Telum II’s accelerator is built into the processor die for the fastest possible, real-time inference on the transaction path. Spyre is a separate PCIe card with 32 AI cores aimed at larger generative-AI workloads that don’t need to run inline with a single transaction.
Are IBM’s performance claims independently verified?
No. IBM’s published figures — more than 450 billion inferences/day and 1ms latency — come from IBM’s own internal and extrapolated benchmarks using a synthetic fraud-detection model, and IBM itself notes results may vary.