ibm – Software Online Review

IBM watsonx.ai and the Granite Model Family: IBM’s Open, Developer-Facing Approach to Enterprise AI in 2026

Posted on November 1, 2026November 1, 2026 by Filip Keser

IBM’s enterprise AI infrastructure gets covered for its silicon — the Power11 chip and the Telum II mainframe accelerator. But IBM is also running a separate, less hardware-centric play: an AI platform called watsonx.ai, paired with a family of openly licensed foundation models called Granite. Together they’re IBM’s answer to the question developers actually ask — not “whose chip is fastest,” but “can I build on this without getting locked into a single vendor’s weights.” Here’s what watsonx.ai and Granite actually offer as of early 2026, verified against IBM’s own documentation and model cards.

At a Glance

Platform	watsonx.ai (IBM’s AI development platform) + Granite (IBM’s open foundation model family)
Granite license	Apache 2.0 (per IBM’s Granite 4.2 release)
Latest Granite generation	Granite 4.2 (language models at 3B, 8B, and 30B parameters)
Model types in the family	Language, vision, speech, embedding, guardrail (“Guardian”), and time-series models
Where to run Granite	watsonx.ai, Hugging Face, Ollama, LM Studio, OpenRouter, Replicate, Weights & Biases, Unsloth, AnythingLLM, Red Hat Enterprise Linux AI
Reported SWE-Bench Verified (pass@1)	47.67 (8B), 57.00 (30B) — IBM’s own published figures
Best for	Enterprises and developers who want an open-weight model they can self-host, fine-tune, or audit, plus a managed platform to deploy it

Short Verdict

Granite is a genuinely open (Apache 2.0) model family that’s small enough to self-host cheaply and is distributed across nearly every popular runtime, which is unusual for a model backed by a large enterprise vendor. watsonx.ai is the more conventional piece: a managed platform for deploying Granite, Meta’s Llama, Mistral’s models, and others, with API-level controls aimed at regulated enterprises. Neither is the most capable option on raw benchmarks against today’s frontier models, but the combination — open weights plus an enterprise deployment layer — is IBM’s real differentiator.

What Is watsonx.ai?

watsonx.ai is IBM’s hosted platform for building, tuning, and deploying AI models, available on IBM Cloud and on-premises through watsonx as a Service. It isn’t a single model — it’s a catalog-and-tooling layer. Developers pick a foundation model from a growing list (IBM’s own Granite family alongside third-party models such as Meta’s Llama and Mistral’s releases), then use watsonx’s APIs, notebooks, and AutoAI tooling to prompt, tune, or build retrieval-augmented pipelines on top of it.

IBM’s own changelog shows a platform under active, fairly rapid iteration. Recent additions include a chat API with explicit controls over model reasoning depth and response detail (added November 2025), a preview “model gateway” that routes requests across multiple LLM providers through one resource hub (October 2025, Toronto region only at launch), a text-classification API for pre-sorting documents before extraction, and a dedicated watsonx Developer Hub with quick-start guides (November 2025). On the other side of the ledger, IBM has also been retiring older capabilities — prompt tuning was removed from all data centers in October 2025, and Federated Learning was removed in September 2025 — which is worth knowing if you’re building on a feature you saw in older IBM marketing material.

What Is the Granite Model Family?

Granite is IBM’s line of foundation models, released under the Apache 2.0 license — meaning the weights are downloadable and usable commercially without the restrictions some competing “open” models carry. The family spans more than text: IBM groups language, vision, speech, embedding, a safety/content-moderation line called Guardian, and time-series forecasting models under the Granite name, though not all of those sub-families have reached the same generation number.

The current flagship generation, Granite 4.2, ships language models at three sizes — 3B, 8B, and 30B parameters — small enough that the 8B and even the 30B variant can realistically run on a single modern GPU or a well-resourced CPU server, rather than requiring a data-center cluster. IBM publishes its own benchmark results for these three sizes:

Benchmark	Granite 4.2 3B	Granite 4.2 8B	Granite 4.2 30B
MMLU-Pro (5-shot)	67.84	74.04	77.60
GPQA (pass@1)	54.80	64.14	66.41
AIME25 (pass@1)	78.33	86.67	89.17
SWE-Bench Verified (pass@1)	N/A	47.67	57.00
Arena-Hard-V2 (win rate)	34.96	65.19	67.93

These are IBM’s self-reported figures from its own Granite documentation, not an independent third-party leaderboard, so treat them as a starting point rather than the final word — as with any vendor-published benchmark, it’s worth checking an independent leaderboard for the specific use case you care about before committing.

Where You Can Actually Use Granite

This is the part that distinguishes Granite from most enterprise-backed model families: IBM has pushed it onto nearly every popular runtime rather than keeping it inside watsonx. According to IBM’s own Granite page, the models are available through:

Hugging Face (the models are hosted directly under the ibm-granite organization)
Ollama and LM Studio, for running locally on a laptop or workstation
watsonx.ai, for IBM’s managed enterprise deployment path
OpenRouter and Replicate, for pay-per-call hosted inference
Weights & Biases and Unsloth, aimed at fine-tuning workflows
AnythingLLM, for retrieval-augmented chat apps
Red Hat Enterprise Linux AI, IBM’s own enterprise Linux distribution with built-in support for running Granite

On Hugging Face, the individual model pages back this up with real usage signals rather than marketing claims: granite-4.2-8b shows roughly 69,000 downloads and granite-4.2-30b around 36,000, with smaller but active download counts on the speech and switch-routing variants — evidence that developers outside IBM’s own customer base are actually pulling these weights.

Pros

Granite’s Apache 2.0 license is genuinely permissive — no field-of-use restrictions to parse before you ship a commercial product
Small model sizes (3B–30B) make self-hosting realistic without a large GPU cluster
Distributed across almost every popular open-model runtime, not locked into IBM’s own platform
watsonx.ai gives enterprises a managed, auditable deployment path if they don’t want to run inference infrastructure themselves
Dedicated Guardian models for content moderation/safety are a distinct, less common offering among open model families

Cons

Benchmark figures are IBM’s own; there’s no independent third-party confirmation cited on the official page
Granite’s largest published language model (30B) is well below the parameter count of today’s frontier proprietary models, so it’s not positioned to compete on raw general-purpose capability
watsonx.ai’s feature set is still being actively reshuffled — capabilities like prompt tuning and Federated Learning have already been removed within the last several months, so documentation and tutorials age quickly
Some Granite sub-families (vision, embedding) lag the language models in generation number and weren’t fully visible in IBM’s own published model list at the time of writing

Who It’s a Good Fit For

Granite suits teams that want to self-host or fine-tune an open-weight model without wading through restrictive licensing, and that don’t need frontier-level general capability — document processing, internal chat assistants, classification, and retrieval-augmented generation are reasonable fits at the 8B–30B range. watsonx.ai suits regulated enterprises (finance, healthcare, government) that want a single managed platform to deploy Granite alongside other models like Llama or Mistral, with IBM’s audit and governance tooling layered on top.

Who Should Look Elsewhere

If you need the strongest possible general-purpose reasoning or coding performance regardless of openness, today’s frontier proprietary models will generally outperform Granite’s current 30B ceiling. And if you have no interest in IBM’s platform tooling and just want an open model, you may find a more actively-benchmarked open-weight alternative with a larger community already built around it.

Bottom Line

IBM’s AI infrastructure story isn’t only about silicon. Granite is a legitimately open, portable model family that doesn’t try to compete at the frontier, and watsonx.ai is a maturing — if fast-changing — managed platform for enterprises that want governance and deployment tooling around it. If your priority is license freedom and runtime flexibility over raw benchmark supremacy, it’s worth evaluating; if you need the absolute top of the leaderboard, look elsewhere first.

FAQ

Is Granite free to use commercially?
Yes — IBM releases Granite models under the Apache 2.0 license, which permits commercial use without the field-of-use restrictions some other “open” model licenses carry.

Do I need watsonx.ai to run Granite?
No. Granite is also distributed through Hugging Face, Ollama, LM Studio, OpenRouter, Replicate, and other runtimes, so you can self-host it entirely outside IBM’s platform.

How big is the largest Granite 4.2 language model?
30 billion parameters, according to IBM’s own Granite 4.2 documentation, alongside 8B and 3B variants.

Is watsonx.ai only for Granite models?
No — IBM’s own changelog shows the platform also hosting Meta Llama and Mistral models for deployment, alongside Granite.

Sources

IBM’s Telum II and the z17: How IBM Built AI Acceleration Directly Into the Mainframe

Posted on October 31, 2026 by Filip Keser

IBM’s broader Power11 and watsonx push covers enterprise AI generally. But IBM’s mainframe line has its own, much narrower AI story: a processor called Telum II, built into the z17 mainframe, that puts an AI accelerator on the same die as the processor cores running a bank’s core transaction workload. It’s a fundamentally different approach to AI acceleration than the GPU clusters behind most generative-AI infrastructure — and it exists for a specific reason: scoring transactions for fraud in real time, without sending data anywhere else.

At a Glance

Item	Detail
Processor	Telum II — 8 cores, 5.5GHz, built on Samsung 5nm process
Mainframe	IBM z17 — announced April 8, 2025; generally available June 18, 2025
On-chip AI accelerator	Up to 24 TOPS per chip; up to 192 TOPS per fully configured 8-chip drawer
Companion accelerator	IBM Spyre Accelerator — 32 AI cores per PCIe card, stackable
IBM’s claimed throughput	More than 450 billion inference operations/day (IBM-reported, ~50% more than z16)
IBM’s claimed latency	1 millisecond response time (IBM-reported internal/extrapolated benchmark)

What Telum II Actually Is

Telum II is IBM’s latest mainframe processor, introduced alongside the z17 system. IBM describes it as built on Samsung’s 5-nanometer process with eight high-performance cores running at 5.5GHz. Cache capacity grew roughly 40% over the prior generation, reaching a virtual L3 cache of 360MB and a virtual L4 cache of 2.88GB shared across the system — large on-chip caches that matter because mainframe transaction workloads are extremely cache-sensitive. Telum II also adds a dedicated data processing unit (DPU) to handle I/O acceleration, offloading that work from the main cores.

The On-Chip AI Accelerator

The part that matters here is Telum II’s integrated AI accelerator — a block of matrix-math hardware sitting on the same die as the processor cores, not a separate card reached over a bus. IBM says each accelerator’s compute capacity is roughly 4x higher than the previous Telum generation, reaching 24 trillion operations per second (TOPS), and added INT8 precision support, which IBM says improves efficiency and enables newer model types. In a fully populated processor drawer, any of the system’s cores can dispatch work to any of the drawer’s eight AI accelerators, which IBM says improves load balancing — and across all eight, IBM cites up to 192 TOPS available when fully configured.

Why This Is Different From GPU-Based AI Acceleration

A GPU cluster is a separate pool of compute that data has to travel to — over PCIe, NVLink, or a network fabric — before and after inference. That round trip adds latency and is usually fine for workloads like training large models or running chatbots, where milliseconds of network time don’t matter. It’s a poor fit for scoring a credit card swipe before the transaction is approved, where the system has a few milliseconds, total, to decide. IBM’s answer is to put a comparatively much smaller AI accelerator directly on the processor die, right next to the cores already executing the transaction. There’s no separate card, no PCIe hop, and no network call: the model runs in the same silicon, close to the same memory, as the business logic it’s scoring. IBM frames the payoff this way in its own benchmarking: more than 450 billion inference operations per day on z17, a claimed 50% increase over z16, with a 1 millisecond response time — figures IBM describes as extrapolated from internal tests using a synthetic credit-card fraud detection model, so they should be read as IBM’s own reported benchmark rather than an independently audited figure.

IBM Spyre: A Second, Complementary Accelerator

Not every AI workload fits on Telum II’s on-chip accelerator, which is sized for high-volume, low-latency scoring rather than large generative models. For that, IBM pairs the mainframe with the IBM Spyre Accelerator, a separate PCIe card containing 32 AI accelerator cores built on an architecture similar to Telum II’s on-chip engine. Multiple Spyre cards can be attached to a single z17 or LinuxONE system, and IBM positions it for generative-AI and larger-model workloads that complement, rather than replace, the real-time scoring done on-chip.

The Fraud Detection Use Case

IBM’s flagship example for Telum II is an ensemble approach to fraud and compliance. A traditional, relatively small neural network running on the on-chip accelerator gives an initial risk score for a transaction in real time; that score can then be combined with a large language model (typically run elsewhere, such as on Spyre) to improve accuracy without slowing down the initial decision. IBM describes applying the same ensemble pattern to detecting suspicious financial activity and supporting regulatory compliance work — cases where the first-pass model has to run inline with the transaction, and a heavier model can refine the result afterward.

On-Chip vs Spyre vs a Typical GPU Accelerator

Accelerator	Location	Connects Via	Best For	Latency Profile
Telum II on-chip	Same die as CPU cores	Integrated — no external link	Small models, real-time transaction scoring	Sub-millisecond, in-line with transaction
IBM Spyre	Separate card	PCIe	Larger models, generative AI	Low, but higher than on-chip
Typical GPU accelerator	Separate card or node	PCIe or network fabric (NVLink/InfiniBand/Ethernet)	Training, large-scale inference	Higher — depends on network hop

Trade-offs

The on-chip accelerator’s 24 TOPS per chip is tiny next to a modern training GPU, but it isn’t trying to compete on raw throughput — it’s optimized to never leave the transaction path.
IBM’s headline performance figures (450 billion inferences/day, 1ms latency) come from IBM’s own internal and extrapolated benchmarks, not third-party testing, and should be treated as vendor-reported.
Running AI on a mainframe only makes sense for organizations already running core transaction workloads on IBM Z; it isn’t a general-purpose AI platform the way a GPU cloud is.
Spyre adds generative-AI capacity, but it’s a separate card with different latency characteristics than the on-chip accelerator — the two aren’t interchangeable.

Bottom Line

Telum II and the z17 represent a narrower, more specific bet than GPU clusters: instead of moving data to a big pool of accelerators, IBM moved a small AI accelerator onto the same chip as the workload it needs to score. For something like real-time fraud detection, where the entire decision window is a few milliseconds, that architecture has a real structural advantage over any design that requires a PCIe or network hop — even though the raw compute numbers look small next to a data-center GPU.

FAQ

Is Telum II a GPU?
No. It’s a general-purpose mainframe CPU with an integrated AI accelerator block on the same die, designed for inline inference on structured transaction data rather than the large matrix workloads GPUs handle.

How fast is the z17’s AI accelerator really?
IBM states up to 24 TOPS per Telum II chip and up to 192 TOPS across a fully configured 8-chip processor drawer. For context, that’s a small fraction of a modern data-center GPU’s throughput — it’s built for latency on small models, not raw volume.

What’s the difference between Telum II’s accelerator and IBM Spyre?
Telum II’s accelerator is built into the processor die for the fastest possible, real-time inference on the transaction path. Spyre is a separate PCIe card with 32 AI cores aimed at larger generative-AI workloads that don’t need to run inline with a single transaction.

Are IBM’s performance claims independently verified?
No. IBM’s published figures — more than 450 billion inferences/day and 1ms latency — come from IBM’s own internal and extrapolated benchmarks using a synthetic fraud-detection model, and IBM itself notes results may vary.

Sources

Artificial Intelligence, Cloud & Infrastructure

IBM Power11 and watsonx: What IBM’s Enterprise AI Infrastructure Actually Offers

Posted on October 30, 2026 by Filip Keser

While AWS, Google, and Microsoft fight over GPU supercluster bragging rights, IBM is running a different playbook: AI infrastructure built around its own Power chips, dedicated AI accelerator silicon, and a software layer (watsonx) aimed squarely at regulated enterprises that want AI running on-premises, not just in a public cloud. Here’s what’s actually in IBM’s stack.

At a Glance

Component	What it is	Status
Power11	IBM’s current server CPU family	Shipping since July 2025
Telum II	Mainframe processor with on-chip AI accelerator	Powers IBM z17 / LinuxONE 5
Spyre Accelerator	Dedicated AI accelerator card (32 cores)	Commercially available on Z/LinuxONE; rolling out to Power11
watsonx	IBM’s AI platform (ai / data / governance / Orchestrate)	Runs on Red Hat OpenShift AI, on-prem or IBM Cloud

Short Verdict

IBM isn’t trying to out-GPU Nvidia’s hyperscaler customers, and it shouldn’t be judged on that basis. Its bet is that a large slice of enterprise AI workloads — banking, insurance, government, telecom — need to run where the data already lives, under existing compliance controls, with hardware-level reliability guarantees that general-purpose cloud VMs don’t offer. Power11 plus the Telum II/Spyre accelerator lineup plus watsonx is a coherent answer to that specific problem. It is a weak answer if what you actually want is the cheapest possible path to training a frontier-scale foundation model.

What’s in IBM’s AI Infrastructure Stack

Power11 is IBM’s current-generation Power server CPU family, announced in July 2025 and the first time IBM simultaneously launched high-end, mid-range, and edge/low-end Power models together. The lineup spans the S1112 (edge/remote deployments with built-in AI acceleration), the S1122 and S1124, the E1150 (up to 120 Power11 cores, 16TB of DDR5 per 4U chassis), and the flagship E1180. IBM’s own materials don’t publish per-chip clock speeds or process node for Power11 — a genuine gap in IBM’s public documentation — but IBM does claim, in its own internal benchmarks, up to 55% faster core performance than Power9 on commercial workloads and up to 45% more processor resources than Power10 on its lower-end systems, along with roughly double the performance-per-watt of comparable x86 servers.

IBM’s AI-specific silicon sits one layer up. Telum II, built on a 5nm process and first detailed at Hot Chips in 2024, is the processor inside IBM’s z17 mainframe and LinuxONE 5 systems; it carries an on-chip AI accelerator and, in a fully configured system drawer, is rated for up to 192 TOPS of on-chip AI inference — aimed at running fraud-detection and risk models directly alongside transaction processing, with no round trip to a separate AI system. The Spyre Accelerator is IBM’s dedicated AI accelerator card: 32 AI cores on a 5nm, 75-watt PCIe card, now commercially available for IBM Z and LinuxONE systems (GA’d in late 2025) and extended to Power11 shortly after. Up to 48 Spyre cards can be clustered on Z/LinuxONE, and up to 16 on Power systems, for larger on-premises inference workloads.

watsonx: The Software Layer

watsonx is IBM’s umbrella AI platform, split into several named products: watsonx.ai (a studio for building and deploying custom AI applications, including IBM’s own Granite foundation models), watsonx.data (data integration and governance across hybrid and multicloud sources), watsonx.governance (AI risk management, regulatory compliance, and explainability tooling), and watsonx Orchestrate (building and deploying AI agents for business process automation).

Critically, watsonx is built on top of Red Hat OpenShift, and watsonx.ai specifically uses OpenShift AI’s accelerator support (which includes GPU kernel contributions from Nvidia, AMD, and Intel) and KServe for model serving. That means watsonx isn’t locked to IBM’s own chips — it can serve models like Llama 3 and Mistral alongside IBM’s own Granite models, and can be deployed fully air-gapped for workloads with strict data-isolation requirements. IBM has also confirmed that watsonx.data is being made available directly on Power11 and that Power11 integrates with Red Hat OpenShift AI.

Pros

On-chip AI acceleration (Telum II) built directly into transaction-processing hardware, aimed at real-time fraud/risk use cases banks and insurers actually run.
watsonx runs on open infrastructure (Red Hat OpenShift AI) rather than a proprietary stack, and supports third-party models alongside IBM’s own Granite models.
Strong hybrid/on-prem story: workloads can run fully air-gapped where data residency or regulatory rules require it.
Built-in quantum-safe encryption and a hardware-level ransomware detection feature (“Power Cyber Vault”) on Power11.

Cons

IBM does not publish basic chip specs (clock speed, process node, per-core detail) for Power11 the way competitors routinely do — third-party outlets fill in some of these numbers, but they aren’t confirmed directly by IBM.
Not positioned, and not competitive, for large-scale foundation-model pretraining at the scale AWS, Google, or Oracle’s GPU superclusters support.
The AI accelerator story is currently split across two different chip families (Telum II for mainframes, Spyre as an add-in card) rather than one unified roadmap, which adds complexity to evaluating the lineup.

Who It’s a Good Fit For

Banks, insurers, telcos, and government agencies that already run IBM Z, LinuxONE, or Power infrastructure and need AI inference (fraud detection, risk scoring, document processing) running close to transactional data under strict compliance and uptime requirements. Also a fit for enterprises that want a hybrid-cloud AI platform built on open infrastructure (OpenShift) rather than a single public cloud’s proprietary stack.

Who Should Look Elsewhere

Teams training large foundation models from scratch, or anyone who just wants the cheapest/fastest path to a large GPU cluster, should look at AWS, Google Cloud, Oracle, or Nvidia’s own cloud partners instead — IBM’s AI hardware story is about enterprise inference and governance, not frontier-scale training.

Bottom Line

IBM’s AI infrastructure play is narrower than the hyperscalers’ but more coherent for its target customer: enterprises that need AI inference running on the same hardware as their core transaction systems, under compliance regimes that make “just use a public cloud GPU instance” a non-starter. Power11, Telum II, Spyre, and watsonx add up to a real, differentiated hybrid-AI stack — just not a frontier-training one.

FAQ

Is watsonx only for IBM hardware?
No. watsonx runs on Red Hat OpenShift AI and can be deployed on IBM Power, IBM Cloud, other public clouds, or on-premises, and it can serve non-IBM models like Llama 3 and Mistral alongside IBM’s own Granite models.

What’s the difference between Telum II and Spyre?
Telum II is the main processor inside IBM’s z17 mainframe and LinuxONE 5 systems, with an AI accelerator built into the chip itself. Spyre is a separate, dedicated AI accelerator card that plugs into Z, LinuxONE, or Power systems for additional AI inference capacity.

Can IBM’s infrastructure train large language models from scratch?
It’s not positioned or marketed for that. IBM’s AI hardware emphasizes on-premises inference and governance for enterprise workloads; large-scale pretraining is better suited to GPU-cluster-focused clouds like AWS, Google Cloud, or Oracle.