Posted in Artificial Intelligence

Best LLM Observability and Evaluation Platforms in 2026: LangSmith vs Langfuse vs Arize

Posted on by Filip Keser

If you are running LLM applications in production, you have probably hit the same wall every team hits: it works great in a notebook, and then it goes sideways in the wild and you have no idea why. Was it the prompt? The retrieved context? A model regression? A flaky tool call three steps into an agent trajectory? LLM observability and evaluation platforms exist to answer that question by capturing every call, every chain, and every score so you can debug, test, and monitor like you would any other production system. This guide compares the three platforms developers reach for most in 2026 — LangSmith, Langfuse, and Arize — on pricing, features, and open-source status, using only what each vendor states on its own site as of November 2026.

At a Glance

ToolStarting PriceOpen SourceBest For
LangSmithFree (Developer plan), then $39/seat/mo on Plus plus usageNo (closed-source product; self-hosted deployment available on Enterprise)Teams already building with LangChain or LangGraph who want tracing, evals, and prompt tooling in one place
LangfuseFree (Hobby plan, 50k units/mo), $29/mo Core, $199/mo ProYes — MIT licensed core, self-hostable with no feature gatesTeams that want a fully open-source, self-hostable observability stack with no vendor lock-in
Arize (AX)Free (AX Free, 25k spans/mo), $50/mo AX ProPartially — the underlying Phoenix project is source-available (Elastic License 2.0), free to self-host; Arize AX itself is a closed commercial SaaS/self-hosted productTeams that want enterprise-grade ML + LLM observability with deep evaluation tooling and an open-source on-ramp via Phoenix

Short Verdict

There isn’t a single winner here — the right pick depends on what you’re already using and how much you care about owning your data. If your stack is built on LangChain or LangGraph, LangSmith’s tracing and evaluation tooling is the path of least resistance, and its free Developer tier covers small projects. If you want an observability layer that is genuinely open source end to end, with no asterisks on the core feature set, Langfuse’s MIT-licensed self-hosted option is the standout — you can run it yourself with unlimited volume and never touch a credit card. If you need the most mature evaluation and monitoring tooling for mixed ML/LLM workloads, or you want a free, no-feature-gated local tracing tool to start with before deciding on a vendor, Arize’s combination of AX (hosted) and Phoenix (self-hosted, source-available) is worth a serious look.

What They Are

LangSmith

LangSmith is LangChain’s observability and evaluation platform, built to trace LLM calls and agent trajectories, run online and offline evaluations, manage prompts, and monitor production usage. It works with LangChain and LangGraph natively but also accepts traces from any framework via OpenTelemetry or its SDKs (Python, TypeScript, Go, Java). Its feature set spans step-by-step tracing with message threading for multi-turn chats, real-time dashboards with cost and latency tracking, LLM-as-judge and code-based online/offline evaluators, dataset and experiment management, and automatic clustering of traces to surface failure patterns. LangSmith is not open source, but LangChain offers managed cloud, bring-your-own-cloud, and self-hosted deployment options for it, and a purpose-built trace database called SmithDB can also be self-hosted inside your own VPC.

Langfuse

Langfuse is an open-source LLM engineering platform covering tracing, prompt management, evaluation, and datasets. Its core is MIT licensed, and according to Langfuse’s own open-source documentation, everything outside the project’s `/ee` (Enterprise Edition) folders — including tracing, evaluations, prompt management, experiments, annotation, and the playground — is MIT licensed and self-hostable with no usage limits. The Enterprise Edition modules (SCIM, extended audit logging, data retention policies) ship as source code too, but require a commercial license key to activate; they’re bundled automatically into Langfuse Cloud. You can self-host via Docker Compose, Kubernetes (Helm charts), or Terraform on AWS, GCP, and Azure, or use the hosted Langfuse Cloud for a managed experience.

Arize (AX and Phoenix)

Arize’s structure is two-layered. Phoenix is Arize’s open(-ish) source tracing, evaluation, experimentation, and prompt-iteration tool — Arize describes it as “fully open-source, free to self-host” with no feature gates, though it’s released under the Elastic License 2.0 (ELv2), a source-available license that is not OSI-approved open source (it restricts, for example, offering Phoenix itself as a competing hosted service). Arize AX is the commercial product built on top: a SaaS (or self-hosted, on Enterprise) platform for OpenTelemetry-compliant tracing, cost/latency/token tracking, multimodal tracing (image, voice, PDF), online/offline evaluations, and unlimited datasets, experiments, and prompts across every paid and free tier.

Pricing and Licensing Breakdown

PlatformFree TierEntry Paid TierMid TierEnterprise
LangSmithDeveloper: $0/seat, up to 5,000 base traces/mo included (pay-as-you-go beyond that), 14-day base trace retention, 1 seatPlus: $39/seat/mo, up to 10,000 base traces/mo included then pay-as-you-go, unlimited seats at $39 each, access to Agent Server/Engine— (Plus scales with usage)Custom pricing, self-hosted/hybrid deployment, custom SSO, support SLA
LangfuseHobby: free, no credit card, 50,000 units/mo, 30-day data access, 2 users, 1,000 req/min ingestionCore: $29/mo, 100,000 units/mo, 90-day data access, unlimited users, 4,000 req/minPro: $199/mo, 100,000 units/mo, 3-year data access, 20,000 req/min (optional Teams add-on $300/mo for SSO/RBAC)Enterprise: $2,499/mo, 100,000 units/mo, 3-year retention, custom ingestion. Self-hosted core is free and unlimited under MIT.
Arize AXAX Free: $0/mo, 25,000 trace spans/mo, 1 GB ingestion, 15-day retention, unlimited users/evals/datasets/promptsAX Pro: $50/mo, 50,000 trace spans/mo, 10 GB ingestion, 30-day retention—AX Enterprise: custom pricing, unlimited spans, custom retention, SaaS or self-hosted. Phoenix (OSS core) is free to self-host under ELv2 with no volume limits.

A few things worth flagging precisely rather than guessing at: LangSmith’s pricing page does not define what counts as a “unit” the way Langfuse does, and it doesn’t publish a price for upgrading base traces to extended (180-day) retention — that’s quoted case by case. Langfuse’s pricing page similarly does not break “units” down into separate trace/event/observation counts; it’s a single blended ingestion allowance per tier. Arize’s page states span and ingestion limits but does not publish a separate “event” limit, so span volume is the most direct apples-to-apples number available. Where a vendor’s own page didn’t give a number, we left it out rather than estimate.

Pros and Cons

LangSmith

  • Deepest native integration with LangChain and LangGraph agent tracing
  • Online and offline evaluators, LLM-as-judge, datasets with automatic versioning, and a playground in one product
  • Flexible deployment: managed cloud, BYOC, or self-hosted (Enterprise)
  • Not open source — you can’t inspect or modify the core product
  • Free tier is capped at a modest 5,000 base traces/month with pay-as-you-go overage beyond that
  • Full feature set and self-hosting both sit behind paid or Enterprise tiers

Langfuse

  • MIT-licensed core with no feature gates on self-hosted tracing, evals, prompts, datasets, or the playground
  • Generous free Hobby cloud tier (50k units/month) with no credit card required
  • Framework-agnostic: SDKs, OpenTelemetry, and API ingestion work regardless of what you build with
  • Self-hosting still requires you to run and maintain the infrastructure (Postgres, ClickHouse, Redis, etc.)
  • Enterprise-only features (SCIM, extended audit logs, retention policies) require a commercial license even if self-hosting
  • Cloud tiers jump sharply in price from Core ($29) to Pro ($199) to Enterprise ($2,499)

Arize (AX / Phoenix)

  • Phoenix gives you a genuinely free, locally runnable tracing and eval tool with no feature gates to start
  • Unlimited users, evaluations, datasets, experiments, and prompts on every AX tier, including Free
  • Strong roots in ML observability mean mature evaluation and drift-style tooling, not just LLM tracing bolted on
  • Phoenix’s Elastic License 2.0 is source-available, not an OSI-approved open-source license — read the terms before assuming “open source” means what it does for Langfuse
  • AX Free and Pro tiers are capped at relatively low span volumes (25k and 50k/month) with short retention (15 and 30 days)
  • Custom code evaluators are unavailable on Free and Pro tiers per Arize’s own pricing page

Who It’s a Good Fit For

LangSmith fits teams already standardized on LangChain or LangGraph who want tracing, evals, and prompt management without integrating a separate vendor, and who are comfortable paying for traces beyond a small free allowance.

Langfuse fits teams that want full control and the option to self-host without ever hitting a paywall on core functionality — startups cost-conscious about trace volume, regulated teams that need data to stay in-house, and anyone framework-agnostic across LangChain, LlamaIndex, raw API calls, or custom agents.

Arize fits teams that already run ML observability workloads and want LLM tracing in the same place, or anyone who wants to try Phoenix locally for free before committing to a hosted evaluation platform.

Who Should Look Elsewhere

If you need the entire stack — including every enterprise feature — to be free and open source with no commercial license anywhere in the picture, Langfuse’s EE-gated modules and Arize’s ELv2-licensed Phoenix both mean reading the fine print before you build critical infrastructure on the assumption everything is unrestricted. If you need very high trace volumes on a tight budget, compare the per-unit overage costs carefully: Langfuse publishes an $8-per-100k-units overage rate, while LangSmith and Arize price additional volume on request or via a usage calculator.

Code Example: Minimal LangSmith Tracing

Enabling LangSmith tracing for any LangChain run is largely a matter of setting environment variables — no code changes to your chain are required:

import os

os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_API_KEY"] = "ls_your_key_here"
os.environ["LANGSMITH_PROJECT"] = "my-rag-app"

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")
response = llm.invoke("Summarize the benefits of LLM tracing in two sentences.")
print(response.content)

The Bottom Line

All three platforms solve the same core problem — making opaque LLM behavior inspectable and testable — but they draw the line between “free,” “open,” and “enterprise” in different places. LangSmith trades openness for the tightest LangChain/LangGraph integration. Langfuse trades a steeper jump between paid cloud tiers for a core product you can run yourself forever under MIT. Arize trades a source-available (not fully open-source) license on Phoenix for one of the more mature evaluation toolsets, especially if you’re coming from an ML observability background. Start by asking which constraint matters most to your team — framework lock-in, data residency, or evaluation depth — and let that decide, rather than picking on name recognition alone.

FAQ

Is Langfuse really free to self-host with no limits?

According to Langfuse’s own open-source documentation, yes for the MIT-licensed core: tracing, evaluations, prompt management, experiments, annotation, and the playground are included with no usage caps when you run it yourself. Only the separate Enterprise Edition modules (SCIM, extended audit logging, data retention policies) require a commercial license key, and those are optional add-ons, not part of the base product.

Is Arize Phoenix open source?

Arize markets Phoenix as “fully open-source, free to self-host,” and it is free with no feature gates, but it ships under the Elastic License 2.0, which is a source-available license rather than an OSI-approved open-source license. In practice that mostly affects whether a third party can resell Phoenix as a competing hosted service — it doesn’t change your ability to self-host and use every feature for free.

Can I use LangSmith without using LangChain or LangGraph?

Yes. LangSmith accepts traces from any framework through its SDKs (Python, TypeScript, Go, Java) or via standard OpenTelemetry, though its deepest integrations — automatic tracing of chains, graphs, and tool calls — are built around LangChain and LangGraph.

What exactly counts toward a free-tier limit on each platform?

This differs by vendor and the vendors are not fully consistent in their terminology. LangSmith counts “base traces” (5,000/month free on Developer). Langfuse counts a blended “units” metric (50,000/month free on Hobby) that isn’t broken out into separate trace/event/observation counts on its pricing page. Arize AX counts “trace spans” (25,000/month free on AX Free) plus a separate 1 GB ingestion cap. Always check a vendor’s current usage/billing docs before assuming your workload fits a free tier.

Which one is easiest to switch away from later?

Because Langfuse ingests traces via OpenTelemetry and framework-agnostic SDKs and is MIT licensed end to end for the core, it carries the least lock-in risk — you can self-host, export your data, or migrate without a vendor relationship standing in the way. LangSmith and Arize AX are both closed-source commercial products, though both also support OpenTelemetry ingestion, which keeps your instrumentation code portable even if the backend isn’t.

Sources

Unknown's avatar

Author:

Innovations, logic & design thinking