LLM Observability in Python: Langfuse vs LangSmith vs Arize Phoenix (2026)

Updated: July 6, 2026

LLM observability in Python is the practice of tracing every prompt, tool call, and model response an application makes, then attaching cost, latency, and quality metrics so you can debug and monitor it in production. In 2026 the three tools competing for that job in Python stacks are Langfuse (open-source, framework-agnostic), LangSmith (LangChain-native SaaS), and Arize Phoenix (open-source, notebook-first). I've shipped all three to production, and honestly, I'll tell you which one to pick before you touch a config file.

• Langfuse is MIT-licensed, framework-agnostic, and roughly 25x cheaper than LangSmith at 1M events per month; ClickHouse acquired it in January 2026.
• LangSmith is closed-source and seat-priced but has native LangGraph tracing, mature evaluation, and built-in Slack alerting that Langfuse still requires webhooks for.
• Arize Phoenix is Elastic-licensed, OpenTelemetry-native, and best-in-class for RAG faithfulness and retrieval evaluation, especially inside notebooks.
• OpenTelemetry GenAI semantic conventions are the escape hatch: instrument once with OTel and swap backends without touching agent code.
• All three can trace the raw OpenAI SDK, LlamaIndex, Pydantic AI, and DSPy. LangChain-first thinking is a 2024 assumption that no longer holds.
• For regulated data, self-host Langfuse or Phoenix; LangSmith self-hosting requires an Enterprise contract most teams cannot justify.

What is LLM observability, and why do you need it?

LLM observability is distributed tracing plus token accounting plus quality evaluation, glued together in one UI. A single user request in a modern agent fans out into ten or twenty internal operations: a retrieval query against a vector store, three tool calls, two LLM calls with structured output, a reranker, and a final generation. When latency spikes to nineteen seconds and someone pings you at 2 a.m., tail -f on the app logs isn't going to tell you which of those steps blew up. You need a trace tree, per-span token counts, and the exact prompt that was sent.

The other reason is cost. I inherited a RAG service last quarter that was quietly spending $18,000 a month because a chain was re-fetching the same 40k-token knowledge-base chunk on every turn. Nothing in Datadog surfaced it. The HTTP metrics looked fine. It took one afternoon in Langfuse's UI, sorting spans by total_tokens descending, to find the leak. Standard APM tools cannot see inside an LLM call. Purpose-built LLM observability can.

Three things distinguish an LLM observability tool from a generic tracer: it understands the shape of a chat completion (system, user, assistant, tool), it aggregates cost by model and by user, and it lets you attach evaluations (a score, a rubric, a human annotation) to any span. If you've read our Python model serving comparison, this is the runtime cousin: same "record everything, query later" philosophy, but for what happens after the model ships.

Langfuse vs LangSmith vs Arize Phoenix: comparison table

Dimension	Langfuse	LangSmith	Arize Phoenix
License	MIT (fully open source)	Proprietary, closed source	Elastic 2.0 (open source)
Self-hosting	Free, Docker Compose in 30 min	Enterprise contract required	Free, single container or K8s
Framework fit	Any (OTel-native)	LangChain / LangGraph first	Any (OpenInference / OTel)
Best for	Operational telemetry, cost analytics	LangGraph agents, mature evals	RAG evaluation, notebook workflows
Cost at 1M events / month	~$101 / mo (Core)	~$2,514 / mo (Plus)	$0 (self-host) or Arize AX SaaS
Prompt management	Built-in, versioned	Prompt Hub, tight LangChain binding	Prompt playground, lighter feature
Native alerting	Webhooks / API	Slack, email, webhook (UI)	Via Arize AX only
Vendor lock-in risk	Low	High	Low

Langfuse: the open-source telemetry backbone

Langfuse is where I land by default for anything that has to run in a customer VPC. It's MIT-licensed, ships as a Docker Compose stack (Postgres, ClickHouse, Redis, S3-compatible object storage), and covers the full observability loop: tracing, prompt versioning, evaluations, and cost analytics. Multi-turn conversations are first-class, and traces are queryable from a Python SDK, a REST API, or straight ClickHouse SQL when the UI can't express what you want. As of mid-2026 the project reports over six million SDK installs per month and about 28,000 GitHub stars, which matters because in an on-call context you want your observability tool to have more issues closed than open.

Instrumenting the OpenAI SDK takes three lines. Nothing else in your agent code changes.

from langfuse.openai import openai  # drop-in wrapper
from langfuse import observe

@observe()
def answer(question: str) -> str:
    resp = openai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": question}],
        metadata={"langfuse_user_id": "[email protected]",
                  "langfuse_tags": ["prod", "support-bot"]},
    )
    return resp.choices[0].message.content

print(answer("What is a p99 latency budget?"))

The @observe decorator opens a parent span, the langfuse.openai wrapper attaches a child span with prompt, completion, model, and token counts, and the metadata fields become filterable facets in the UI. If you were already using Instructor or Pydantic AI for structured outputs, both work through the same wrapper. The JSON schema surfaces as a tool call on the span.

Where Langfuse still has rough edges: alerting is done via API and webhooks rather than a native "notify me when p95 latency crosses 4 seconds" toggle. You can wire it into PagerDuty in fifteen minutes, but LangSmith gives you that UI out of the box. Evaluations are also more DIY. You can score traces via the API, use LLM-as-judge templates, or push external eval framework results back in, but if you want zero-glue CI/CD gating, LangSmith's evaluate() is more polished.

LangSmith: the LangChain-native SaaS

LangSmith is the fastest path to a working trace if your stack is LangChain or LangGraph. Set LANGCHAIN_TRACING_V2=true and LANGCHAIN_API_KEY and every chain, tool call, and retriever gets traced automatically. There's no code change. LangGraph Studio, which ships alongside, is genuinely the best agent IDE I've used. You can visualize the graph, set breakpoints on nodes, modify state mid-run, and resume from a checkpoint. If you're debugging why a multi-agent system deadlocked at step 47, that IDE saves hours.

import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "lsv2_..."
os.environ["LANGCHAIN_PROJECT"] = "support-bot-prod"

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(model="gpt-4o-mini")
# every invocation now streams to LangSmith with zero code changes
print(llm.invoke([HumanMessage(content="What is p99?")]).content)

Two hard truths about LangSmith. First, self-hosting requires an Enterprise license, which is a hard blocker for teams that cannot ship customer data to a third-party SaaS. Second, pricing scales with seats and traces. LangSmith Plus at 1M events per month runs around $2,514 for a single seat with 14-day retention, per the latest published pricing math. If your team has ten engineers all needing dashboard access, that number climbs fast. In exchange you get very good evaluation tooling, a Prompt Hub that plugs directly into LangChain's prompt-loading API, and native OpenTelemetry ingestion (added in 2026, and long overdue).

Arize Phoenix: notebook-first RAG evaluation

Arize Phoenix is the tool I pick when the primary risk is retrieval quality, not latency. It's Elastic 2.0 open source, OpenTelemetry-native through the OpenInference spec, and it runs happily inside a Jupyter notebook, which is where most retrieval-augmented generation work still gets debugged. Phoenix has native RAGAS integration, and the built-in evaluators for faithfulness, answer relevance, and context precision are genuinely production-ready. I've used them to gate a nightly RAG benchmark job that would block a deploy on regression.

import phoenix as px
from openinference.instrumentation.openai import OpenAIInstrumentor
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor

session = px.launch_app()  # local UI at http://localhost:6006

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(
    OTLPSpanExporter(endpoint="http://localhost:6006/v1/traces")))
trace.set_tracer_provider(provider)

OpenAIInstrumentor().instrument()  # auto-trace every openai call

from openai import OpenAI
client = OpenAI()
client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain retrieval precision."}],
)
# open http://localhost:6006 - the trace, tokens, and cost are already there

Phoenix has two tiers you need to understand. The open-source Phoenix container is free forever, has no user cap, and stores traces in Postgres or SQLite. Arize AX is the paid SaaS built on the same trace format, and adds classical ML monitoring (drift, data quality, feature attribution) plus enterprise auth. If your team already lives in modern dataframe tools like the ones covered in our Polars and DuckDB guide, AX is the LLM-plus-ML unification story worth watching. If you only need LLM traces, stay on open-source Phoenix.

The weak spot is prompt management. Phoenix has a playground but no serious versioned prompt registry. If your team wants to promote prompts between environments with an audit trail, Langfuse and LangSmith both win here.

Pricing at 1M events per month, honestly

Cost-per-prediction is what I optimize for, and observability is a cost line item too. Here's the math at one million events per month, which is roughly where a mid-size production RAG bot lands.

• Langfuse Cloud (Core): around $101 per month, unlimited users, 30-day retention. Volume-based, so it stays predictable as team size grows.
• Langfuse self-hosted: roughly $150 per month in infrastructure (a small ClickHouse node, Postgres, Redis, and S3-compatible storage on a hyperscaler). The MIT license itself is free, but factor in the DevOps time to patch ClickHouse.
• LangSmith Plus: around $2,514 per month for a single seat with 14-day retention. Additional seats stack. Enterprise self-host is a separate contract.
• Arize Phoenix self-hosted: the same ~$150/month infrastructure story as Langfuse, minus the ClickHouse dependency (Phoenix can run on Postgres alone at moderate scale).
• Arize AX: priced per event and per model monitored. Request a quote, and don't assume it will undercut LangSmith.

At the low end (100k events, three seats), the gap narrows dramatically and LangSmith is often the cheapest total-cost-of-ownership option because you don't pay for anyone's time to keep ClickHouse alive. Under about 50,000 traces a month, LangSmith's 5,000-trace free tier or Langfuse Cloud's free tier make this a non-conversation. The decision only starts to bite at real volume.

OpenTelemetry GenAI: the vendor-neutral escape hatch

The single most important development in this space in 2026 is the OpenTelemetry GenAI semantic conventions. The GenAI Special Interest Group, formed in April 2024, has standardized the span attribute names, metric names, and event schemas that describe LLM calls, agent steps, tool invocations, and vector database queries. As of May 2026 the conventions are still marked Development in the OTel spec, but Datadog, Google Cloud, AWS, Azure, Langfuse, and Phoenix all ingest them natively.

What that means in practice: if you instrument once with the OTel GenAI conventions, you can send the same traces to Langfuse today, Phoenix next quarter, and Datadog in production, all without changing your agent code. Vendor lock-in becomes a routing config, not a rewrite. Auto-instrumentation libraries (OpenInference from Arize, OpenLLMetry from Traceloop, OpenLIT) cover most agent frameworks with two or three lines of setup and emit spans following the GenAI conventions.

from openlit import init
init(otlp_endpoint="http://langfuse:4318")  # or phoenix, or datadog
# every openai / anthropic / ollama / langchain call is now traced
# to whatever backend that endpoint points to, using OTel GenAI attrs

My recommendation for any greenfield agent in 2026: start with OTel GenAI conventions. Pipe them into Langfuse or Phoenix for the LLM-specific UI, and simultaneously into your existing APM (Datadog, Honeycomb) for correlation with the rest of the request. This is the same "instrument once, ship everywhere" pattern you'd use for HTTP tracing, and it applies to LLMs now.

How do you trace OpenAI calls in Python?

The bar for instrumenting the raw OpenAI SDK is now one to three lines of code across every tool in this comparison. The differences are what you get for those lines and how tied you become to a vendor. All four snippets below produce spans that carry the prompt, the completion, the model name, prompt and completion tokens, streaming timings, and request/response IDs.

Langfuse (drop-in wrapper)

from langfuse.openai import openai  # replaces `import openai`
# every openai.chat.completions.create call is now traced, no other change

LangSmith (env vars, LangChain-first)

import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
# LangSmith also traces raw OpenAI/Anthropic via the @traceable decorator
from langsmith import traceable

@traceable(run_type="llm")
def call_llm(prompt: str) -> str: ...

Arize Phoenix (OpenInference)

from openinference.instrumentation.openai import OpenAIInstrumentor
OpenAIInstrumentor().instrument()  # auto-instrument, one line

OpenLIT / MLflow (vendor-neutral)

import mlflow
mlflow.openai.autolog()  # ships spans anywhere OTel points to

Don't skip streaming timings. On any interactive product, time-to-first-token is a bigger user-experience signal than total latency, and every one of these will capture it if you leave the default configuration alone.

Which LLM observability tool should you pick?

Three heuristics, in order of decisiveness.

If you cannot send data to a SaaS, pick Langfuse or Arize Phoenix. Both are open source and self-hostable. Langfuse if you also want prompt management and cost analytics baked in. Phoenix if you're a RAG-heavy team and your engineers live in notebooks. LangSmith is off the table without an Enterprise contract, which usually means a six-figure minimum.

If your codebase is deeply LangChain / LangGraph and will stay that way, LangSmith is the fastest way to production. LangGraph Studio alone justifies it for complex agent debugging. Just budget accordingly. At 1M events a month with a five-person team you're north of $3,000/month, and switch costs are non-trivial. Route through OTel where you can to keep the exit door open.

If you are running a modern Python agent stack (Pydantic AI, the OpenAI SDK directly, or a mix of frameworks like DSPy and LlamaIndex), Langfuse is the default answer. It's framework-agnostic, cheap at scale, and the OTel path means you're not painting yourself into a corner. Pair it with Phoenix in your notebook workflow for RAG evaluation, and you've covered both production observability and iteration-time debugging without lock-in.

One more thing. What I would not do in 2026 is roll your own. I've seen two teams try to build an LLM observability layer on top of raw Prometheus and Loki, and both projects ate two engineer-quarters before landing at feature parity with the open-source options. If you're already paying the on-call tax for a real service, use one of these three.

Frequently Asked Questions

Is Langfuse better than LangSmith for production?

For most 2026 Python stacks, yes. Langfuse is MIT-licensed, framework-agnostic, roughly 25x cheaper at 1M events per month, and can be self-hosted for free. LangSmith wins only when you're all-in on LangChain/LangGraph and value LangGraph Studio and native alerting more than open-source flexibility.

Can you self-host LangSmith?

Yes, but only with an Enterprise contract. Unlike Langfuse (MIT) and Arize Phoenix (Elastic 2.0), LangSmith is proprietary. Small and mid-sized teams generally can't access self-hosting, which is why Langfuse dominates the on-prem and regulated-industry conversation.

Does Arize Phoenix work with the raw OpenAI SDK?

Yes. Phoenix ships an OpenInference instrumentor for the OpenAI SDK that auto-traces every completion, embedding, and streaming call with one line: OpenAIInstrumentor().instrument(). It also supports LlamaIndex, LangChain, DSPy, Anthropic, and any OpenTelemetry-instrumented framework.

What is the OpenTelemetry GenAI semantic convention?

It's a standardized set of span attributes, metrics, and event schemas for describing LLM calls, agent steps, and tool invocations. Developed by the OTel GenAI SIG since April 2024, it lets you instrument once and switch backends (Langfuse, Phoenix, Datadog) without changing agent code. It's still in Development status as of mid-2026 but broadly adopted.

Which LLM observability tool is best for RAG evaluation?

Arize Phoenix is best-in-class for RAG evaluation thanks to native RAGAS integration and production-ready evaluators for faithfulness, answer relevance, and context precision. Langfuse supports RAG evaluation via its LLM-as-judge templates and external eval integrations, but requires more glue code.

How much does LangSmith cost at production scale?

LangSmith Plus is approximately $2,514 per month for one seat at 1M events with 14-day retention. Additional seats and longer retention increase the price. At the same volume, Langfuse Cloud Core is around $101 per month with unlimited users, and self-hosted Langfuse or Phoenix costs roughly $150 per month in infrastructure.