Structured LLM Outputs in Python: Instructor vs Outlines vs Pydantic AI (2026)
Instructor, Outlines, and Pydantic AI each solve structured LLM outputs a different way. Here is how to choose in 2026, with working code and FastAPI patterns.
Structured LLM outputs in Python are typed, schema-validated objects (usually Pydantic models) returned by an LLM instead of freeform text. The three tools people actually reach for in 2026 are Instructor (post-generation validation with automatic retries), Outlines (pre-generation constrained sampling that guarantees a valid schema), and Pydantic AI (structured outputs baked into a full agent loop). I ship FastAPI services for a living, so I've had to pick between these three more times than I'd like to admit. So, let's walk through when each one is the right call.
Instructor 1.15.4 (released June 28, 2026) is the default choice for API-based LLMs. It patches your existing OpenAI/Anthropic/Gemini client and adds a Pydantic response_model parameter.
Outlines uses a finite state machine to mask invalid tokens during generation, guaranteeing 100% schema compliance without retry overhead. The catch: it needs local model weights or a compatible inference server.
Pydantic AI is not just a validator. It's an agent framework where typed outputs, tool calling, and dependency injection share one API.
All three rely on Pydantic v2 under the hood, so schema definitions are portable. You can swap libraries without rewriting your models.
For high-volume classification tasks, benchmarks consistently show Outlines is faster because there are no wasted API calls on invalid JSON.
Post-generation validation (Instructor, Pydantic AI) works with any hosted API; pre-generation constraints (Outlines) require you to control the sampler.
What are structured LLM outputs?
A "structured output" is a response from an LLM that conforms to a schema you defined ahead of time (a Pydantic BaseModel, a JSON Schema, or a regex) rather than a blob of natural language you have to parse. Instead of writing "Extract the name, age, and email from this text and return JSON", you declare a class like User(BaseModel): name: str; age: int; email: EmailStr and get back a validated User instance. This matters because every real production LLM workflow eventually needs to plug into something that isn't an LLM: a database write, an API call, a Kafka topic, a scikit-learn pipeline. Untyped strings do not survive that boundary.
Historically, developers relied on prompting tricks ("return JSON, only JSON, no markdown fences, I mean it") and hoped for the best. Honestly, that approach fails maybe 5-15% of the time even on strong models, and you get to hunt for the bug in production. The three libraries I cover here each attack the problem from a different angle: validate afterward and retry on failure, constrain generation so failure is impossible, or fold validation into an agent that self-corrects. Understanding which layer of the stack you're pushing on is the whole point.
If you've done data validation in Python before, say Pandera schemas for DataFrames, this will feel familiar. The mental model is the same: declare the shape you expect, let the library enforce it, and stop hand-rolling try/except json.JSONDecodeError everywhere.
The three approaches in the 2026 landscape
Before we drop into code, the comparison table below is the mental map I keep in my head when someone on our team asks which library to use for a new service. The dimensions are the ones that actually matter for a backend engineer: does it work with the hosted API we already pay for, does it need me to run infrastructure, and does it play nicely with our async FastAPI stack?
API-based extraction, migration from raw OpenAI SDK
High-throughput self-hosted inference
New agent projects, tool-heavy workflows
One thing to notice: none of these are direct competitors in every sense. Outlines and Instructor solve different problems (constrain the model vs. validate the model's output), and Pydantic AI wraps Instructor-style validation inside a bigger box. It's completely reasonable to use two of them in one codebase: Instructor for simple extraction endpoints and Pydantic AI for the agent workflows.
Instructor: post-generation validation with Pydantic
Instructor is what I reach for first, and it's the answer for most people. It patches your existing LLM client and adds a response_model parameter that takes any Pydantic model. If validation fails, Instructor feeds the Pydantic error back to the model and asks it to try again, up to max_retries times (default 3). The official Instructor documentation covers the full API, but here's the shape of it.
import instructor
from pydantic import BaseModel, EmailStr, Field
from openai import OpenAI
class Contact(BaseModel):
name: str
email: EmailStr
role: str = Field(description="Job title or role at the company")
client = instructor.from_openai(OpenAI())
contact = client.chat.completions.create(
model="gpt-4o-mini",
response_model=Contact,
max_retries=3,
messages=[{
"role": "user",
"content": "Extract the contact from: 'Reach out to Ana Silva "
"([email protected]), Head of Data at Acme.'",
}],
)
print(contact.email) # [email protected], already validated as an email
The thing I love about this pattern: it's just an OpenAI SDK call with one extra kwarg. Nothing about my error handling, async setup, retries at the HTTP layer, or observability changes. The library shipped version 1.15.4 on June 28, 2026, and the current recommended entry point is instructor.from_provider("openai/gpt-4o-mini"), which normalizes credential handling across providers so you can move from OpenAI to Anthropic to Gemini by changing the string.
Instructor also supports streaming partial objects (you get a stream of increasingly-complete Pydantic models as the LLM types) and iterables, which is how you extract a list of items without loading the whole response first. If you're already familiar with how async httpx + asyncio works for I/O-bound Python, the async variant of Instructor slots in exactly where you'd expect.
Outlines: constrained token sampling that can't produce invalid JSON
Outlines takes a fundamentally different angle. Instead of hoping the model produces valid output and validating afterward, Outlines builds a finite state machine from your schema and, at each sampling step, masks any token that would violate the schema. The model literally cannot emit an unclosed brace, an integer where a boolean should be, or a field name that doesn't exist. Schema compliance goes from "very likely" to mathematically guaranteed.
from outlines import models, generate
from pydantic import BaseModel
from typing import Literal
class Ticket(BaseModel):
category: Literal["billing", "bug", "feature-request", "other"]
priority: Literal["low", "medium", "high"]
summary: str
model = models.transformers("meta-llama/Llama-3.1-8B-Instruct")
generator = generate.json(model, Ticket)
ticket = generator(
"Classify: 'My invoice from June is missing line items, urgent.'"
)
print(ticket) # Ticket(category='billing', priority='high', summary=...)
The upside is that this cannot fail schema validation. That matters more than it sounds. On a batch of 100,000 support tickets, even a 2% failure rate means 2,000 retries, which means 2,000 extra API-model calls, which means real money. Outlines gets you 100% first-pass rate. The Outlines GitHub repo has a benchmarks section showing that for regex-constrained generation, the overhead is actually negative. Outlines is faster than free-form because the sampler can skip tokens with only one valid choice entirely.
The catch is infrastructure. Outlines needs to control the sampler, which means either running a local model via transformers, or using an inference server that exposes token-level hooks: vLLM, TGI, or SGLang. If you're already self-hosting an LLM inference server like vLLM, integrating Outlines is a small step. If you're calling the OpenAI API from a Vercel Function, Outlines is not for you.
Pydantic AI: structured outputs inside an agent loop
Pydantic AI is the official agent framework from the Pydantic team itself, so structured outputs aren't bolted on. They're the primary abstraction. You define an Agent, give it a result_type that's any Pydantic model, and every call returns an instance of that type. Where Instructor hands you back a validated model, Pydantic AI hands you back a whole agent run, complete with tool calls, message history, and structured results.
from pydantic import BaseModel
from pydantic_ai import Agent, RunContext
class SupportReply(BaseModel):
tone: str
body: str
escalate_to_human: bool
agent = Agent(
"openai:gpt-4o-mini",
result_type=SupportReply,
system_prompt="You draft empathetic replies. Escalate anything legal.",
)
@agent.tool
def lookup_order(ctx: RunContext[None], order_id: str) -> dict:
return {"order_id": order_id, "status": "delivered", "carrier": "DHL"}
result = agent.run_sync(
"Customer says order 8842 never arrived. Draft a reply."
)
print(result.data.body)
print(result.data.escalate_to_human) # False
What you get on top of validation: a real agent loop, dependency injection via RunContext, tool registration decorators, message history you can persist to a database, and a graph API for multi-step workflows. If validation fails, Pydantic AI doesn't just retry the same call. It feeds the error back through the agent's reflection mechanism so tool results can influence the correction. See the Pydantic AI documentation for the full agent API.
The tradeoff: Pydantic AI is heavier than Instructor. If all you need is "call an LLM, get back a typed object", you're using 10% of the framework and paying for the rest. But the moment you need tools, memory, or multi-step logic, Instructor stops being enough and you'd end up rebuilding those primitives yourself.
How do I choose between Instructor, Outlines, and Pydantic AI?
Here's the decision tree I use when a colleague asks. Start with the question "am I self-hosting the model?" If yes, and you need guaranteed schema compliance at high volume (batch classification, entity extraction over millions of docs), Outlines wins on cost and latency. If no (you're calling OpenAI, Anthropic, Gemini, etc.), move to the next question.
The next question is "am I building an agent?" An agent means multi-step reasoning, tool calls, or memory. If yes, jump straight to Pydantic AI. You'll rebuild half of it on top of Instructor otherwise. If not (you're doing single-shot extraction, classification, summarization), go with Instructor. It's the smallest thing that works, integrates with any provider, and doesn't lock you into a framework.
A common in-between case: you're building a small tool-using agent, but the tools are simple (a database lookup or two) and you don't need memory. Both Instructor's tool-calling helpers and Pydantic AI would work. I'd pick Pydantic AI here anyway. The second tool you add will be easier, and adding a message history later is a two-line change instead of a rewrite. The overhead is real but small.
Retry strategy, validators, and error feedback
The retry loop is where these libraries genuinely earn their keep, and it's the piece most engineers underestimate. When Instructor gets a Pydantic ValidationError, it doesn't just retry blindly. It appends the error message to the conversation and asks the model to fix its output. Something like: "Your last response failed validation: field 'email' is not a valid email address. Please try again." Modern models correct on the first retry ~95% of the time when given this feedback.
You can steer this hard by adding custom validators. Say you're extracting SKUs and they must exist in your product catalog:
from pydantic import BaseModel, field_validator
from db import product_exists # your own lookup
class OrderLine(BaseModel):
sku: str
quantity: int
@field_validator("sku")
@classmethod
def sku_must_exist(cls, v: str) -> str:
if not product_exists(v):
raise ValueError(f"SKU {v!r} is not in the catalog")
return v
Now Instructor will retry, with the specific error "SKU 'XYZ-123' is not in the catalog", until the model produces a valid SKU or hits max_retries. This is enormously powerful and something you cannot do with raw JSON mode. I've used this pattern to constrain LLM outputs to real database entities, and it's the difference between a demo and something you can put in front of customers.
Serving structured outputs from a FastAPI endpoint
Because I live in FastAPI land, here's the pattern that matters. Instructor's async client and FastAPI's async dependencies compose without any friction. The response model of your endpoint is the response model of your LLM call. That symmetry is why this stack feels so tight in Python.
The Contact model does triple duty: it validates the LLM output, it's the FastAPI response schema (so OpenAPI docs are automatic), and if you want to persist the result you can pass it straight to SQLAlchemy or drop it into a pandas DataFrame. One class, four boundaries. This is the pattern that finally sold me on Python for LLM-adjacent backends. I'm not maintaining three parallel type systems.
Common pitfalls and how to avoid them
A few things I've gotten burned by, so you don't have to. First: markdown-fenced JSON. Some models (older Llama variants especially) love to wrap JSON in triple-backtick blocks even when you tell them not to. Instructor's default parser handles this now, but if you see ValidationError on obviously-correct-looking output, check the raw response. The fence is usually the culprit. Setting mode=instructor.Mode.TOOLS forces function-calling mode and sidesteps the fence problem entirely on providers that support it.
Second: don't wrap Optional[str] around everything to be safe. All-optional schemas give the model too much room; it just returns nulls when it doesn't want to think. Make required fields required and let validation force the retry. That's the whole point of the library.
Third: streaming partial objects looks great in demos but is easy to misuse. Partial models can have None where the final model won't, so downstream code must tolerate incomplete state. If you don't need streaming for UX reasons, don't turn it on.
Finally: token cost. Every retry doubles your latency and API bill on that request. Log the retry count and alert on p95 retries > 0. In my last project, a schema change silently pushed p50 retries from 0 to 2 because we added a field the model wasn't grounded on. Cost went up 200% overnight before we caught it in Grafana.
Frequently Asked Questions
What is the difference between Instructor and Pydantic AI?
Instructor is a lightweight wrapper that adds a response_model parameter to any LLM client. It does one thing: extract validated Pydantic objects from single calls. Pydantic AI is a full agent framework from the Pydantic team that includes structured outputs, plus tool calling, dependency injection, message history, and multi-step workflows. Use Instructor for extraction, Pydantic AI for agents.
Can Outlines work with the OpenAI API?
Not natively for token-level constraints. Outlines needs control over the sampler, which the OpenAI API doesn't expose. There are wrappers that use OpenAI's JSON mode as a fallback, but you lose the FSM guarantee. To get Outlines' full benefits you need local model weights or a compatible inference server like vLLM, TGI, or SGLang.
Which is faster: Instructor or Outlines for structured outputs?
For high-volume simple tasks (classification, extraction over many items), Outlines is significantly faster because it can't produce invalid output, so there are zero retries. The Rust port of its FSM algorithms in 2026 also removed most compile overhead. Instructor wins on setup speed (one line vs. running your own inference server), so at low volume the difference is dwarfed by API latency.
Do I still need Pydantic if I use Outlines?
You don't strictly need it. Outlines accepts JSON Schema, regex, or context-free grammars directly. But defining your schema as a Pydantic model gives you IDE autocomplete, downstream validation, and portability across libraries. Nearly everyone using Outlines in production defines their schemas as Pydantic BaseModel classes and passes the model in.
How do structured outputs handle nested objects and lists?
All three libraries handle arbitrarily nested Pydantic models and list[Model] fields. Instructor's create_iterable streams a list one item at a time, so you can start processing before the full list is generated. Outlines supports nesting via JSON Schema references and enforces list length bounds if you specify them. Pydantic AI passes nested schemas through unchanged.
Langfuse vs LangSmith vs Arize Phoenix for LLM observability in Python: pricing, self-hosting, OpenTelemetry support, and real production tradeoffs from someone who has shipped all three.
A hands-on guide to Delta Lake in Python with delta-rs 1.6: install, write and read from pandas/Polars, MERGE upserts, time travel and RESTORE, OPTIMIZE with Z-ORDER, plus a production checklist. No Spark or JVM required.
DSPy 3.0 turns prompt engineering into a compile step. Build typed LLM programs in Python, then let MIPROv2 optimize prompts against a metric on your data.
