DSPy 3.0 Python Guide (2026)

Q: Can DSPy fine-tune models, not just prompts?

Yes. The BootstrapFinetune optimizer distills a compiled DSPy program into a fine-tuned smaller model by generating training data from the compiled prompts, then submitting a supervised fine-tuning job through the configured provider. It's the recommended path once a DSPy program is working and you want to shrink inference cost or latency.

Updated: July 4, 2026

DSPy is a Python framework from Stanford NLP that lets you program language models with typed signatures and composable modules instead of writing brittle string prompts, then compile those programs into optimized prompts (or fine-tuned weights) using automatic optimizers like MIPROv2. In practice, that means you describe what you want an LLM to do (inputs, outputs, and a control flow of modules like Predict, ChainOfThought, and ReAct), and DSPy figures out the exact prompt text, few-shot demonstrations, and even instruction wording that maximizes a metric on your data. DSPy 3.0, released in mid-2026, ships a stable API, streaming, MLflow-native tracing, and a rebuilt optimizer stack that turned "prompt engineering" into a compile step rather than a Slack thread.

DSPy replaces hand-tuned prompt strings with typed Signatures (input/output schemas) and reusable Modules, so LLM code looks like PyTorch, not f-strings.
DSPy 3.0 (July 2026) stabilizes the dspy.LM client, adds native streaming, MLflow autologging, and promotes MIPROv2 and SIMBA as the default optimizers.
Optimizers (formerly "teleprompters") take a metric plus a small training set and automatically generate few-shot demos and improved instructions. No manual prompt iteration needed.
DSPy composes cleanly with retrieval (Qdrant, Weaviate, LanceDB, ColBERTv2), tool use via ReAct, and any provider that speaks the OpenAI-compatible API, including local vLLM or Ollama backends.
Compared with LangChain, DSPy is smaller and opinionated toward optimization, and does not ship its own agent runtime. Compared with LangGraph, it's a program-and-compile framework rather than a graph executor.
Production readiness improved sharply in 3.0, thanks to typed dspy.Prediction objects, deterministic caching, async streaming, and first-class error surfacing.

What is DSPy used for?

DSPy is used for building LLM systems whose behavior is optimized against a metric rather than hand-crafted in prompt strings. The typical use cases are retrieval-augmented generation (RAG), classification and extraction pipelines, multi-hop question answering, tool-using agents, and synthetic data generation. Basically, anywhere you would previously write a prompt template with a few examples and iterate by eyeball. Instead of the prompt-tweak loop, you write a Python program made of DSPy modules, define a metric (exact match, F1, ROUGE, LLM-as-judge, anything callable), collect 20 to a few hundred labeled examples, and hand the whole thing to an optimizer that searches over few-shot demonstrations and instruction wording.

The design philosophy comes from the original DSPy paper by Khattab et al. (2024), which reframed prompt engineering as a compilation problem. The developer specifies the computation graph, and the compiler figures out prompts that make each node work well on the target task. In real projects that shifts effort from "why is GPT-4o refusing this today?" to "what's my metric and how much labeled data do I have?", which is a much more tractable question. Honestly, I found this reframing painful at first (I like tweaking prompts more than I should), but it survives model swaps in a way hand-tuned strings never did.

What's new in DSPy 3.0

DSPy 3.0 landed in July 2026 and is the first release the team formally labels production-grade. The headline changes: the legacy dspy.OpenAI, dspy.Anthropic, and dspy.HFModel clients are gone, replaced by a single provider-agnostic dspy.LM("openai/gpt-4o-mini") that routes through litellm. The old Teleprompter namespace is fully renamed to dspy.teleprompt optimizers, and MIPROv2 plus the new SIMBA optimizer are the recommended defaults for most tasks.

Streaming became a first-class citizen: dspy.streamify wraps any module and yields token deltas plus a final Prediction. Async is native, and every module exposes acall so the runtime batches concurrent LM calls automatically. Caching moved from an on-disk pickle to a content-addressed store with configurable TTL, and MLflow autologging now captures the compiled program, every LM call, and optimizer traces without extra glue code. Finally, dspy.Prediction objects are typed with the signature's output fields, so IDE autocomplete and mypy work end-to-end.

Installing DSPy and configuring an LM

DSPy 3.0 requires Python 3.10 or newer. Install with uv or pip, and pick provider extras only if you want the convenience:

# uv (recommended in 2026, see our companion pieces on modern Python tooling)
uv add "dspy>=3.0" openai

# or with pip
pip install "dspy>=3.0" openai

Configuration is a single call. Every module you define later reads the LM from dspy.settings, so you rarely pass it explicitly:

import os
import dspy

lm = dspy.LM(
    "openai/gpt-4o-mini",
    api_key=os.environ["OPENAI_API_KEY"],
    max_tokens=1024,
    temperature=0.0,
    cache=True,       # content-addressed cache, on by default
)
dspy.settings.configure(lm=lm)

The "openai/gpt-4o-mini" prefix is a litellm route string, so "anthropic/claude-opus-4-8", "gemini/gemini-2.5-pro", "ollama/llama3.1:8b", and "openai/localhost:8000/v1" (for a self-hosted vLLM) all work with the same client. That single seam is what lets you compile once and swap providers at deploy time, a feature developers on the DSPy GitHub repo repeatedly point to as the biggest quality-of-life win in 3.0.

Signatures and modules: your first DSPy program

A Signature is a typed contract for one LM call: named input fields, named output fields, and an optional docstring that becomes part of the instruction. A Module is a callable that runs one or more signatures. The built-in dspy.Predict module is the base case, one signature, one LM call.

import dspy

class ClassifySentiment(dspy.Signature):
    """Classify the sentiment of a customer support message."""
    message: str = dspy.InputField(desc="Raw customer message text")
    sentiment: str = dspy.OutputField(desc="One of: positive, neutral, negative")
    confidence: float = dspy.OutputField(desc="Confidence between 0 and 1")

classify = dspy.Predict(ClassifySentiment)

result = classify(message="Shipping took three weeks and support ghosted me.")
print(result.sentiment, result.confidence)
# negative 0.94

Notice what's not here: no prompt string, no JSON parsing, no "you are a helpful assistant" preamble. DSPy generates a prompt from the signature at call time and parses the output into a typed dspy.Prediction. If the LM produces malformed output, DSPy retries with a repair prompt automatically. You can inspect the exact prompt sent with dspy.settings.lm.inspect_history(n=1), and that's a habit worth building early, because the real leverage of DSPy comes from trusting the compiler, not from micro-managing the string.

Composing modules is plain Python. Subclass dspy.Module, instantiate the sub-modules in __init__, and write ordinary control flow in forward. This is the pattern every non-trivial DSPy program follows, including the RAG example later in this guide.

ChainOfThought, ReAct, and multi-step programs

dspy.ChainOfThought is a drop-in replacement for Predict that adds a hidden reasoning output field before the declared outputs. That single change usually improves accuracy on math, multi-hop QA, and any task requiring intermediate steps by 5 to 20 points, which is why it's often the first optimization worth trying:

class AnswerMathWord(dspy.Signature):
    """Solve a grade-school math word problem."""
    question: str = dspy.InputField()
    answer: int = dspy.OutputField(desc="Final integer answer")

solve = dspy.ChainOfThought(AnswerMathWord)
pred = solve(question="A store sold 12 shirts on Monday and thrice that on Tuesday. Total?")
print(pred.reasoning)   # step-by-step chain
print(pred.answer)      # 48

For tool-using agents, dspy.ReAct implements the classic reason-act loop against a list of Python tools. Each tool is just a function with a docstring and typed arguments. DSPy converts them into a tool schema at call time and lets the LM interleave Thought, Action, and Observation steps until it emits a final answer:

def search_wiki(query: str) -> str:
    """Return a short summary from Wikipedia for the given query."""
    # your real search implementation here
    return wiki_client.summary(query)

def calculator(expression: str) -> float:
    """Evaluate a simple arithmetic expression like '12 * 3.14'."""
    return float(eval(expression, {"__builtins__": {}}))

agent = dspy.ReAct("question -> answer", tools=[search_wiki, calculator], max_iters=6)
pred = agent(question="How many years between Ada Lovelace's birth and Alan Turing's birth?")
print(pred.answer)

The "question -> answer" shorthand is an inline signature; you can write full class-based signatures whenever you need field descriptions or richer types. Combined with our companion piece on structured LLM outputs in Python, ReAct plus typed signatures cover the majority of what teams reach for LangChain-style agent frameworks to do, with far less code.

Optimizers: BootstrapFewShot, MIPROv2, and SIMBA

Optimizers are DSPy's core value proposition. Given a program, a metric, and a training set of dspy.Example objects, an optimizer searches over few-shot demonstrations and instruction wording to maximize the metric, then returns a compiled program with those choices baked in. The three you'll use most in 2026:

Optimizer	What it does	Training set size	Cost	When to use
`BootstrapFewShot`	Uses the teacher LM to generate few-shot demos, keeps the ones that pass the metric	10 to 50 examples	Low ($1 to $5)	First pass, quick wins, no instruction rewriting
`MIPROv2`	Joint Bayesian search over instructions and demos, with a proposer LM	50 to 500 examples	Medium ($10 to $100)	Default 3.0 recommendation; best accuracy/cost trade-off
`SIMBA`	Stochastic mini-batch ascent, resembles online RL over prompts	200+ examples	Higher	Large training sets, non-differentiable metrics, agents
`BootstrapFinetune`	Distills a compiled program into a fine-tuned smaller model	500+ examples	Highest	Once you have a working DSPy program and want to shrink inference cost

A minimal compile loop looks like this:

from dspy.teleprompt import MIPROv2

def em_metric(example, pred, trace=None):
    return example.answer.lower() == pred.answer.lower()

trainset = [dspy.Example(question=q, answer=a).with_inputs("question")
            for q, a in load_qa_pairs("train.jsonl")]

optimizer = MIPROv2(metric=em_metric, auto="medium")   # "light" | "medium" | "heavy"
compiled_solver = optimizer.compile(solve, trainset=trainset)

compiled_solver.save("solver_v1.json")   # portable, reloads with .load()

The compiled artifact is a small JSON file containing selected demos and generated instructions. Ship it as part of your Docker image and the runtime does zero optimization. That's the whole point. For a deep dive on evaluating LLM programs the way DSPy expects, see the official DSPy optimization guide.

Building a RAG pipeline with DSPy

RAG is the canonical DSPy program because retrieval and generation are separate modules with separate failure modes, which is exactly what optimizers are good at untangling. I hit this exact split shipping a support-ticket RAG last spring: the answer step looked fine in isolation, but the retrieval step was silently returning stale docs, and only the metric+optimizer combo made that obvious. Here's a compact multi-hop RAG program using a vector store you already have (Qdrant, Weaviate, LanceDB, and Chroma all ship DSPy adapters; the interface is a callable that returns a list of dspy.Passage):

import dspy

# Assume `retrieve` is any callable: str -> list[str] of passage text.
# The dspy.retrievers namespace ships adapters; a plain function works too.

class GenerateAnswer(dspy.Signature):
    """Answer the question using ONLY the provided context."""
    context: list[str] = dspy.InputField()
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="A concise, factual answer")

class MultiHopRAG(dspy.Module):
    def __init__(self, retrieve, k: int = 4):
        super().__init__()
        self.retrieve = retrieve
        self.k = k
        self.generate_query = dspy.ChainOfThought("question, context -> search_query")
        self.answer = dspy.ChainOfThought(GenerateAnswer)

    def forward(self, question: str) -> dspy.Prediction:
        context: list[str] = []
        for _ in range(2):    # two retrieval hops
            q = self.generate_query(question=question, context=context).search_query
            context.extend(self.retrieve(q)[: self.k])
        return self.answer(context=context, question=question)

rag = MultiHopRAG(retrieve=my_qdrant_search)
print(rag("Which Python library introduced Copy-on-Write DataFrames?").answer)

Compile this with MIPROv2 and a metric like answer_exact_match, and you typically see 10 to 30 point accuracy gains over the same graph with hand-written prompts, without changing a single line of the module code. For a broader landscape of Python LLM tooling that composes with DSPy in production, our guide to LLM inference servers like vLLM, TGI, and SGLang covers the serving layer this program will eventually run against.

How is DSPy different from LangChain and LangGraph?

The short answer: DSPy is a compiler for LLM programs, LangChain is a runtime with hundreds of integrations, and LangGraph is a state-machine executor for agents. They aren't direct competitors. Many teams use LangChain for I/O adapters and DSPy for the actual reasoning modules. Still, the choice of "which one does the LLM call live in?" matters.

Dimension	DSPy 3.0	LangChain 0.3	LangGraph 0.4
Core abstraction	Signatures + Modules + Optimizers	Runnables (LCEL)	StateGraph nodes
Prompt authoring	Auto-generated from signatures, tuned by optimizer	Hand-written templates	Hand-written templates
Metric-driven optimization	First-class (MIPROv2, SIMBA)	None built-in	None built-in
Integrations	Fewer, but litellm-based	Very broad (500+)	Uses LangChain integrations
Agent runtime	`ReAct` module, no persistent state	Agent classes	Explicit graph with persistent state, checkpoints, human-in-the-loop
Typing	Full type hints on Predictions	Partial (Pydantic on tools)	Full (TypedDict states)
Best for	Optimizable pipelines: RAG, classification, extraction	Rapid prototyping with many services	Long-running stateful agents

A practical rule of thumb: if you have a metric and labeled examples, start in DSPy. If you have neither yet, LangChain's runnables let you sketch faster. If your problem is a long-running agent with checkpoints and interruption, LangGraph's state model is worth the extra ceremony. All three interoperate. A DSPy module can be wrapped as a LangChain Runnable, and a LangGraph node can call a compiled DSPy program.

Does DSPy work with local LLMs?

Yes, and this is one of the biggest advantages of the 3.0 client change. Because every provider goes through litellm, any endpoint that speaks the OpenAI Chat Completions API works: vLLM's OpenAI-compatible server, TGI, Ollama, LM Studio, llama.cpp's llama-server, and SGLang all drop in with a URL change:

# Local vLLM serving Llama-3.1-8B-Instruct
lm = dspy.LM(
    "openai/meta-llama/Meta-Llama-3.1-8B-Instruct",
    api_base="http://localhost:8000/v1",
    api_key="EMPTY",
    max_tokens=1024,
)
dspy.settings.configure(lm=lm)

# Ollama
lm_ollama = dspy.LM("ollama_chat/llama3.1:8b", api_base="http://localhost:11434")

Local-model workflows benefit especially from optimizers, because a MIPROv2-compiled prompt on an 8B open model often matches an unoptimized prompt on GPT-4o for narrow tasks, at a fraction of the inference cost. If you're running quantized weights, our guide to LLM quantization with GPTQ, AWQ, bitsandbytes, and GGUF covers the trade-offs that matter before you point DSPy at a local server.

Production: caching, streaming, and MLflow tracing

DSPy 3.0's production story is finally boring in a good way. Caching is on by default and keyed by the full serialized request, so repeated calls during evaluation cost zero tokens. Streaming is enabled per module:

stream_solve = dspy.streamify(solve)
async for chunk in stream_solve(question="..."):
    if isinstance(chunk, dspy.streaming.StreamResponse):
        print(chunk.chunk, end="", flush=True)
    elif isinstance(chunk, dspy.Prediction):
        final = chunk

MLflow autologging captures every LM call, the compiled program, and the optimizer's search trace. Enable it once, and every subsequent DSPy call shows up in your MLflow UI alongside metrics and artifacts:

import mlflow
mlflow.dspy.autolog()   # available in mlflow >= 2.20

with mlflow.start_run(run_name="rag-v1"):
    compiled = optimizer.compile(rag, trainset=trainset)
    mlflow.log_metric("dev_em", evaluate(compiled, devset))

For teams already using our recommended experiment-tracking stack (see our comparison of MLflow vs W&B vs Comet), the integration is one line, and it shows LLM calls in the same place as your classical ML runs. That's a real improvement over the fragmented telemetry most LLM stacks had through 2025.

Frequently Asked Questions

Is DSPy production-ready in 2026?

Yes. DSPy 3.0 is the first release the maintainers explicitly market as production-grade. It ships stable APIs, typed predictions, native async and streaming, deterministic caching, and MLflow autologging. Teams at Databricks, Zenlytic, and multiple Stanford spinouts run compiled DSPy programs in production RAG and classification pipelines.

Do I need labeled data to use DSPy?

Not to use it, but you do to benefit from optimizers. You can run dspy.Predict and dspy.ChainOfThought with zero examples. The payoff of DSPy over raw prompting comes when you have 20 to a few hundred labeled inputs plus a metric, at which point MIPROv2 or BootstrapFewShot can compile a measurably better program.

What are DSPy signatures?

A DSPy Signature is a typed contract for one LM call. You declare input and output fields with type hints and optional descriptions, and DSPy generates the prompt, parses the LM response into a typed Prediction object, and retries on malformed output. Think of them as function signatures for language-model calls.

How do you optimize prompts with DSPy?

You pick an optimizer (MIPROv2 is the 3.0 default), define a metric function that scores predictions against ground truth, build a training set of dspy.Example objects, and call optimizer.compile(program, trainset=...). The optimizer searches over few-shot demonstrations and instruction wording, then returns a compiled program with the best combination baked in.

Can DSPy fine-tune models, not just prompts?

Yes. The BootstrapFinetune optimizer distills a compiled DSPy program into a fine-tuned smaller model by generating training data from the compiled prompts, then submitting a supervised fine-tuning job through the configured provider. It's the recommended path once a DSPy program is working and you want to shrink inference cost or latency.