TL;DR:

  • DSPy (Declarative Self-improving Python) lets you specify what a prompt should do — the input/output signature — rather than how it should say it; an optimiser then compiles the most effective prompt automatically using your evaluation examples
  • This replaces manual prompt tuning for production pipelines where you have labelled examples and a metric: DSPy consistently outperforms hand-engineered prompts on structured reasoning tasks
  • DSPy 2.5 (released early 2026) is production-stable with async support, typed signatures, and a suite of optimisers including MIPROv2, which is the current state-of-the-art for few-shot prompt compilation

If you’ve spent time carefully wording a chain-of-thought prompt, then re-wording it after a model upgrade changed the behaviour, then re-wording it again after switching from GPT-4o to Claude — DSPy addresses exactly that problem. The prompt string is not the right abstraction for building reliable LLM pipelines. A compiled program that adapts to the model is.

The Core Idea

In standard LLM development, you write a prompt as a string. When the model changes, or the task requirements shift, you rewrite the prompt. This is manual, non-transferable, and opaque — the prompt that works is the one that works, with no principled explanation of why.

DSPy reframes this. You write a signature — a type-annotated description of the module’s inputs and outputs — and optionally a docstring describing the transformation. DSPy’s runtime generates a concrete prompt from the signature and the module type (chain-of-thought, ReAct, etc.). The optimiser then automatically improves the prompt (or selects few-shot examples) using your dataset and a scoring function.

import dspy

# Configure the LM
lm = dspy.LM("claude-sonnet-4-6", max_tokens=2000)
dspy.configure(lm=lm)

# Define a signature — input/output types and descriptions
class ClassifyIntent(dspy.Signature):
    """Classify customer support message intent."""
    message: str = dspy.InputField(desc="Customer message text")
    intent: Literal["billing", "technical", "complaint", "general"] = dspy.OutputField()
    confidence: float = dspy.OutputField(desc="0.0 to 1.0 confidence score")

# Use the signature in a module
classifier = dspy.Predict(ClassifyIntent)

# Run inference
result = classifier(message="My invoice shows the wrong amount this month")
print(result.intent)      # "billing"
print(result.confidence)  # 0.94

The dspy.Predict module generates a structured prompt from ClassifyIntent, calls the LM, and parses the typed output. No prompt string written — DSPy handles it.

Module Types

DSPy ships with several built-in module types for common patterns:

# Chain-of-thought reasoning (adds "Let's think step by step" pattern)
cot_classifier = dspy.ChainOfThought(ClassifyIntent)

# ReAct agent with tool calling
class SearchAndSummarise(dspy.Signature):
    """Search for information and produce a summary."""
    query: str = dspy.InputField()
    summary: str = dspy.OutputField()

react_agent = dspy.ReAct(SearchAndSummarise, tools=[search_tool, retrieve_tool])

# Multi-hop reasoning with self-consistency
multi_hop = dspy.ChainOfThoughtWithHint(ClassifyIntent)

Composing Modules into Programs

DSPy modules compose like Python classes. Complex pipelines are just nested module calls:

class SupportTriage(dspy.Module):
    def __init__(self):
        self.classify = dspy.ChainOfThought(ClassifyIntent)
        self.extract_details = dspy.ChainOfThought(ExtractIssueDetails)
        self.generate_response = dspy.Predict(GenerateResponse)

    def forward(self, message: str) -> str:
        intent_result = self.classify(message=message)
        
        if intent_result.intent == "billing":
            details = self.extract_details(
                message=message, 
                intent=intent_result.intent
            )
            response = self.generate_response(
                issue_type="billing",
                details=details.summary
            )
        else:
            response = self.generate_response(
                issue_type=intent_result.intent,
                details=message
            )
        
        return response.text

# Use it
triage = SupportTriage()
result = triage(message="My invoice shows the wrong amount this month")

Optimisers: The Core Differentiator

Manual prompting ends here — optimisers compile the program using examples and a metric.

# Your labelled dataset
trainset = [
    dspy.Example(
        message="My card was charged twice",
        intent="billing"
    ).with_inputs("message"),
    dspy.Example(
        message="I can't log into my account",
        intent="technical"
    ).with_inputs("message"),
    # ... more examples
]

# Define a metric
def accuracy_metric(example, prediction, trace=None):
    return example.intent == prediction.intent

# Compile with BootstrapFewShot (fast, good for small datasets)
from dspy.teleprompt import BootstrapFewShot

optimiser = BootstrapFewShot(metric=accuracy_metric, max_bootstrapped_demos=4)
compiled_classifier = optimiser.compile(dspy.ChainOfThought(ClassifyIntent), trainset=trainset)

# The compiled classifier has better few-shot examples baked in
result = compiled_classifier(message="I was overcharged last month")

MIPROv2 is the more powerful optimiser for production use — it optimises both the instruction text and the few-shot examples simultaneously using Bayesian optimisation:

from dspy.teleprompt import MIPROv2

optimiser = MIPROv2(
    metric=accuracy_metric,
    auto="medium",  # "light" | "medium" | "heavy" — controls search budget
)

compiled = optimiser.compile(
    SupportTriage(),
    trainset=trainset,
    valset=valset,
    num_trials=20,
    minibatch=True
)

MIPROv2 typically requires 30–100 LM calls per optimisation run depending on dataset size, but the compiled result then runs faster and more accurately than the unoptimised baseline. For production pipelines where you have even 50 labelled examples, running MIPROv2 once is almost always worth it.

Typed Outputs and Validation

DSPy 2.5 added robust support for Pydantic models as output types, making it easy to get structured data back without writing output parsers:

from pydantic import BaseModel, Field

class IssueReport(BaseModel):
    issue_type: str
    account_id: Optional[str] = None
    amount: Optional[float] = None
    urgency: Literal["low", "medium", "high"]
    requires_human: bool

class ExtractIssue(dspy.Signature):
    """Extract structured issue details from a customer message."""
    message: str = dspy.InputField()
    report: IssueReport = dspy.OutputField()

extractor = dspy.Predict(ExtractIssue)
result = extractor(message="I was charged $49.99 instead of $9.99 this month")
print(result.report.amount)          # 49.99
print(result.report.requires_human)  # True
print(result.report.urgency)         # "medium"

DSPy handles the prompt construction to elicit the structured output and validates the result against the Pydantic model, retrying if parsing fails.

Async Support and Production Integration

DSPy 2.5 added native async support for concurrent agent pipelines:

import asyncio

async def process_batch(messages: list[str]) -> list[str]:
    tasks = [triage.acall(message=msg) for msg in messages]
    results = await asyncio.gather(*tasks)
    return [r.text for r in results]

# In an async context (FastAPI endpoint, agent loop, etc.)
responses = await process_batch(incoming_messages)

For production deployments, save and load compiled programs rather than re-running optimisation on every deploy:

# Save compiled program
compiled.save("support_triage_compiled.json")

# Load on startup
triage = SupportTriage()
triage.load("support_triage_compiled.json")

When DSPy Makes Sense

DSPy delivers the most value when:

  • You have a repeatable task with measurable quality (classification, extraction, summarisation with ground truth)
  • You have at least 20–50 labelled examples for optimisation
  • You need the pipeline to work reliably across multiple LM providers or model versions
  • You’re currently spending significant time tuning prompt strings manually

DSPy adds less value for:

  • One-off generation tasks where you won’t be evaluating quality at scale
  • Tasks where there’s no good automatic metric and all evaluation is qualitative
  • Simple single-step prompts that don’t have structured outputs

The key indicator: if you are or would be manually tuning a prompt and evaluating it against examples, DSPy can automate that loop.

Getting Started

pip install dspy

# With optional extras
pip install dspy[anthropic]  # Anthropic Claude support
pip install dspy[openai]     # OpenAI support
import dspy

# Configure default LM
dspy.configure(lm=dspy.LM("claude-sonnet-4-6"))

# Or with explicit settings
dspy.configure(
    lm=dspy.LM("claude-sonnet-4-6", temperature=0.0, max_tokens=1000),
    rm=dspy.ColBERTv2(url="http://your-colbert-server")  # optional retrieval model
)

References