DSPy and GEPA Explained: How LLM Programs Learn to Write Their Own Prompts

Here’s a workflow almost everyone building with LLMs recognizes.

You write a prompt. It mostly works. You find a case where it fails, so you add a sentence. That breaks two other cases, so you add another sentence to patch those. A week later your prompt is 400 words of accumulated scar tissue, half of it contradicting the other half, and you’re afraid to touch it because you no longer know which line is load-bearing.

This is prompt engineering as most teams practice it: hand-tuning a brittle string, one regression at a time, with no way to know if a change actually helped or just moved the failures somewhere you weren’t looking.

DSPy and GEPA are two of the most important answers the field has produced to this problem. DSPy says: stop writing prompts, write programs. GEPA says: let the program learn to write its own prompts, from feedback, in plain language. Together they point at a future where the prompt isn’t a thing you craft by hand at all. It’s a thing your system optimizes.

Let’s unpack both, because they’re genuinely worth understanding even if you never run them.

Dog sitting calmly in a burning room saying “This is fine”

^ a 400-word prompt with eleven contradictory instructions, in production, the night before a demo

Part 1: What DSPy actually is

DSPy comes out of Stanford NLP (the same group behind a lot of the retrieval and language-model-program research of the last few years). The name stands for Declarative Self-improving Python, and its tagline tells you the whole philosophy: “Program, don’t prompt, your LLMs.”

The core idea is a separation of concerns that prompt engineering smears together. When you write a raw prompt, you’re simultaneously specifying three different things in one tangled string:

What you want the model to do (the task)
How you want it to reason about it (chain-of-thought? tool use? just answer?)
The exact wording that happens to coax the model into doing it well

DSPy pulls these apart into three building blocks.

Signatures: declare the task, not the prompt

A signature is a typed declaration of a task’s inputs and outputs. You say what goes in and what comes out, and DSPy figures out the actual prompt text.

class ExtractEvent(dspy.Signature):
    """Extract event details from an email."""
    email: str = dspy.InputField()
    event_name: str = dspy.OutputField()
    date: str = dspy.OutputField()

Notice what’s missing: there’s no “You are a helpful assistant,” no “Return only valid JSON,” no examples, no formatting instructions. You declared the shape of the task. The literal prompt string is now DSPy’s job, not yours.

Modules: pick a strategy, swap it freely

A module wraps a signature with a prompting strategy. This is the “how it reasons” layer, and you change it without touching the task definition.

extract = dspy.Predict(ExtractEvent)        # direct answer
extract = dspy.ChainOfThought(ExtractEvent) # think step by step first
extract = dspy.ReAct(ExtractEvent, tools=[...]) # reason + call tools in a loop

Predict just asks. ChainOfThought makes the model reason before answering. ReAct runs a reasoning-and-tool-use loop. There’s also ProgramOfThought, BestOfN, Refine, and others. The point is that swapping Predict for ChainOfThought is a one-token change, not a prompt rewrite. Your task definition never moves.

You compose these modules into ordinary Python programs: a retrieval step feeds a reasoning step feeds a summarizer, with normal control flow in between. The LLM calls become composable functions instead of monolithic prompt blobs.

Optimizers: the part that makes it “self-improving”

This is the piece that makes DSPy more than a tidy abstraction layer. Because your task is declared (signatures) and your strategy is structured (modules), DSPy can automatically generate and tune the actual prompts against a metric you define.

You hand an optimizer three things: your program, a metric (a function that scores an output), and some training examples. It then searches for the prompt instructions and few-shot demonstrations that maximize your metric. DSPy ships several of these: BootstrapFewShot (which generates its own few-shot examples by running the program and keeping the ones that score well), MIPROv2 (which jointly optimizes instructions and demonstrations), COPRO, SIMBA, and the one this post is really about: GEPA.

The mental model: you write the program once, declaratively. The optimizer compiles it into the specific prompt strings that actually perform. Change models, and you re-optimize instead of re-engineering by hand. That’s the “compiler for prompts” framing you’ll hear people use, and it’s a fair one.

DSPy is in production at Shopify, Databricks, and others, so this isn’t a research toy. But the abstraction is only as good as the optimizer behind it. Which brings us to the interesting part.

Part 2: The optimizer problem, and why GEPA is a big deal

Here’s the question every optimizer has to answer: given that the current prompt is imperfect, how do you find a better one?

Two broad families have dominated.

Reinforcement learning (e.g. GRPO and friends) treats the LLM system like a policy. Run it thousands of times, score each run with a scalar reward, and nudge the weights toward higher reward. It works, but it’s brutally expensive. You’re compressing everything the system did on a given task, all that reasoning, every tool call, the full trajectory, into a single number like 0.0 or 1.0. The model has to infer what it did wrong from a reward signal that contains almost no information about why. That’s why RL needs so many rollouts: it’s learning from an incredibly low-bandwidth teacher.

Earlier prompt optimizers like MIPROv2 are far more sample-efficient, but they largely search over instruction and example combinations guided by scalar scores. Better, but they’re still mostly steering by the number.

GEPA’s insight is almost embarrassingly simple in hindsight: language is a much richer learning signal than a scalar reward, so use it.

GEPA stands for Genetic-Pareto, and it was introduced in the 2025 paper “GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning” (Agrawal et al.), a collaboration across UC Berkeley, Stanford, and others, accepted as an oral at ICLR 2026. The headline claims are worth stating plainly because they’re striking:

It beats GRPO (a strong RL method) by ~6% on average across six tasks, and by up to 20% on individual ones.
It does so using up to 35x fewer rollouts.
It beats MIPROv2, the leading prompt optimizer, by over 10%, including +12% accuracy on AIME-2025 (competition math).

Up to 35x fewer rollouts for better results is the kind of number that makes people re-read the abstract. So how does it pull that off?

Surprised Pikachu face meme

^ the RL team realizing reflective prompt evolution matched their result with 35x less compute

Part 3: How GEPA actually works

GEPA evolves the text of your system, the prompts, instructions, even code, through reflection. Three ideas do the heavy lifting.

Idea 1: Reflect in natural language, not on a scalar

Instead of collapsing a run into a reward number, GEPA keeps the whole execution trace: the inputs, the reasoning the model produced, the tool calls it made, what those tools returned, and where things failed. Then it hands all of that to a reflection LM (the paper recommends a strong model here, something like a frontier model with a generous token budget) and essentially asks: “Here’s what the system did and how it scored. Diagnose what went wrong, and propose a better instruction.”

This is the crux. A reward of 0 tells the model it failed. A trace plus the sentence “the agent retrieved the right document but then ignored the date field, which is why the answer was wrong” tells it failed and exactly how to fix it. That second signal is worth dozens of blind rollouts. GEPA learns high-level rules from a handful of trials precisely because each trial carries so much more information.

Idea 2: Feedback functions, not just metrics

In DSPy’s GEPA implementation, you don’t just write a metric that returns a score. You write one that can also return text, a ScoreWithFeedback object carrying both a number and an explanation. This is where you inject domain knowledge:

In a retrieval task, feed back which documents were retrieved.
In code generation, feed back the actual compiler or test error.
In a multi-stage pipeline, feed back which stage broke and the error it threw.

If you give GEPA no text, it falls back to a useless default (“This trajectory got a score of 0.4”). Give it rich textual feedback, and the reflection LM has something real to reason about. The quality of your feedback function is the single biggest lever on how well GEPA performs. Hold that thought, it’s the bridge to why this matters for production teams.

Idea 3: A Pareto frontier instead of one “best” prompt

Here’s the genetic part. The naive way to evolve a prompt is to always keep the single highest-scoring candidate and mutate that. The problem: you converge to a local optimum and stagnate. A prompt that’s great on average might be terrible on a specific slice of cases, and greedily chasing the average throws away the variant that nailed that slice.

GEPA instead maintains a Pareto frontier: the set of candidate prompts where each one is the best at something, the best on at least one evaluation instance, even if it’s mediocre overall. It then picks which candidate to mutate next probabilistically, favoring the ones that “win” on more instances. This preserves complementary strategies: prompt A is great at billing questions, prompt B is great at refunds, and GEPA can eventually merge their winning lineages rather than forcing you to pick one. It’s evolution with a memory of every niche its population has conquered.

Putting the loop together

Stripped to its skeleton, GEPA’s optimization loop is:

Pick a candidate prompt from the Pareto frontier.
Sample a minibatch of training examples.
Run the program and capture full execution traces.
Score + gather feedback via your metric (number and text).
Reflect: the reflection LM reads the traces and feedback, diagnoses failures, and proposes a new instruction for the targeted module.
Evaluate the mutated candidate on the minibatch; if it looks better, validate it on the full set.
Update the Pareto frontier, optionally merging strong lineages.
Repeat until your budget (auto: "light" | "medium" | "heavy", or a hard cap on metric calls) runs out.

Mutate, reflect, keep what wins on some niche, occasionally cross-breed. It’s natural selection, except the selection pressure is a language model reading its own mistakes and writing itself better instructions.

What this all means

Step back from the mechanics and there are two ideas here that outlast the specific tools.

First: prompts are becoming artifacts you compile, not strings you write. DSPy’s bet is that hand-tuning prompt wording is a transitional craft, the assembly language of LLM programming, and that the durable thing is the declarative program plus the metric. You’ll specify intent and outcomes; an optimizer will produce the wording. We’ve seen this movie before with compilers and with query optimizers. The hand-tuned version never fully dies, but it stops being where most people spend their time.

Second, and more important: the bottleneck shifts from the prompt to the feedback. GEPA’s whole advantage is that it learns from rich, textual, why-laden feedback instead of a thin scalar. Which means the limiting factor on a GEPA-style system isn’t the optimizer anymore. It’s whether you can produce a feedback signal that actually explains why outputs were good or bad. A great optimizer with a return 0.0 metric is a Ferrari with no fuel.

And that’s exactly where most teams hit a wall.

The catch nobody mentions: GEPA is only as good as its feedback

It’s easy to read the GEPA results and think the hard part is solved. Run the optimizer, get +12%, go home. But look again at what the optimizer needs: execution traces and textual feedback that explains why each run succeeded or failed.

On a benchmark like AIME, that feedback is free, the answer is either right or wrong, and the trace is self-contained. In a real product, it’s the opposite. Your “metric” is whether a user actually got what they wanted from a messy, ambiguous, multi-turn conversation. There’s no answer key. The signal you’d want to feed GEPA, “this conversation failed because the agent resolved the wrong intent,” or “the user gave up after the third tool call,” doesn’t exist in your stack as structured text. It’s buried, unlabeled, across thousands of production transcripts.

So you end up with a powerful optimizer and nothing rich to feed it. You fall back to a thin proxy metric, thumbs-up rate, or task-completed-yes/no, and you’ve quietly thrown away the exact thing that made GEPA work: the why.

Hackerman meme typing confidently at multiple screens

^ what running GEPA feels like once your feedback function actually explains why production conversations failed

Where production feedback comes from

This is the part of the loop we think about constantly at Agnost AI, because it’s the part the benchmarks hide.

GEPA wants feedback like “the agent retrieved the right doc but ignored the date,” attached to real cases. That’s not something you write by hand for ten thousand conversations. It has to be generated from production behavior. Which means you need a layer that:

reads every production conversation, not a sample;
infers what the user actually intended, versus what the agent did;
clusters failures by cause (“wrong tool,” “bad retrieval,” “premature give-up,” “resolved the wrong intent”) rather than by topic; and
emits that as structured, textual feedback you can hand straight to an optimizer’s metric function.

That last bullet is the bridge between an LLM observability stack and a GEPA-style improvement loop. The trace tells you what happened. The intent-and-cause layer tells you why it was wrong in user terms. Wire the second one into your feedback function, and GEPA suddenly has fuel that looks like real product failures instead of a benchmark’s clean answer key.

This is roughly the workflow Agnost AI is built around: read production conversations, infer intents and failure causes, and turn them into signals you can actually optimize against, whether that optimization is a human reading a cluster and tightening a tool description, or a GEPA loop rewriting an instruction from feedback you generated. The optimizer is the easy half. The feedback signal is the half that decides whether any of it works.

Wrapping it up

DSPy reframes LLM development from prompt-crafting to programming: declare the task with signatures, choose a reasoning strategy with modules, and let optimizers compile the actual prompts. GEPA is the standout optimizer because it learns from natural-language reflection on execution traces instead of scalar rewards, keeps a Pareto frontier of complementary prompt variants instead of one greedy best, and as a result beats reinforcement learning with up to 35x fewer rollouts and edges out the prior best prompt optimizer by over 10%.

But the lesson that generalizes past either tool is this: the prompt was never the hard part. The feedback is. GEPA works because language is a richer teacher than a number, and the moment you accept that, the question stops being “how do I write a better prompt?” and becomes “where does my high-quality, why-laden feedback come from?”

If you’re building an agent and you want to actually run a loop like this, the missing piece usually isn’t the optimizer. It’s a feedback signal that explains why your real conversations succeed or fail. That’s the thing we built Agnost AI to produce.

Check out Agnost AI if you want the feedback half of the loop, structured intent and failure signals from your real conversations, ready to feed whatever optimizer you’re running.

TL;DR: DSPy lets you program LLMs (signatures + modules + optimizers) instead of hand-tuning prompts. GEPA is its breakout optimizer: it reflects on full execution traces in natural language, evolves a Pareto frontier of complementary prompts, and beats reinforcement learning with up to 35x fewer rollouts and ~+10% over MIPROv2. The real takeaway is that GEPA’s power comes entirely from rich textual feedback, so in production the bottleneck moves from writing prompts to generating a feedback signal that explains why your conversations fail.

Reading Time: ~10 min