GEPA squeezes 7x gains from three examples

Lakshya A. Agrawal took the stage at AI Engineer to pitch reflection over reinforcement, and he brought receipts. His team’s GEPA optimizes prompts and agents not with rewards alone but by reading the whole trace and rewriting the instruction.

The math is stark.

GEPA hit twice the gain of GRPO using just three examples and one round of reflection, versus 25,000 rollouts for the RL baseline, per the talk. Agrawal, presenting the GEPA effort, showed Qwen 38B optimizing itself with no external teacher, then kept compounding the lead with a few more reflective iterations. The point is sample efficiency, when rollouts take hours and call expensive tools, gradients waste the diagnostic signal sitting in chains of thought and tool errors.

The trick is text. Instead of nudging weights, GEPA nudges language, where changing "generate a one-line summary" to "10-line summary" flips behavior in one edit. It mines a rollout for what worked and what failed, can even pull docs or guides, and proposes a new prompt. Under the hood it is an evolutionary loop with a Pareto pool that keeps every candidate that wins on at least one training example, not just the top scorer.

That Pareto pool matters when a loop would stall. In a head-to-head search tree, the naive LLM-in-a-loop kept polishing the same local optimum until the budget ran out, while GEPA’s diverse pool explored more branches and converged higher. Across four benchmarks the pool accounted for more than half the lift, nearly doubling gains versus looping.

The prompts it finds read like specifications, not hacks. For a second hop in multi-hop QA, GEPA learned first-hop docs tend to cover one entity and the second hop should recover related documents, plus when to summarize before answering. For AMD’s new NPU XDNA2 accelerator, where GPT-4 failed for lack of public docs, GEPA lifted an agent from 4.25% to 30.52%, a 7x jump, and even learned to avoid including ADF.h, a library that ships but does not work on that generation.

The same engine generalizes beyond prompts. Agrawal introduced Optimize Anything, a universal API that treats any text artifact as optimizable if you can score it: CUDA kernels with compile and profile feedback, agent harnesses as Python files, or scheduling heuristics with cost and SLA violations. You supply problems and an evaluator that returns a score plus any side information, and it proposes candidates via the same reflect and Pareto selection cycle.

Demos were deliberately cheap. A four-line chain-of-thought starter for ARC-style tasks became a six-step agent in 16 rounds that lifted Gemini Flash from 32.5% to 89.5%, and a two-step agent lifted GPT-4.1 nano by 20% on MATH 500. For coding skills, a GPT-5 mini agent trained with GSkill went from 24% to 93% on Go repo issue resolution, and those same skills applied to Claude Sonnet 4.5 hit 100% while cutting time per issue by almost half. Agrawal also cited external wins: Databricks tuning GPT OSS 120B to beat Claude Opus at 90x lower cost, a 35% OCR error cut for vision models, and early Snorkel gains within 20 hours of release.

For teams shipping agents, the workflow shift is concrete. The standalone gepa-ai/gepa library now ships a GEPAAdapter where you implement evaluate() and make_reflective_dataset() and get the propose-test-reflect loop out of the box, including an MCP adapter that optimizes tool descriptions.

Reflection is not free. It assumes you can collect rich, honest traces and score them, and Agrawal noted the loop bakes in evaluation quirks if your metric is noisy. For subjective tasks his workaround is a flywheel: annotate about 50 production traces, use GEPA to learn an LLM-as-judge prompt, then optimize the agent against that judge.