BlogNovember 15, 2026Code agents · Reinforcement learning12 min read
SWE-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self-Distillation
A software-engineering agent has to find the bug, reproduce it and fix it, but its training signal usually comes from one test result at the end. SWE-Ladder gives a frozen teacher only the slice of the verified repair that the current stage needs, and climbs the stages in repair order.

Contents
TL;DR
Resolving a repository issue takes three skills: localize the faulty code, reproduce the failure, and patch the code. Reinforcement learning trains all three from a single test result on the final patch. On-policy self-distillation adds a teacher that sees privileged evidence, but in practice it is shown the full reference patch at every step, which tells it what to change even while the agent is still working out where to look.
SWE-Ladder splits the verified repair of each training issue into three pieces of evidence: the changed files and lines for localization, the fail-to-pass tests for reproduction, and the diff for patching. A rule-based router labels every step of an end-to-end rollout with its stage from the tool call, a frozen teacher sees only the matching evidence, and the stages are distilled one at a time in repair order while GRPO keeps optimizing the whole trajectory.
With Qwen3-4B-Instruct-2507 on SWE-bench Verified, SWE-Ladder raises pass@1 from 9.6% to 16.4%, against 10.4% for GRPO and 12.4% for a teacher that sees the full patch. On the OpenHands scaffold it goes from 13.0% to 17.6%, where every baseline stays at or below the base model. Reversing the stage order drops it to 11.4%.
1What one reward, or one full patch, misses
An SWE agent edits a repository over dozens of tool calls, and the usual training signal is whether its final patch passes the designated tests. That makes it hard to tell which decision caused a failure. To see what different signals actually teach, we measure three intermediate behaviours on SWE-bench Verified alongside pass@1: whether the agent edits the files changed by the reference repair (localization), whether it writes and runs a script that shows the failure (reproduction), and whether it re-runs that script successfully after its last edit (verification).
Two patterns stand out. The outcome reward does not reliably improve these behaviours: under GRPO, pass@1 barely moves, from 9.6% to 10.4%, while reproduction falls from 59.4% to 48.6%. Showing the teacher the full patch raises pass@1 but lowers post-edit verification, from 22.6% to 16.2%. During a localization step, a teacher that already sees the replacement code is pulled toward editing rather than searching; the same effect has been reported in single-turn reasoning, where a teacher conditioned on the full reference makes the student skip intermediate steps and verify less.
At each step, the teacher should see only the part of the verified repair that the current stage needs, and the stages should be taught in the order a repair uses them: find the code, make the failure concrete, then fix it.
2SWE-Ladder
SWE-Ladder trains a single end-to-end agent. It never constrains which tools the agent calls; it only decides which evidence the teacher sees and which tokens that teacher supervises. Figure 1 plays a real 21-step rollout from the paper through the router, then climbs the three phases of training.
Student rollout · an oauthlib issue, 21 steps, labelled by the router
Frozen teacher · same base model · sees only
Distillation mask on the rollout
L = LGRPO + 0.15 · LOPSD(active stage)
Routing steps to stages
A deterministic, tool-level state machine labels each step. Creating or running a file whose name contains repro, editing a test file or running the test suite proposes REP; editing any other source file proposes PATCH; searches, file views and read-only shell commands are exploration. A proposal only moves the state forward in the order LOC → REP → PATCH, and backtracking depends on the state: in REP, three consecutive explorations return to LOC; in PATCH, a single test run demotes to REP and a single exploration returns to LOC. The labels only route the teacher's context and the distillation mask. They never truncate a rollout or change its reward.
On 2,261 held-out stage-labelled steps, the router agrees with reference labels on 92.0% of them. A learned TF-IDF classifier agrees on 98.63%, yet trains a worse agent: it predicts each step independently and fragments trajectories into isolated labels, while the stateful rules produce contiguous stage blocks.
Stage-specific evidence
All evidence comes from each training issue's verified repair, with nothing generated by an external model. The teacher prompts share the student's system instructions and task prompt; only an appended evidence block differs.
| Stage | What the teacher sees | Added tokens, median / mean |
|---|---|---|
| LOC | changed non-test file paths, old-side hunk ranges and the pre-patch source lines, but not the replacement code | 406 / 544 |
| REP | fail-to-pass test identifiers and names, reference test code, candidate test changes when available | 1,249 / 2,928 |
| PATCH | the verified repair diff | 669 / 907 |
| all three (unrouted) | the union, as a full-reference teacher would see it | 2,709 / 4,375 |
Climbing in repair order
Training runs 70 iterations in three phases, each continuing from the previous checkpoint: 30 for LOC, then 20 for REP, then 20 for PATCH. In each phase, the teacher, a frozen copy of the base checkpoint, is conditioned on that phase's evidence, and a top-16 token-level distillation loss is applied only to response tokens labelled with the active stage, with weight λ = 0.15. GRPO stays active on every valid response token throughout. The order follows how a repair uses the evidence: on-policy distillation only supervises states the student actually reaches, so patch evidence helps only once the student can find the code to change, and the reproduction script is what it later runs to check its patch.
3Results
All methods start from Qwen3-4B-Instruct-2507 without task-specific supervised fine-tuning and train on 1,149 issues from SWE-smith, SWE-ReBench, R2E-Gym and SWE-Gym, with 16 tasks × 4 rollouts per iteration. We evaluate on the 500 issues of SWE-bench Verified, under the R2E-Gym scaffold and the OpenHands scaffold.
SWE-bench Verified, n = 500
pass@1 (%)
On R2E-Gym, SWE-Ladder reaches 16.4% pass@1, against 10.4% for GRPO and 12.4% for GRPO+OPSD. It also takes fewer steps than the base model, 19.3 against 26.5, and completes 98.6% of its trajectories, whereas GRPO completes only 36.0% before the resource limit. SDAR, a gated distillation method given the same patch evidence, falls to 4.6%. On OpenHands, the three baselines end at or below the base model's 13.0% (GRPO 11.2%, GRPO+OPSD 12.4%, SDAR 6.8%), while SWE-Ladder reaches 17.6% with 16.6 steps and 99.4% completion.
4What changes along the ladder
We evaluate the checkpoint at the end of each phase, LOC30, REP50 and PATCH70, and track the three behaviours together with pass@1 (Figure 3). The LOC phase improves all four. The REP phase keeps improving localization, but reproduction and verification fall below the base model. The PATCH phase recovers both, and PATCH70 ends above the base model on all four metrics, the only method in the comparison to do so.
R2E-Gym, end of each phase
Change from the base model (percentage points)
5What matters
Every stage contributes, and routing alone is not enough. Removing LOC, REP or PATCH from the schedule lowers pass@1 from 16.4% to 11.2%, 13.2% or 14.0%, and every such variant falls below the base model on reproduction and verification. Removing LOC hurts most, consistent with localization being the stage the others build on. Training all three stages at once, with each routed token subset distilled from its own evidence but without the order, reaches only 12.4% and loses reproduction (35.6%) and verification (12.6%).
Stage ablation, R2E-Gym
Change from the base model (percentage points)
The order matters. With the same 30/20/20 budget, reversing the order (PATCH → REP → LOC) gives 11.4%, and putting reproduction first (REP → LOC → PATCH) gives 12.6%, even though PATCH is still last. Both also take more steps. The evidence must match the stage it supervises: with a 30-iteration budget, supervising LOC tokens with LOC evidence gives 14.4%, against 13.6% with patch evidence; supervising PATCH tokens with patch evidence gives 13.2%, against 11.8% with localization evidence, which reveals too little to write a fix.
Training order, 70 iterations
pass@1 (%)
Evidence vs supervised stage, 30 iterations
pass@1 (%)
6Training dynamics
Comparing the first and last ten iterations, GRPO's optimization signals deteriorate: mean entropy falls from 0.0300 to 0.0061, gradient norm from 1.820 to 0.187 and training reward from 0.236 to 0.055, while responses nearly double in length, from 8.29k to 16.73k tokens. SWE-Ladder stays steadier: entropy 0.0251 → 0.0221, gradient norm 2.353 → 2.397, and responses grow modestly from 10.09k to 11.62k tokens. GRPO+OPSD and SDAR also keep optimization signal late in training, so preserved entropy or gradient magnitude alone does not explain SWE-Ladder's held-out results. These are single runs without independent seeds.
| First → last 10 iterations | Entropy | Gradient norm | Response tokens |
|---|---|---|---|
| GRPO | 0.0300 → 0.0061 | 1.820 → 0.187 | 8.29k → 16.73k |
| SWE-Ladder | 0.0251 → 0.0221 | 2.353 → 2.397 | 10.09k → 11.62k |
Citation
@article{cai2026sweladder,
title = {SWE-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self-Distillation},
author = {Cai, Xudong and Lu, Zhengxi and Yan, Yuchen and Shao, Jian and Xiao, Jun and Zhuang, Yueting and Shen, Yongliang},
journal = {arXiv preprint arXiv:2610.33667},
year = {2026}
}References
- Jimenez et al. (2024). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR.
- Xia et al. (2024). Agentless: Demystifying LLM-based Software Engineering Agents.
- Jain et al. (2025). R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents.
- Wang et al. (2025). OpenHands: An Open Platform for AI Software Developers as Generalist Agents. ICLR.
- Shao et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (GRPO).
- Agarwal et al. (2024). On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. ICLR.
Training configuration, reward shaping and the full evidence examples are in the paper.