BlogNovember 15, 2026Code agents · Reinforcement learning12 min read

SWE-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self-Distillation

A software-engineering agent has to find the bug, reproduce it and fix it, but its training signal usually comes from one test result at the end. SWE-Ladder gives a frozen teacher only the slice of the verified repair that the current stage needs, and climbs the stages in repair order.

A student robot climbs a ladder with LOC, REP and PATCH rungs toward a green check mark, while a frozen teacher robot holds up only a failing-test card; the localization card and a face-down patch card wait at its feet
Contents
  1. TL;DR
  2. What one reward misses
  3. SWE-Ladder
  4. Results
  5. Along the ladder
  6. What matters
  7. Training dynamics
  8. Citation

TL;DR

Resolving a repository issue takes three skills: localize the faulty code, reproduce the failure, and patch the code. Reinforcement learning trains all three from a single test result on the final patch. On-policy self-distillation adds a teacher that sees privileged evidence, but in practice it is shown the full reference patch at every step, which tells it what to change even while the agent is still working out where to look.

SWE-Ladder splits the verified repair of each training issue into three pieces of evidence: the changed files and lines for localization, the fail-to-pass tests for reproduction, and the diff for patching. A rule-based router labels every step of an end-to-end rollout with its stage from the tool call, a frozen teacher sees only the matching evidence, and the stages are distilled one at a time in repair order while GRPO keeps optimizing the whole trajectory.

With Qwen3-4B-Instruct-2507 on SWE-bench Verified, SWE-Ladder raises pass@1 from 9.6% to 16.4%, against 10.4% for GRPO and 12.4% for a teacher that sees the full patch. On the OpenHands scaffold it goes from 13.0% to 17.6%, where every baseline stays at or below the base model. Reversing the stage order drops it to 11.4%.

1What one reward, or one full patch, misses

An SWE agent edits a repository over dozens of tool calls, and the usual training signal is whether its final patch passes the designated tests. That makes it hard to tell which decision caused a failure. To see what different signals actually teach, we measure three intermediate behaviours on SWE-bench Verified alongside pass@1: whether the agent edits the files changed by the reference repair (localization), whether it writes and runs a script that shows the failure (reproduction), and whether it re-runs that script successfully after its last edit (verification).

Two patterns stand out. The outcome reward does not reliably improve these behaviours: under GRPO, pass@1 barely moves, from 9.6% to 10.4%, while reproduction falls from 59.4% to 48.6%. Showing the teacher the full patch raises pass@1 but lowers post-edit verification, from 22.6% to 16.2%. During a localization step, a teacher that already sees the replacement code is pulled toward editing rather than searching; the same effect has been reported in single-turn reasoning, where a teacher conditioned on the full reference makes the student skip intermediate steps and verify less.

The idea

At each step, the teacher should see only the part of the verified repair that the current stage needs, and the stages should be taught in the order a repair uses them: find the code, make the failure concrete, then fix it.

2SWE-Ladder

SWE-Ladder trains a single end-to-end agent. It never constrains which tools the agent calls; it only decides which evidence the teacher sees and which tokens that teacher supervises. Figure 1 plays a real 21-step rollout from the paper through the router, then climbs the three phases of training.

Student rollout · an oauthlib issue, 21 steps, labelled by the router

LOCREPPATCH

 

testsOne verifier result for the whole rollout → GRPO updates every response token, whatever its stage.
Phase 3 · PATCHiterations 51–70
Phase 2 · REPiterations 31–50
Phase 1 · LOCiterations 1–30

Frozen teacher · same base model · sees only

Choose a phase

Distillation mask on the rollout

L = LGRPO + 0.15 · LOPSD(active stage)

Figure 1. SWE-Ladder on one rollout. Top: the 21 steps of an oauthlib repair (paper Table 7), grouped as in the paper and labelled LOC, REP or PATCH by the rule-based router from each tool call. The labels are not monotone: re-running the reproduction script while patching demotes the state to REP (step 12), and a third consecutive inspection returns it to LOC (step 15). Bottom: the three training phases. In each phase a frozen copy of the base model sees one stage's evidence and distils only the steps with that label; GRPO still trains on every step. Evidence blocks are the schematic examples of the paper's Figure 3.

Routing steps to stages

A deterministic, tool-level state machine labels each step. Creating or running a file whose name contains repro, editing a test file or running the test suite proposes REP; editing any other source file proposes PATCH; searches, file views and read-only shell commands are exploration. A proposal only moves the state forward in the order LOC → REP → PATCH, and backtracking depends on the state: in REP, three consecutive explorations return to LOC; in PATCH, a single test run demotes to REP and a single exploration returns to LOC. The labels only route the teacher's context and the distillation mask. They never truncate a rollout or change its reward.

On 2,261 held-out stage-labelled steps, the router agrees with reference labels on 92.0% of them. A learned TF-IDF classifier agrees on 98.63%, yet trains a worse agent: it predicts each step independently and fragments trajectories into isolated labels, while the stateful rules produce contiguous stage blocks.

Stage-specific evidence

All evidence comes from each training issue's verified repair, with nothing generated by an external model. The teacher prompts share the student's system instructions and task prompt; only an appended evidence block differs.

StageWhat the teacher seesAdded tokens, median / mean
LOCchanged non-test file paths, old-side hunk ranges and the pre-patch source lines, but not the replacement code406 / 544
REPfail-to-pass test identifiers and names, reference test code, candidate test changes when available1,249 / 2,928
PATCHthe verified repair diff669 / 907
all three (unrouted)the union, as a full-reference teacher would see it2,709 / 4,375

Climbing in repair order

Training runs 70 iterations in three phases, each continuing from the previous checkpoint: 30 for LOC, then 20 for REP, then 20 for PATCH. In each phase, the teacher, a frozen copy of the base checkpoint, is conditioned on that phase's evidence, and a top-16 token-level distillation loss is applied only to response tokens labelled with the active stage, with weight λ = 0.15. GRPO stays active on every valid response token throughout. The order follows how a repair uses the evidence: on-policy distillation only supervises states the student actually reaches, so patch evidence helps only once the student can find the code to change, and the reproduction script is what it later runs to check its patch.

3Results

All methods start from Qwen3-4B-Instruct-2507 without task-specific supervised fine-tuning and train on 1,149 issues from SWE-smith, SWE-ReBench, R2E-Gym and SWE-Gym, with 16 tasks × 4 rollouts per iteration. We evaluate on the 500 issues of SWE-bench Verified, under the R2E-Gym scaffold and the OpenHands scaffold.

SWE-bench Verified, n = 500

pass@1 (%)

Figure 2. Results on two agent scaffolds (paper Table 2). env_done is the share of trajectories that finish before the resource limit; for average steps, lower is better. The dashed line marks the base model. GRPO+OPSD and SDAR use the same full-patch evidence as each other.

On R2E-Gym, SWE-Ladder reaches 16.4% pass@1, against 10.4% for GRPO and 12.4% for GRPO+OPSD. It also takes fewer steps than the base model, 19.3 against 26.5, and completes 98.6% of its trajectories, whereas GRPO completes only 36.0% before the resource limit. SDAR, a gated distillation method given the same patch evidence, falls to 4.6%. On OpenHands, the three baselines end at or below the base model's 13.0% (GRPO 11.2%, GRPO+OPSD 12.4%, SDAR 6.8%), while SWE-Ladder reaches 17.6% with 16.6 steps and 99.4% completion.

4What changes along the ladder

We evaluate the checkpoint at the end of each phase, LOC30, REP50 and PATCH70, and track the three behaviours together with pass@1 (Figure 3). The LOC phase improves all four. The REP phase keeps improving localization, but reproduction and verification fall below the base model. The PATCH phase recovers both, and PATCH70 ends above the base model on all four metrics, the only method in the comparison to do so.

R2E-Gym, end of each phase

Change from the base model (percentage points)

Figure 3. Localization, reproduction, verification and pass@1 at the base model and at the end of each phase (paper Figure 7). Base values are 33.6%, 59.4%, 22.6% and 9.6%; hover a point for the absolute value. Step through the checkpoints or press play to climb the ladder.

5What matters

Every stage contributes, and routing alone is not enough. Removing LOC, REP or PATCH from the schedule lowers pass@1 from 16.4% to 11.2%, 13.2% or 14.0%, and every such variant falls below the base model on reproduction and verification. Removing LOC hurts most, consistent with localization being the stage the others build on. Training all three stages at once, with each routed token subset distilled from its own evidence but without the order, reaches only 12.4% and loses reproduction (35.6%) and verification (12.6%).

Stage ablation, R2E-Gym

Change from the base model (percentage points)

SWE-Ladderfull SWE-Ladder, for reference
Figure 4. Stage ablation (paper Table 3). Each variant drops one stage from the schedule, or trains all three at once; outlines mark the full SWE-Ladder. Only the full ladder improves all four metrics.

The order matters. With the same 30/20/20 budget, reversing the order (PATCH → REP → LOC) gives 11.4%, and putting reproduction first (REP → LOC → PATCH) gives 12.6%, even though PATCH is still last. Both also take more steps. The evidence must match the stage it supervises: with a 30-iteration budget, supervising LOC tokens with LOC evidence gives 14.4%, against 13.6% with patch evidence; supervising PATCH tokens with patch evidence gives 13.2%, against 11.8% with localization evidence, which reveals too little to write a fix.

Training order, 70 iterations

pass@1 (%)

Evidence vs supervised stage, 30 iterations

pass@1 (%)

Figure 5. Left: three stage orders with the same budget (paper Table 4); average steps per trajectory are shown under each bar. Right: matched against swapped evidence for LOC and PATCH targets (paper Table 5).

6Training dynamics

Comparing the first and last ten iterations, GRPO's optimization signals deteriorate: mean entropy falls from 0.0300 to 0.0061, gradient norm from 1.820 to 0.187 and training reward from 0.236 to 0.055, while responses nearly double in length, from 8.29k to 16.73k tokens. SWE-Ladder stays steadier: entropy 0.0251 → 0.0221, gradient norm 2.353 → 2.397, and responses grow modestly from 10.09k to 11.62k tokens. GRPO+OPSD and SDAR also keep optimization signal late in training, so preserved entropy or gradient magnitude alone does not explain SWE-Ladder's held-out results. These are single runs without independent seeds.

First → last 10 iterationsEntropyGradient normResponse tokens
GRPO0.0300 → 0.00611.820 → 0.1878.29k → 16.73k
SWE-Ladder0.0251 → 0.02212.353 → 2.39710.09k → 11.62k

Citation

@article{cai2026sweladder,
  title   = {SWE-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self-Distillation},
  author  = {Cai, Xudong and Lu, Zhengxi and Yan, Yuchen and Shao, Jian and Xiao, Jun and Zhuang, Yueting and Shen, Yongliang},
  journal = {arXiv preprint arXiv:2610.33667},
  year    = {2026}
}

References

  1. Jimenez et al. (2024). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR.
  2. Xia et al. (2024). Agentless: Demystifying LLM-based Software Engineering Agents.
  3. Jain et al. (2025). R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents.
  4. Wang et al. (2025). OpenHands: An Open Platform for AI Software Developers as Generalist Agents. ICLR.
  5. Shao et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (GRPO).
  6. Agarwal et al. (2024). On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. ICLR.

Training configuration, reward shaping and the full evidence examples are in the paper.