Learning to Localize, Reproduce, and
Patch via Progressive Privileged Self-Distillation
One end-to-end software-engineering agent, trained with stage-specific evidence in repair order. No
teacher, no stage labels, and no fixed pipeline at test time.
✓ one policy at test time: no teacher, no stage labels
// the idea
Show the teacher only what each stage needs, and teach the stages in repair order.
teacher_evidence.diffOPSD → SWE-Ladder
-Teacher sees the full reference patch at every step: the same evidence at every stage.
+LOC steps get the changed files and line ranges.
+REP steps get the fail-to-pass tests.
+PATCH steps get the reference diff, one stage at a time, in that order.
Reinforcement learning with verifiable rewards gives one signal for the final patch. On-policy
self-distillation (OPSD) adds dense token-level guidance, but a teacher shown the full reference patch at
every step gives the same evidence at every stage. SWE-Ladder turns each verified repair
into three stage-specific evidence views and distills them one stage at a time, training a single
end-to-end agent that sees no privileged evidence at inference time.
Rollout · GRPO
repo+issue
student⇄env
no privileged evidence
<function=search>"proxy.get" oauthlib/
a1
a2
a3
a4
a5
a6
Test results✓✓✗✓
group of rollouts
Group-relative reward
LGRPO
every valid token
Rule-based stage router
LOC→REP→PATCH
Forward in repair order, with state-dependent backtracking.
A rule-based router labels each step of an end-to-end rollout as LOC,
REP, or PATCH. In each phase, a frozen teacher receives only
that stage's evidence and distills only that stage's tokens, while GRPO keeps optimizing the full
trajectory. Example contents are illustrative, as in the paper figure.
Paper figure
rung 01 · method
Three evidence views, one router, one stage at a time.
Stage-specific evidence
The verified repair and its tests are converted into three evidence views. The student never sees any of
them. All teacher variants share the student's system and task instructions; only the appended evidence
block differs.
stagerouter seesteacher evidencedistilled tokens
LOCSearch, file view, read-only shellChanged non-test files, old-side hunk ranges, and pre-patch source linesResponse tokens labeled LOC during the LOC phase
REPRepro script creation/execution, test file edits, test runsFail-to-pass test identifiers/names, reference test code, and candidate test changesResponse tokens labeled REP during the REP phase
PATCHSource-file editsThe verified repair diffResponse tokens labeled PATCH during the PATCH phase
# Stage labels are annotations for distillation only. They do not constrain
tool choices, truncate rollouts, or change the reward.
The 70-iteration total matches the GRPO control: both runs take the same 70 policy updates on the same
data and differ only in where privilege is applied.
Objective
Only the active stage's tokens get the OPSD term.
L = LGRPO + λ · LOPSD
λ
0.15
teacher top-K
16
teacher
frozen copy of the base checkpoint
OPSD term
stop-gradient top-K reverse-KL surrogate
per iteration
16 tasks × 4 rollouts = 64 trajectories
entropy bonus
0.005
The stage router
The router is a deterministic tool-level state machine. It consumes only the parsed tool call and advances
forward in repair order, with state-dependent backtracking.
REP → LOC after three consecutive exploration actions.
PATCH → REP after one test run.
PATCH → LOC after one exploration action.
LOC → PATCH on a patch proposal, skipping REP.
rung 02 · results
Higher pass@1, shorter trajectories, and agents that finish on their own.
All methods start from Qwen3-4B-Instruct-2507 and are evaluated on the 500 instances of SWE-bench
Verified without task-specific supervised fine-tuning before RL. GRPO+OPSD shows the teacher the
full reference patch at every step; SDAR is a gated OPSD method for agents, given the same patch
evidence.
SWE-bench Verified, 500 instances
R2E-Gym scaffold · 64K context · max 150 steps · temperature 0.7
methodpass@1 (%) ↑avg. steps ↓env_done (%) ↑
Base
GRPO
GRPO+OPSD full patch
SDAR
SWE-Ladder
Ablations (R2E-Gym, pass@1 %)
Every stage matters, and so does the order. Dashed line: full method.
full method
SWE-Ladder16.4
remove one stage
without LOC11.2
without REP13.2
without PATCH14.0
change the schedule
all stages at once12.4
reversed order11.4
REP first12.6
Agreement is not utility
Router agreement with held-out stage labels.
92.0%Rule-based routerused by SWE-Ladder
98.63%Neural routertrained on Qwen3.8-Max annotations
!The neural router agrees with the labels more often, yet trains a worse agent:
annotation agreement alone does not determine training utility.
rung 03 · analysis
The privileges are not interchangeable.
A causal probe restores real environments from base-model trajectories, adds one kind of privilege, and
executes the candidate next actions in containers. Each privilege helps most in its own stage.
Effect of each privilege on the base policy's next action
Shading is relative within each column; the matched-stage privilege is outlined.
Paired one-sided Wilcoxon: LOC vs. none p = 0.029, LOC vs. whole-answer p = 0.035; REP vs. none p = 0.004,
REP vs. whole-answer p = 0.014. Samples are small (93 and 46 moments) and gold-file contact is a proxy for
localization.
The routed student stays anchored to its stage teachers
Stage-matched reverse KL to each privileged teacher (lower is closer).
After 70 RL steps the unrouted student drifts away from all three stage teachers.
Stage funnel on SWE-bench Verified
Behavioral rungs over 500 trajectories each; highlighted is best in column.
loc recallreproduceverifysolvebase0.3360.5940.2260.096whole-answer, 70 steps0.3800.6940.1620.116routed, 70 steps0.4400.5870.2560.141
Only solve is test-verified; the other rungs are heuristics over trajectory text.
Training dynamics on SWE-bench Verified
Localization, reproduction, verification, and solve rate for SWE-Ladder, SDAR, GRPO+OPSD, and the
all-at-once variant. Vertical guides mark the LOC, REP, and PATCH phases of SWE-Ladder.
top of the ladder · takeaways
Don’t show the teacher the whole answer at every step. Give localization,
reproduction, and patching each its own evidence,
in the order a repair happens.
A teacher shown the full reference patch at every step gives every stage the same evidence. Routing the evidence by stage does better.
12.4% → 16.4%GRPO+OPSD (full patch) vs. SWE-Ladder, R2E-Gym pass@1
#02
Order matters.
Teaching the stages all at once, in reverse, or reproduction first all fall short of repair order.
11.4–12.6%other schedules vs. 16.4% in repair order
#03
Every stage contributes.
Removing any one stage of privileged supervision lowers pass@1.
11.2 / 13.2 / 14.0without LOC / REP / PATCH
#04
A simple router trains a better agent.
A neural router matches annotator labels more often but yields lower downstream pass@1.
92.0% vs. 98.63%rule-based vs. neural agreement
#05
Shorter rollouts that end on their own.
No teacher, no stage labels, and no fixed pipeline at test time: one end-to-end policy.
19.3 steps · 98.6%avg. steps and env_done on R2E-Gym
// code
Train a stage-routed agent.
The stage router, privileged-context builders, top-K distillation loss, 24 run definitions, and
evaluation on SWE-bench Verified. Built on
SWE-Master (which vendors rllm and R2E-Gym),
OpenHands, and
verl.
# graft onto the pinned upstream workspace$ bash integration/install.sh --workspace "$WS"
$ python3 integration/check_install.py --workspace "$WS"
# render and launch the smoke test$ python3 scripts/tools/render_config.py \
training/configs/smoke_stage_routing.yaml \
-o /tmp/run.yaml
// cite
citation.bib · arXiv:2610.33667
@article{cai2026sweladder,
title = {SWE-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self-Distillation},
author = {Cai, Xudong and Lu, Zhengxi and Yan, Yuchen and Shao, Jian and Xiao, Jun and Zhuang, Yueting and Shen, Yongliang},
journal = {arXiv preprint arXiv:2610.33667},
year = {2026}
}