Skip to content
Agent-R2

Agent-R² Scaling Agents for Realistic Productivity Work via Real-Source Data Synthesis

Teng Pan1,2, Zixuan Wang1,2, Yuchen Yan1, Qifei Wu2, Yao Chen2, Yingqi Qu2, Ruiqing Zhang2, Yongliang Shen1,†

1Zhejiang University2Baidu Inc.†Corresponding author

4k from 10k
task instances distilled from web-corpus candidates
51.0 vs 48.0
GDPval Pass@1, Agent-R²-4B vs Qwen3-235B-A22B-Thinking-2507
14.8 vs 13.8
OfficeQA Pass@1 (out of domain), same comparison
+18.6
GDPval Pass@1 points over the Qwen3-4B-Thinking-2507 base

The data bottleneck

Real work needs data that reflects real work.

Productivity tasks ask an agent to follow a long-horizon professional workflow, read heterogeneous files and hand back a usable deliverable. Existing training data covers these skills one at a time. The paper argues that what is missing is not more data but data that reflects real work: real intents, authentic files, and deliverable-level standards.

Workflow manually authored workflows, which do not scale blueprints mined from a web corpus with occupational activity seeds (O*NET)
Files LLM-generated files without real layouts, gaps or cross-file references public files found on the web, verified, and kept in their original format
Success “an output file exists” fine-grained rubrics over evidence, analysis and deliverable quality

Method

From an occupational seed to a trained agent

Three synthesis stages turn a web-corpus segment into a task with real attachments and a rubric; an expert solves it to produce reference files and a trajectory; the rubric is checked and refined. The model is then trained with SFT and rubric-based RL.

01

Workflow extraction

Stage 1

Web corpus C

DLR-Web segments (filtered from FineFineWeb); each carries discipline metadata, mapped to an occupational category.

Occupational activity seeds G · O*NET

  • Examine legal data to determine advisability of defending or prosecuting law
  • Evaluate findings and develop strategies in preparation for presentation of cases.
πsyn · Qwen3.6-Plus workflow filter file-oriented filter

Workflow blueprint W

  1. Analyze legal precedents from similar contested elections to assess …
  2. Examine Virginia election records and certification documents …

+ information dependencies · required file types · expected outputs

02

Attachment searching

Stage 2
web_search Virginia recount contest procedures filetype:pdf

[PDF] Recounts and Contests - Virginia Department of Elections

“Virginia election law does not allow a losing candidate …”

title & snippet relevant → download

bashview_image inspect cover page and table of contents drop files > 50 pages or > 2 MB
  • PDF15_Recounts_and_Contests_2024_FINAL.pdf
  • DOCX2022-DPS-Factsheets-Voting.docx
  • XLSX2024_Policy_Survey_Codebook.xlsx
Satisfy? yes files jointly cover W’s information and operation needs → keep (W, Fc) no after max turns → discard
03

Instruction & rubric

Stage 3
sandbox paginated files → page images · spreadsheets → sheet tables · originals kept

Explore & design · πsyn, multi-turn

Inspects files, searches and fetches web evidence when needed, and chooses the attachment set Fa ⊆ Fc.

PDFDOCXXLSX

Instruction I

“…(Background)… Prepare a comprehensive legal memorandum analyzing the viability of an election contest and developing a litigation strategy. …(Details)”

Rubric R

  • Correctly identifies that a recount petition is permissible because the 0.8% margin is within the 1% threshold specified in Virginia law
  • Legal citations reference both the Virginia Code sections and the Bost case with pinpoint citations
04

Reference & refine

Stage 3

Expert πexp · Claude-Opus-4.5 solves (I, Fa, R)

  1. bashparse the recount PDF and the codebook sheet
  2. view_imagecheck rendered pages
  3. web_searchlocate the case law cited
  4. web_fetchread the Virginia Code section
  5. bashwrite the memorandum

trajectory τ · 18.0 steps, 41.8k tokens on average

Frreference deliverable · legal memorandum

Verify Fr against R

  • recount threshold item
  • pinpoint citations item

unmet and unreasonable → revise the rubric
unmet but reasonable → expert revises Fr once

Final task

(I, Fa, Fr, R) + verified τ

4k instances kept from 10k corpus candidates

05

SFT, then rubric-based RL

Training

SFT · 3k tasks

Qwen3-4B-Thinking-2507 imitates the expert trajectories (split into per-turn samples, 2 epochs) → Agent-R²-4B-SFT

RL · 1k tasks · GRPO in the sandbox, group of 4

  • πa1 a2 … atFd1
  • πa1 a2 … atFd2
  • πa1 a2 … atno file
  • πa1 a2 … atFd4

Render deliverables for the judge

PDFPPTXDOCX → page images

XLSX→ recalculated values + formulas (Markdown) + page images

MLLM-as-a-judge · Qwen3.6-Plus

scores I + Fd against rubric R and reference Fr

  • r10.74
  • r20.52
  • r30
  • r40.81

r = Score(Fd, Fr, R) / max score, and r = 0 if no deliverable

Policy update

A1 +A2 ≈ 0A3 −A4 +

Group advantages update the policy, with a KL term to the reference model. Outcome reward only, no penalty on tool-call errors → Agent-R²-4B

Example task from the paper’s Figure 2 (a Virginia election contest). The search query, trajectory steps, verification marks and reward values are illustrative.

Paper figures
Paper Figure 2: Agent-R² data synthesis pipeline with Stage 1 Workflow Extraction, Stage 2 Attachment Searching and Stage 3 Task Synthesis.
Data synthesis pipeline: workflow blueprints from web-corpus evidence, verified public attachments, and instructions, rubrics and references synthesized in the sandbox.
Paper Figure 3: rubric-based RL. Multi-turn rollouts produce deliverables that are rendered and scored by an MLLM judge; group advantages and KL divergence drive the policy update.
Rubric-based RL: rendered deliverables are scored by an MLLM judge against rubrics and references; the policy is updated with group advantages and KL regularization.

Setup

Task, environment and training recipe

Task

A task is x = (I, Fa, Fr, R): instruction, attachments available while solving, reference files used only for evaluation, and a task-specific rubric. The agent must produce the deliverable files Fd that I asks for.

Environment

A persistent sandbox with four general tools: bash, web_search, web_fetch, view_image. Text-only solvers answer view_image through an auxiliary vision model (Qwen3-VL-4B-Thinking for Agent-R² variants).

Recipe

3k tasks for SFT and 1k disjoint tasks for RL. GRPO with batch size 16, group size 4, up to 40 assistant turns, 65,536 max response length, temperature 1.0, learning rate 1×10−6. Final checkpoints, no benchmark-based selection.

Results

A 4B agent ahead of a 235B one

GDPval is the in-domain benchmark, split by the type of deliverable it asks for: office tasks, and non-office tasks (audio, video and other media). Pass@k allows up to k attempts and scores the first attempt that produces a deliverable.

GDPval · Table 1

All six columns
ModelOfficeNon-OfficeFull
P@1P@3P@1P@3P@1P@3

OfficeQA · out of domain · Table 2

Document-grounded questions over office source files (official Markdown version). Pass@k: at least one of k attempts is correct.

ModelPass@1Pass@3
Qwen3-4B-Thinking-25072.24.5
Qwen3-30B-A3B-Thinking-25072.95.7
Qwen3-235B-A22B-Thinking-250713.825.6
Agent-R²-4B-SFT11.421.7
Agent-R²-4B14.826.8

Agent-R²-4B: 14.8 vs 13.8 Pass@1 and 26.8 vs 25.6 Pass@3 against Qwen3-235B-A22B-Thinking-2507; the base model scores 2.2.

Analysis

Fewer shortcuts, longer grounded exploration

Shortcut behavior: the agent writes a plausible-looking deliverable before it has grounded, built or verified it. The base model makes few tool calls and stops early; SFT teaches sustained exploration; rubric-based RL turns that into reliable completion.

Behavior on GDPval · Figure 4

BaseSFTRL
Completion rate (%)86.878.290.9
Tool-call steps6.739.033.1
Tool-call errors2.8816.312.9

SFT lengthens exploration (and its errors); RL keeps the depth, cuts errors and raises completion.

Direct evidence on shortcuts · Table 3 (%)

BaseSFTRL
Re-checking the deliverable ↑0.591.795.8
Writing despite an unfixed error ↓55.016.011.0
Numerical-error fix rate ↑60.779.685.7
Deliverables with placeholder text ↓12.59.65.7
Traceable numbers in deliverables ↑31.234.638.8

Ablations · GDPval Pass@1

Real vs. synthetic files SFT, 1k matched pairs

synthetic files41.4

real files43.3

Expert trajectories use 2.54× more tool calls on real files (20.4 vs 8.0).

Rubric reward RL from the same SFT model

w/o rubrics49.5

w/ rubrics51.0

Fine-grained rubrics add 1.5 points over the rubric-free reward.

More data or RL? Table 10

4k SFT45.5

3k SFT + 1k RL51.0

Imitating all 4k trajectories stops at 45.5 (3k SFT: 45.3); the gain comes from RL.

Data

What the synthesized tasks contain

Real web-sourced attachments · Table 7

TypeCountAvg size (MB)Mean pages
PDF5,0101.0015.0
DOCX3,1490.2513.7
XLSX1,2580.20–
PPTX3900.7823.6

The mix follows what is actually available on the web: more PDFs, fewer presentations.

2.51
attachments per task on average
15.3
rubric items per task (69.6% content, 30.4% form)
18.0
steps per expert trajectory, 41.8k tokens on average
3k + 1k
tasks for SFT and for RL, with no overlap

Takeaways

  1. Ground every stage in real sources. Workflows come from web-corpus evidence and occupational seeds, files from the web, and rubrics from the verified workflow-file context.
  2. Real files matter. Matched tasks with real attachments train a better SFT model than synthetic files (43.3 vs 41.4) and demand 2.54× more expert tool calls.
  3. Rubrics make RL count. Scoring rendered deliverables against fine-grained rubrics lifts Agent-R²-4B from 45.3 (SFT) to 51.0 on GDPval and reduces shortcut behavior.

Quick start

Run the pipeline

Fill URLs, keys and the proxy once in Agent-R2/settings/settings.yaml; every module reads it. See the README for data placeholders and the sandbox.

1 · Synthesize data

cd Agent-R2/data_sythesis
python run.py   # scenario → search → question → solver → refine
python prepare_sft_data_split.py   # → data/sft.jsonl
python prepare_rl_data.py          # → rl_data/data{N}.zip

2 · SFT (ms-swift)

cd ms-swift
MODEL_PATH=/path/to/Qwen3-4B-Thinking-2507 bash run_qwen3.sh

3 · Rubric-based RL (verl)

cd verl
python recipe/agent_r2/preprocess.py
MODEL_PATH=/path/to/sft_checkpoint bash recipe/agent_r2/run_agent_r2.sh

4 · Evaluate

cd Agent-R2
python gdpval/run.py
python officeqa/run.py

Citation

BibTeX

@article{pan2026agentr2,
  title   = {Agent-R$^2$: Scaling Agents for Realistic Productivity Work via Real-Source Data Synthesis},
  author  = {Pan, Teng and Wang, Zixuan and Yan, Yuchen and Wu, Qifei and Chen, Yao and Qu, Yingqi and Zhang, Ruiqing and Shen, Yongliang},
  journal = {arXiv preprint arXiv:2611.08219},
  year    = {2026}
}