Repository examples
When to use minimal-demo, Agent packages, or a dataset converted from an upstream bench.
| Path | Use when |
|---|---|
examples/datasets/minimal-demo | Dialog / terminal / multi-agent cases; docker by default |
examples/agents | dsh / nooa / miniswe / terminus-2 / openhands overlay packages for --agent (builtin ACP is --agent pi) |
examples/datasets/tau3-airline-5 | Abbreviated five-task airline conversion: dual roles + separate scoring |
List members with ageval tasks <path>. Copy layout from minimal-demo when authoring.
tau3-airline-5
This dataset ports the tau2-bench (τ³-bench) airline domain onto ageval’s outer lifecycle: dual-role user / service dialog over ACP, tools and DB via an in-package bridge, PASS only from evaluator.py (not run.py completion). This repository ships five members (airline-00 … airline-04); shared bridge code and domain assets live under root shared/lib and shared/assets so each task need not ship its own lib/ copy.
Full runs need the tau2 pin in the package requirements.txt; ageval lock does not. Package notes: examples/datasets/tau3-airline-5/README.md. Shipping the package does not make the results more credible.
Datasets published only on Hub
examples/datasets/ holds minimal-demo and a five-task tau3-airline-5 cut. The full 50-task airline domain and other popular-bench conversions are not checked into this repository; they ship via Hub publish plus a whole-dataset upload so clone size and CI paths stay bounded:
| Upstream | Hub package id (org my-lab) | Notes |
|---|---|---|
| Terminal-Bench 2.0 | terminal-bench-2 / light terminal-bench-2-10 | docker + Harbor-style pytest verifier |
| MARBLE coding | marble-coding / light marble-coding-10 | docker shared-container multi-agent coding |