First run
Lock config, run a task, find the result directory.
The dataset argument is always a dataset root (the directory with ageval.yaml) or a registry ref (<dataset_id>@<version>), never tasks/<id>/.
Run from the Hub
ageval registry list
ageval lock official/minimal-demo@0.1.3 --task terminal-jsonl-agg
ageval run official/minimal-demo@0.1.3 --task terminal-jsonl-agg
ageval lock <org>/<name>@<version> --task <task-id>
ageval run <org>/<name>@<version> --task <task-id>Login and registry list hit the public Hub (https://ageval.zjureal.com) by default — you do not need AGEVAL_REGISTRY_URL first. A registry ref downloads the published dataset into the local cache (ageval cache inspects / purges it), then executes under the locked config. Default profiles mostly use docker, which needs a local Docker engine; a live coding-agent run also needs an ACP entry and credentials.
In-repo examples
The public smoke examples/datasets/minimal-demo ships with a repo checkout:
Before you run
minimal-demo defaults to docker, so make sure the Docker engine is up. ageval executors -v should show your agent engine_ready / acp_entry_ready; a missing key shows credential_missing.
ageval tasks examples/datasets/minimal-demo
ageval lock examples/datasets/minimal-demo --task terminal-jsonl-agg
ageval run examples/datasets/minimal-demo --task terminal-jsonl-agg
ageval run examples/datasets/minimal-demo --task terminal-jsonl-agg --probeOn a TTY, ageval run prints a short recap (single task: logs; full dataset: summary path). A pipe or --json still writes the JSON document. Evidence lives under .ageval/runs/<id>/.
| Term | Meaning |
|---|---|
| dataset | Root ageval.yaml plus task members |
| Attempt | One execution after lock; directory .ageval/runs/<id>/ |
| environment | local / Docker / e2b / ssh / daytona |
ageval executors -v
ageval view examples/datasets/minimal-demo --no-browser--probe is lock + preflight only. --profiles replaces the profiles file. --agent and --profiles are mutually exclusive. --model requires --agent.