Hub

Hub (apps/hub) is the web UI on top of Registry: browse shared datasets, plugins, agents, and run results. The public instance is ageval.zjureal.com. Sign in with Sign in with GitHub in the browser; the CLI stays on Device Flow.

Local dev

# Terminal A — Registry first (OAuth configured)
export VITE_REGISTRY_PROXY_TARGET=http://127.0.0.1:8700
uv run --extra registry python -m services.registry.app --host 127.0.0.1 --port 8700

# Terminal B
cd apps/hub
pnpm install
pnpm dev   # http://127.0.0.1:5174

Or docker compose -f services/registry/docker-compose.yml up -d for Hub at http://127.0.0.1:8080 plus Registry. Release images live on GHCR: ghcr.io/zju-real/ageval-hub and ageval-registry.

Path	Page
`/`	Redirects to `/leaderboard`
`/home`	Personal home (signed in)
`/leaderboard`	Default landing. Leaderboard plaza: index of listed Dataset suites and Agent / Model Performance (Dataset view by default; `?view=agent` / `?view=model`)
`/datasets`	List (Explore / Your organizations, Explore default)
`/datasets/:id`	README · Tasks · Shared (when `shared/` present) · Overlays (when `overlays:` is declared) · Hosted · Leaderboard
`/datasets/:id/tasks/:task`	README · Files (Local \| Shared \| Overlays) · Jobs
`/plugins`	Plugin marketplace
`/plugins/:id`	Plugin detail (declared extension point timeline · file preview · CLI install)
`/agents`	Agent catalog (built-in Agent packages, then uploads)
`/inbox`	Pending listing and Performance requests the caller can decide
`/agents/:id`	Binding, files, registered models, Performance
`/models`	Models plaza (lab-grouped directory tables · modality filter · Cmd/Ctrl+F search)
`/models/:canonical`	Model encyclopedia (Overview spec · Performance)
`/organizations`	Your orgs · Create / Join · dataset / plugin / agent counts
`/organizations/:orgId`	Overview (members · datasets · plugins · agents) · Secrets (members) · Settings
`/users/:login`	Public user profile (official orgs and public datasets / plugins / agents; signed-out OK)
`/datasets/:id/tasks/:task/attempts/:runId`	Remote Attempt detail (needs uploaded run artifacts)
`/datasets/:id/suites/:suiteRunId`	Suite run detail (Profiles / Plugin / Jobs / Share)

Ops detail: apps/hub/README.md, services/registry/README.md.

Datasets and tasks

The list is grouped by organization: the group head shows the org name and description; rows carry name, description, task count, and last update. Open one for its README and task file previews. On a wide screen, a rail at the right edge jumps to an organization: collapsed bars scale with the package count, hover shows the mark and name, and a click seats that table under the sticky header. The rail hides on a narrow screen, and when fewer than two organizations remain.
Version switching: entitled users see draft (current); everyone else switches among published releases. An org owner can publish the draft, change visibility, or delete a version.
When the package declares root shared/ or overlays:, the task Files header gains Shared / Overlays scopes (Local by default).
Jobs under a task list uploaded runs; click through for trajectories (the layout is close to the local Viewer).

Dataset list grouped by organization — Datasets, grouped by organization.

Leaderboard

Catalog sidebar leads with Leaderboard (/leaderboard); / redirects here. It is an index of the three existing surfaces, not a fourth ranking. The Dataset view groups by dataset (name + description as the group head) and lists public listed suites; Agent / Model views reuse the existing Performance gates (plaza collect or Agent-org consent). The three tables may disagree on membership. Signed-out is OK. Internal, draft-bound, and unlisted suites stay on the dataset Leaderboard tab — they never appear on the plaza. On a wide screen, the same right-edge rail jumps to the current view's groups: a dataset, an Agent organization (Builtin has no org mark), or a lab. Table, Pareto, and Waffle stay on that dataset group. The rail hides on a narrow screen, and when fewer than two groups remain.
The plaza Dataset view switches Table / Pareto / Waffle with the same control as a dataset Leaderboard (right of the Dataset tab). Charts are per dataset group. Scores sit on each inner row and are not comparable across datasets. There is no rank column. Agent / Model views stay tables.

Public lists only suites that are complete, bound to a release, and approved for listing by the dataset org's owner; Internal shows caller-visible incomplete or draft-bound runs in the same table.
The same tab switches Table / Pareto / Waffle. Pareto’s x-axis is the sum of every job in the suite (cost, tokens, or time); left is cheaper, right is more. When the agent did not report USD, cost is tokens × catalog price — estimated, not a billed invoice. Higher pass rate at no extra cost forms the frontier.
Table headers sort. Dataset / Agent Model cells prepend a lab mark when the overlay uniquely joins the catalog; the plaza Model view already marks the group head, so the inner Model column does not repeat it. Harness / Agent cells prepend the builtin product mark when the short id matches.
Click a row for the suite run page: Profiles / Plugin / Jobs, with secret-free YAML and rerun commands. On a Dataset table, Harness and Model (when they match) open the agent page and model encyclopedia; the suite detail header uses the same links. In-app text links stay ink until hover; off-site links stay blue. Uploaders get Share, Delete, and Attach — link a published model / agent for provenance.
Listing and Performance requests are batch-approved by org owners in the Inbox.

deep-swe waffle, model above harness — Waffle on the same board. One square is one trial. The column title is the model, with the harness under it.

Plugins, agents, and models

The plugin / agent marketplace uses cards: description, tags, downloads, and stars. Detail pages show the declared extension-point timeline, file preview, and the CLI install command.
Agent details add bindings, files, and Performance.
Models in the sidebar is a catalog grouped by lab (context, price, and more; searchable; a row opens its encyclopedia page). Prices are catalog list prices, not invoices. A lab is listed only when it has a logo; labs that would show as a letter, and their models, stay out of the catalog. On a wide screen, the right-edge rail jumps to a lab. The rail hides on a narrow screen, and when fewer than two labs remain.
Official orgs, their members, and their plugins carry a verified badge.

Plugin marketplace as cards — Plugins. One card is one package.

Create an org or join by invite code; the code is shown in full only once, at creation.
Owners / admins manage members and roles by GitHub login and can transfer the org; members can publish under it.
Private results can be shared with an org or user; the signed-in Home collects your uploaded jobs, packages, plugins, and agents.

vs Viewer

	Viewer	Hub
Data	Local `.ageval/suite-runs/`	Registry
Entry	`ageval view`	Browser SPA
Job artifacts	Local run dirs	Must be uploaded first

Jobs rows with uploaded run artifacts open detail; others show Not uploaded.

What to upload so Hub shows it

You want	Upload	How
A Public Leaderboard row	A complete, release-bound suite listed by the Dataset org owner	`ageval publish --draft` then `ageval release` (or a direct `ageval publish`) → `ageval run <dataset>` without `--task` → `upload-suite --suite-run <id>` → `ageval results request --kind leaderboard_list --suite-run <id>` (owner approves in Hub Inbox)
Leaderboard Internal	A caller-visible incomplete or draft-bound suite	Same `upload-suite`; does not appear on Public
Task Jobs with openable Attempt / trajectory	Suite + attempt evidence	`upload-suite --with-attempts`; incomplete / draft-bound suites stay here
Patch one finished slot on an uploaded suite	New Attempt + suite slot pointer	Local `--resume-suite --replace-slot`, then `upload-suite --suite-run <id> --task <id> [--run <run_id>]`. Hub/Viewer default to current; the version list is `patch N` plus that Attempt’s start time
Archive one Attempt only	`ageval results upload --run <run_id>`	May exist in Registry, but usually not on Leaderboard

# Correct path for Leaderboard
ageval run <dataset> --profiles <profiles.yaml>
ageval results upload-suite <dataset|id@version> --suite-run <8-hex> --public --with-attempts

# Not enough for Leaderboard: single-task attempt upload alone
# ageval run <dataset> --task <id>
# ageval results upload <dataset> --run <run_id>

A snapshot share is a read-only link for one local suite, including one that is still running. The default link freezes the summary and Attempt evidence present at upload. upload-suite still refuses an unfinished summary. ageval results share still grants org or user access. Leaderboard listing still goes through upload-suite and an owner approval.

ageval results share-snapshot <dataset> --suite-run <id> --hub-url https://hub.example
ageval results share-snapshot <dataset> --suite-run <id> --run <run_id> --hub-url https://hub.example
ageval results share-snapshot <dataset> --run <run_id> --hub-url https://hub.example
ageval results share-snapshot <dataset> --suite-run <id> --live --hub-url https://hub.example
ageval results sync-snapshot <dataset> --suite-run <id>

--suite-run uploads that suite. --run uploads one Attempt. Both together upload that one job inside the suite. The command prints {hub}/s/{token}. The local Viewer icon on an opened suite can create a frozen snapshot or a live link. A job stays a frozen snapshot. The dialog explains the upload; confirming it shows the link and a copy button. Opening the icon again shows that link, and can remove the share. A live link says it updates when a job finishes. Anyone with the link can open it without signing in. The page shows the jobs and Attempt evidence from the moment of upload. Later edits on disk or in Hub do not change a default link.

--live keeps that same {hub}/s/{token} and updates it when a job finishes. If the live link exists before the suite starts, the runner also sends counts and status about every 45 seconds. A link created while the suite is already running still updates when a job finishes; counts in between come from sync-snapshot --heartbeat. sync-snapshot sends the current summary again after the runner stops, on the same token. The Hub page refreshes while the link is live. An Attempt page shows the files already uploaded.

ageval results revoke-snapshot <token-or-url> deletes the link. It works for a frozen link and a live link. Local .ageval/suite-runs and .ageval/runs stay. The suite's public or private setting and any leaderboard listing stay as they were. The link does not expire.

The body limit is the same 512 MiB cap as upload-suite. api_key values that are not environment locators are removed before upload and again on each live update. The link cannot be imported as a catalog suite or sent to the leaderboard.

Dataset Leaderboard lists every suite that passed the board gate. The header version switcher only changes README / Tasks. The Select to the right of Public filters rows by the suite's dataset_version (default All versions).

Point Hub at the same Registry as ageval login / ~/.ageval/credentials (local dev often 127.0.0.1:8700).

On this page