Repository navigation
Closed
Copy link
Labels
P2Planned: limited in scope, has a way around it, or a missing featurePlanned: limited in scope, has a way around it, or a missing featureenhancementNew feature or requestNew feature or request
Description
Priority
medium. Most of the change is in the web layer: one new route, the card and table toolkits' templates and scripts, and the builder's script. In the backend, the Agents page's listing moves from web/agents.py into standard/recipe.py with the same output, and the protocol names change to sentence case.
Components
- app/nullius (backend)
- app/web (routes)
- templates and static files
- docs and the site
- catalogue (benchmarks, agents, capabilities)
- the standard (ports, kits, relay, runs)
- CI, workflows and skills
Today
- The builder's Agent step draws its own tiles, search and pill rows in
command-builder.js, from the builder payload. The Agents page draws its cards with the card toolkit (_cards.html,tables.js) from a listing built inweb/agents.py. The two views of one agent differ: the tile has no From and no proven ports, and the Agents page card puts the version in its foot. - The step has two rows of pills: protocol (All, ACP, a command, its own script, reference) and fit (Any, Can run it, Cannot). A chip, "Other benchmarks' agents", shows or hides other benchmarks' own agents. For GSM8K the step opens on 23 tiles, and 16 of them cannot run it.
- A benchmark's Original tile has "reference" as its protocol, so the protocol pills offer "reference".
- The step has no table view. The Agents and Capabilities pages have one.
- A filter can hide an agent that the recipe has chosen.
- Card fields have no fixed position. A greyed card puts its reason under the protocol, and the foot wraps on some cards in the builder.
- On the Agents page, "Runs on: OSWorld1" shows only Codex, because the From filter hides a benchmark's own agents by default.
- A toolkit dropdown with one value chosen shows a count ("Filter 1"), not the value.
Wanted
- The Agent step and the Agents page show one listing (
recipe.entries), drawn with the card toolkit. The builder loads the step as a fragment,GET /new-run/agents?benchmark=<key>. - A card can be a pick: a toggle with
aria-pressedthat sends anullius-pickevent. A chosen card shows a check mark. A card that cannot be chosen is disabled and says why. - Each piece of information has one slot on every agent card, in the builder and on the Agents and Capabilities pages: name and version on the title row, the reason it cannot run under it, proven ports as badges, and a foot with the protocol on the left and From on the right.
- Protocol names are in sentence case ("Its own script", "A command"). A benchmark's Original reads "As published" and is not a protocol.
- Controls: search, a Fit dropdown that starts on "Can run it", a Filter menu (Protocol, From, Proves), and a cards/table toggle. The toggle switches in place, keeps the picks and is remembered.
- The table view has a narrow checkbox column on the left. A click on a row picks it. Name takes the spare width.
- A chosen agent stays visible under any search or filter, also after a recipe import.
- On the Agents page, Runs on shows a benchmark's own agents when it chooses that benchmark.
Acceptance
-
GET /new-run/agents?benchmark=<key>returns the Agent step's cards, built fromrecipe.entries. An unknown benchmark returns 404. - The Agents page output is unchanged by the move of its listing into
recipe.entries, with a test. - A pick card is a button with
aria-pressed. A click, Space or Enter flips it and sendsnullius-pick. A disabled card does not flip and shows its reason, with a test. - The greying of each card on the step equals the builder's fit for every benchmark, with a test.
- The step opens with Fit on "Can run it". A dropdown with one value chosen names that value on its button.
- A chosen card stays visible under any search or filter, with a test.
- The table view picks as the cards do: a checkbox per row, and a row click flips it, with a test.
- The cards/table toggle keeps the recipe and its picks, and does not reload the page.
- Every agent card puts name and version, reason, ports, protocol and From in the same slots, on the builder and on the Agents page.
-
docs/builder.mddescribes the step's controls. -
app/ui_tests/test_recipe_execution_fixtures.pyand its fixtures are unchanged.
Dependencies and related
- feat(web): show the Workspaces page as cards, with a card layout shared by every card page #238 (merged): the shared card layout.
- feat(catalogue): sort and filter the catalogue pages, and show them as cards or a table #240 (merged): the catalogue pages' filters, sort and cards/table toggle, which the step reuses.
- Standard: a benchmark's own agents are one option of its Original setup, not agents to choose #245 (closed) and feat(standard): choose a benchmark's own agents like any agent, one job each #246 (merged): a benchmark's own agents as agents, which the step lists first.
Metadata
Metadata
Assignees
Labels
P2Planned: limited in scope, has a way around it, or a missing featurePlanned: limited in scope, has a way around it, or a missing featureenhancementNew feature or requestNew feature or request
Activity