Repository navigation
Closed
Feature
Copy link
Labels
P2Planned: limited in scope, has a way around it, or a missing featurePlanned: limited in scope, has a way around it, or a missing featureenhancementNew feature or requestNew feature or request
Description
Priority
medium. Most of the change is in the web layer: one new route, the card and table toolkits' templates and scripts, and the builder's script. In the backend, the benchmarks' listing moves into standard/recipe.py, and the Benchmarks page output does not change.
Components
- app/nullius (backend)
- app/web (routes)
- templates and static files
- docs and the site
- catalogue (benchmarks, agents, capabilities)
- the standard (ports, kits, relay, runs)
- CI, workflows and skills
Today
- The builder's Benchmark step draws its own tiles, search and pill rows in
command-builder.js. It merges three sources: the standard payload, the Harbor list fromrecipes/launch.py, and catalogue lookups. The Benchmarks page draws its cards with the card toolkit (_cards.html,tables.js) from a listing built inweb/catalogue.py. - The step has two rows of pills: category, and Any, Agents, Capabilities.
- The step has no table view, no sort and no runner filter. The Benchmarks page and the Agent step (feat(builder): draw the Agent step with the Agents page's cards, as cards or a table #249) have them.
- The card toolkit has no single choice. A pick card is a toggle, and its table row has a checkbox.
- On a phone, a card table's name column breaks a long name inside a word ("LongMemEv al").
- On a chosen pick card, the category tag and an agent's initials take the card's own background colour, so their fill disappears. In the dark theme, an outlined tag loses its outline.
Wanted
- The Benchmark step and the Benchmarks page show one listing (
recipe.benchmarks), drawn with the card toolkit. The builder loads the step as a fragment,GET /new-run/benchmarks. - A pick can be a single choice (
one: true). A click on a card chooses it and lets the others go. A second click keeps it chosen. In the table view, the row has a radio. - Card fields: mark, name, and a check when chosen. An "Agents" or "Capabilities" badge. A Tasks stat for the chosen version. The category tag, and "Community" for a community benchmark.
- Controls: search, the category pills, an Evaluates dropdown (Agents, Capabilities), a Filter menu (Runner), Sort by (Name, Tasks), and a cards/table toggle. The toggle switches in place, keeps the choice and the draft, and is remembered.
- The chosen benchmark stays visible under any search or filter, also after a recipe import.
- A card table's name wraps only between words.
- Inside a chosen pick card, tags and marks keep their fill and their outline, in both themes.
Acceptance
-
GET /new-run/benchmarksreturns the step's cards, built fromrecipe.benchmarks, with?view=tablefor the table. - The Benchmarks page output is unchanged by the move of its listing into
recipe.benchmarks, except for one new sprite icon, with a test. - A single-choice pick lets the other picks go and stays chosen on a second click. Its table row has a radio, and a row click chooses it, with a test.
- A benchmark with versions shows the task count of the version that the recipe names, and the count changes when the version changes, with a test.
- The chosen benchmark stays visible under any search or filter, also after a recipe import.
- The cards/table toggle keeps the recipe and does not reload the page.
- A card table's name does not break inside a word at 390 px.
- Tags and marks on a chosen pick card keep their fill in both themes.
-
docs/builder.mddescribes the step's controls. -
app/ui_tests/test_recipe_execution_fixtures.pyand its fixtures are unchanged.
Dependencies and related
- feat(catalogue): sort and filter the catalogue pages, and show them as cards or a table #240 (merged): the catalogue pages' filters, sort and cards/table toggle, which the step reuses.
- Builder: the Agent step draws its own tiles, apart from the Agents page's cards #248 (closed) and feat(builder): draw the Agent step with the Agents page's cards, as cards or a table #249 (merged): the same change for the Agent step.
- feat(catalogue): add OSWorld desktop-env-v1.0.3 as OSWorld1's default version #261 (merged): OSWorld1's versions, which count different tasks.
Metadata
Metadata
Assignees
Labels
P2Planned: limited in scope, has a way around it, or a missing featurePlanned: limited in scope, has a way around it, or a missing featureenhancementNew feature or requestNew feature or request
Activity