Skip to content

fix(llm): carry registry model card metadata - #265

Open
wolfy-j wants to merge 3 commits into
masterfrom
fix/llm-registry-card-metadata
Open

wolfy-j wants to merge 3 commits into
masterfrom
fix/llm-registry-card-metadata

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Model cards built from llm.model registry entries have no metadata field. A catalog that merges registry cards with cards from other sources (for example, a database) and reads card.metadata gets metadata only from the other sources. A registry entry cannot declare any.

Cause

models._build_model_card copies a fixed list of fields into the card. metadata is not on that list.

Fix

_build_model_card copies entry.data.metadata onto the card. When the entry has none, it uses {}, the same default pricing gets (entry.data and entry.data.pricing or {}). A declared value is carried unchanged, including a non-table one, so a misdeclared entry stays visible to the consumer instead of reading as empty metadata. _build_model_card validates no payload field, and pricing follows the same rule.

The value comes from entry.data, not entry.meta, because the card already splits its fields this way:

  • Index fields (name, title, class, capabilities, priority) come from entry.meta.
  • Payload fields (max_tokens, output_tokens, pricing, providers, dimensions, fallback) come from entry.data.

metadata is a free-form payload table like pricing. It is declared at the top level of the entry, next to pricing in the README's "Registering Models" example. The registry stores those top-level keys in entry.data.

Other changes:

  • ModelCard in llm.lua gets metadata: table, typed the same way as pricing: table.
  • The README documents the field. llm itself does not read it.

Tests

New cases in src/llm/test/discovery/models_test.lua ("card metadata"):

  • A fixture entry's data.metadata appears on the card returned by get_by_name, get_all and get_by_class.
  • An entry without metadata gets an empty table.
  • An entry declaring metadata = 42 yields a card whose metadata is 42.

Results:

  • wippy test ... test -- models_test: 25 passed. The non-table case fails against a build that replaces the value with {} (expected 42, got {}).
  • Full llm suite: 2911 tests, 0 failed, 9 skipped (the e2e cases that need provider credentials).
  • wippy lint --level error: clean.

Model cards built from llm.model registry entries dropped the entry's
metadata table, so catalogs reading card.metadata could not take it from
the registry. _build_model_card copies entry.data.metadata onto the
card, defaulting to an empty table like pricing.
@wolfy-j
wolfy-j requested a review from skhaz November 14, 2026 00:02
The card copies entry.data.metadata as declared, like pricing. Replacing a non-table value with an empty table hides a misdeclared entry; the file validates no other payload field at build time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants