Repository navigation
Conversation
…rd (#1194) Every exported record now carries `attributionGround` beside the stored `attribution` claim: the per-key claimed/observed/state record the job page already reads, reconstructed by the same rule for rows written before #1186 and null when nothing was recorded. The rule moves to `teamStats.ts` so the page and the export share it. The column joins REQUIRED_JOB_COLUMNS, so a pre-#1186 database is refused by name, and DATASET_SCHEMA_VERSION moves to 2 while a version-1 export still verifies against its own manifest.
HugoXK-bot
approved these changes
Nov 21, 2026
HugoXK-bot
left a comment
Collaborator
There was a problem hiding this comment.
Verdict: PASS at 5b59219
- Shape and version are one change:
DATASET_SCHEMA_VERSION = 2(src/lib/evolution/dataset.ts:20),attributionGround: AttributionGround | nullonEvolutionRow(:44) and redacted inredactRow(:166);contentHashOfhashes the whole records array (:255-259), so the ground sits inside the content hash and the version bump is required, not cosmetic. - One read exports both halves:
j.attribution_ground_jsonjoins the SELECT (src/lib/evolution/export.ts:24), the claim stays the stored object (:54), andstoredGround ?? reconstructAttributionGround(attribution, team_stats_json !== null)(:55) uses the single rule atsrc/lib/teamStats.ts:55-69, which the page now calls (src/views/pages.ts:513) instead of keeping its own copy. - Back-compat reproduced, not read: I rebuilt
test/fixtures/evolution-dataset-v1.jsonwith the base builder at044c7515from its own records and got byte-equal manifest and records, idds_c0f66c8fe1cda136; at this tipverifyDatasetrecomputes overmanifest.schemaVersion(dataset.ts:369), so v1 returns ok and a tip-built dataset verifies at 2. - The refusal is real:
attribution_ground_jsonjoinsREQUIRED_JOB_COLUMNS(dataset.ts:27),assertExportablenames the missing column (:434-440), and the CLI calls it beforeloadEvolutionRows(scripts/evolution-shadow.ts:185), so the new SELECT cannot hit a pre-#1186 database. - Red then green: 28/28 at the tip; at base
044c7515with only the committed tests, 18 pass / 3 fail naming version 1≠2, the missing refusal and the loaded record with noattributionGround. No UI surface — only the helper moved and the rendered string is unchanged. Non-blocking: the fixture's rows are synthetic (finishedAt: ""), so "a real dataset produced at 044c751" proves the version path and its bytes, not a production export.
Ran: npx tsx --test test/evolutionDataset.test.ts test/evolutionDatasetVerify.test.ts test/attributionGrounding.test.ts (28/28 in a tree built as base 044c7515 + this diff, all 10 blobs equal to head); npx tsx --test test/evolutionDataset.test.ts test/evolutionDatasetVerify.test.ts with only the committed test files at base (18 pass, 3 fail as quoted); npx tsx probe_fixture.ts (base builder rebuild of the v1 fixture, manifest and records byte-equal, id ds_c0f66c8fe1cda136).
Not run: full npm run check locally (CI ci-ok 136933907113 success at this tip); Docker, SMTP and an export of a live database.
Written-by: HugoXK-bot
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Written-by: Fledge Platform Dev-4
Fixes #1194
The defect
An exported evolution dataset record carried
attributiononly — the claim — and nothing about what the instance observed.src/lib/evolution/export.tsselected the job row withoutattribution_ground_json, andsrc/lib/evolution/dataset.tsredactedattributionbut had no ground field, so a consumer of the dataset could not tell a claim from an observation: the exact distinction the job page already draws throughgroundAttributionand the legacy reconstruction from #1186.What changed
src/lib/teamStats.ts: the legacy reconstruction half of the page'sattributionGroundFormoves here asreconstructAttributionGround(recorded, observedOnly); the job page and the export now call one rule. The page's behaviour is unchanged (its existing grounding tests pass untouched).src/lib/evolution/export.ts: selectsj.attribution_ground_json; each row gainsattributionGround = storedGround ?? reconstructAttributionGround(parsed claim, team_stats_json !== null).attributionstaysparse(attribution_json), exactly as before — the claim is not transformed.src/lib/evolution/dataset.ts:EvolutionRow.attributionGroundis typedAttributionGround | nulland redacted through the existingredactValue;DATASET_SCHEMA_VERSIONmoves to2;attribution_ground_jsonjoinsREQUIRED_JOB_COLUMNS.docs/decisions/0003-agentdescent-evolution.mdand the dataset paragraph indocs/architecture.mdname both fields and version 2. Onechangelog.d/1194.added.md.No schema migration (
#1186already added the column and itsALTER TABLE), no change to the split, the hash rule, the consent filter or the page.What the record looks like at this tip
A probe against a
Storewith one stored-ground row and one pre-#1186 row (claim plus team stats), exported throughloadEvolutionRows+buildDataset:{ "schemaVersion": 2, "records": [ { "jobId": "job_legacy", "attribution": { "writtenBy": "Dev-1" }, "attributionGround": { "steps": [{ "key": "writtenBy", "claimed": null, "observed": "Dev-1", "state": "grounded" }] } }, { "jobId": "job_stored", "attribution": { "writtenBy": "Dev-2", "reviewedBy": "Reviewer" }, "attributionGround": { "steps": [{ "key": "writtenBy", "claimed": "Dev-2", "observed": "Dev-1", "state": "unobserved" }] } } ] }The legacy row is reconstructed by the page's rule (observation, no claim); the stored row keeps its claim and carries the observation beside it. A row with nothing recorded carries
attributionGround: null— the loader test pins that, and the unit test pinsnullfor an input row without a ground record.Acceptance, item by item
test/evolutionDataset.test.ts— "a record carries the observed ground beside the claim…" (claim unchanged, stored ground travels through, no-record row isnull) and "the loader reads the stored ground and reconstructs a legacy row, never guessing grounded".test/evolutionDatasetVerify.test.tsloadstest/fixtures/evolution-dataset-v1.json— a real dataset produced by the pre-change builder at044c7515(ds_c0f66c8fe1cda136) — andverifyDatasetreturns ok atschemaVersion: 1.assertExportable(['kind','evidence_json','team_stats_json'], 38)throws namingattribution_ground_json;REQUIRED_JOB_COLUMNS.lengthis 4.Red at the base
At
044c7515,npx tsx --test test/evolutionDataset.test.ts:a record carries the observed ground…→expected: 2, actual: 1(schema version).the frozen schema refusal is gone…→Missing expected exception(noattribution_ground_jsonrequirement).the loader reads the stored ground…→ the loaded record had noattributionGround.At this tip, the same three suites together: 28/28 pass.
Ran:
npm run check— pass (lint baseline 11 warnings in 8 entries, typecheck, 3296/3296 tests, build)node scripts/version.mjs check—version: 0.4.5 oknpm run mock:check—ok (14 files)node scripts/changelog.mjs check—4 fragment(s) oknpx tsx --test test/evolutionDataset.test.ts test/evolutionDatasetVerify.test.ts test/attributionGrounding.test.ts— 28/28Not run: Docker image, real SMTP, a production export of the live database.
Uncertainty and residual risk
team_stats_json !== nullas "this column holds an observation". A row whose team stats are malformed JSON takes that branch and reconstructs from whateverparseAttributionJsonreturns; that is the page's current answer, not a new one.parseAttributionGroundfalls back to reconstruction rather than exporting the malformed object — same as the page.redactValueruns over the ground so role names pass the one recogniser; the shape is cast back toAttributionGroundbecause redaction preserves strings.MISSING_JOB_COLUMNSmessage tells the operator to start the current Fledge once, which applies theALTER TABLE; that is the existing migration path, not a new one.Alternatives considered
buildDatasetfromattribution+teamStats: rejected — the builder only sees rows, not whether the stored ground column had a value, and the loader is the one place that reads it.DATASET_SCHEMA_VERSIONat 1: rejected — the record shape is a data contract; a consumer must know which fields to expect.