Skip to content

#37 co-core 0.38.0: JsonExtractor, the jsonpath corpus - #56

Merged
gregoryfoster merged 5 commits into
mainfrom
37-co-core-0.38.0
Oct 20, 2026
Merged

gregoryfoster merged 5 commits into
mainfrom
37-co-core-0.38.0

Conversation

@gregoryfoster

Copy link
Copy Markdown
Contributor

Refs #37 (the Watcher notice, the deploy, its live checks and the Archiver notice come after the merge).

co-core 0.37.0 → 0.38.0 on all three packages; processor_version 0.38.0+1 (LOCAL_GENERATION stays 1). Plan: docs/plans/2026-10-20-37-co-core-0.38.0.md.

Catch-up reading (0.37.0 → 0.38.0, 0.37.1 between), checked in code

git diff v0.37.0 v0.38.0 -- packages/:

  • content.process / content.derived models: ContentProcessCommand(Emit), ProcessingComplete*, ProcessingFailed*, ProcessingFailureReason are identical with docstrings stripped (AST compare). The rest of changes.py is the capture pair (cannobserv#571).
  • Bus: envelope.py registers the capture pair and moves info_item_primary_changed's key to …:occurred_at (#578); streams.py adds CONTENT_CAPTURE. Processor reads neither. dead_letter, exceptions, co_core_aio, co_core_sync: version strings only.
  • canonical_text, blob store, hashing: unchanged. spec_fingerprint: serialization moved to the shared _canonical_json, same json.dumps arguments; no spec1: moved.
  • Extract: JsonExtractor (#354), xpath on HTML (#557), ignore_selectors and the uncompilable-selector refusal (#570), the dispatch map, the translator, .json in resolve_dispatch_essence.
  • Dependencies: python-jsonpath>=2.2,<3 (locks 2.2.2, runs in the contained child) and soupsieve declared (already present).
  • CHANGELOG 0.37.1, 0.38.0: no security entries.

No code change in src/.

Goldens (co-core-direct)

On main's corpus, unchanged, 0.38.0 moves exactly three cases:

refused_jsonpath       refused -> sha256:d6be94c49f4d0c9c4084abadf66e4974c89e591c729aa792438ee51d47dee587 (64 B)
refused_xpath_on_html  refused -> sha256:c331fdd4416460510d486e08aa51245cf42b53482beee1e70f2f369dff81962f (90 B)
unknown_essence_is_html sha256:52492823a71e22338921f03375537ca3f356a747c98a72c30795b33c9d852d45 (82 B)
                     -> sha256:d1da45850448ef0dc317a67de8685e7a6e505db6fe94a1cd553786a888f74eac (74 B, JsonExtractor)

Corpus edits, then the committed regeneration (co-core 0.38.0: 12 moved, all added/removed):

  • renamed: refused_jsonpath → json_jsonpath, refused_xpath_on_html → html_xpath (same digests as above);
  • unknown_essence_is_html re-pointed at application/x-unknown: back on the HTML fallback, v1 digest 52492823… unchanged; a test holds its essence outside EXTRACTOR_BY_ESSENCE;
  • added: json_full_page, json_vnd_api (both d1da4585…), json_jsonpath_titles (7ac46158…, text exactly Agenda item one\nItem two: Ask 2's narrowing), html_ignore_selectors (54fb39f5…), and refusals refused_empty_jsonpath, refused_jsonpath_on_html, refused_jsonpath_on_unlisted_plus_json (application/problem+json → HTML), refused_css_on_json.

Every other golden, every spec_fingerprint, unchanged. Real corpus: 9/9 unchanged (all text/html).

Tests

  • Trap 1: test_a_case_that_starts_or_stops_being_refused_is_listed re-pointed at refused_css_on_xml.
  • JSON parse failures (NaN, Infinity, 1e400, too-deep nesting, unpaired surrogate) and a .. past python-jsonpath's depth limit raise ValueError, not a refusal; through the child, under containment, they come back raised / ValueError:, the PDF path → extraction_error, terminal.
  • Whole-corpus child test runs the JSON cases under containment required (Landlock ABI 6).
  • uv run pytest: 679 passed, 14 skipped (credential-file denials absent here, as on main); ruff clean; uv lock --locked clean.

Either adoption order is safe (resolve_dispatch_essence)

D5 names it as a reason to coordinate, but Processor never calls it (grep -rn resolve_dispatch_essence src/: none). Watcher resolves the essence on its own pin; Processor dispatches what arrives. An old Watcher keeps sending the ambiguous essence → HTML, as before. A new Watcher's application/json → HTML on today's Processor, JsonExtractor after this bump. So: notice on CannObserv/watcher#368, not coordination.

Docs

Spec: amendment 2026-10-20, D5 "today", §2, §4 refusal row, §5 pin + sub-bullet, §7 JSON coverage and pin test, quick reference, OQ2. AGENTS.md: pin, Owns (JSON).

🤖 Generated with Claude Code

gregoryfoster and others added 5 commits October 20, 2026 04:09
…ath corpus

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… 0.38.0 pin (red)

refused_jsonpath and refused_xpath_on_html renamed json_jsonpath and
html_xpath; unknown_essence_is_html re-pointed at application/x-unknown.
The generator's refusal-listing test moves to refused_css_on_xml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…regenerated

processor_version 0.38.0+1. python-jsonpath 2.2.2 enters the lock and the
contained child. Goldens regenerated co-core-direct: on the unchanged corpus
exactly refused_jsonpath, refused_xpath_on_html and unknown_essence_is_html
moved; with the corpus edits, only the renamed and new cases are listed, and
unknown_essence_is_html keeps its v1 digest on application/x-unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… name their essence

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gregoryfoster
gregoryfoster merged commit 540975e into main Oct 20, 2026
2 checks passed
@gregoryfoster
gregoryfoster deleted the 37-co-core-0.38.0 branch October 20, 2026 04:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant