Skip to content

Make validate.sh pass, add a Python port of the validator and an import check - #11

Merged
AdaWorldAPI merged 16 commits into
masterfrom
ccr-39105428-bcm2m2
Oct 28, 2026
Merged

AdaWorldAPI merged 16 commits into
masterfrom
ccr-39105428-bcm2m2

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Important Note:

It is recommended NOT to merge pull requests on weekends.

Summary

./validate.sh now passes over the whole repository: Validation successful., exit 0, 3672 files. On master it stops at the first file it cannot accept. Every fix below was found by running the validator jar itself, stage by stage.

Merge together with AdaWorldAPI/OGAR#364. OGAR's CI reads this repository's master. Two OGAR tests pin what this PR changes (the entities its reader and the lance-graph cache disagree on, and the number of Accounting files), so the two PRs have to land as a pair.

How the validator works (read from its bytecode)

bin/ogit-validator.jar runs three stages, and a stage only runs if the one before it passed.

  1. Per file. This stage runs only when more than one file is given.
    • Every ogit* prefix must equal http://www.purl.org/ + the prefix, with . turned into /, + /. The first prefix that doesn't aborts the whole run.
    • The file name must equal the local name of the file's first subject, plus .ttl. A mismatch is reported but does not abort.
    • The file's canonical path must match /<namespace dir>/(.*)/<name>, case-insensitive. The first path that doesn't aborts the whole run.
  2. Rules. Ten SPARQL queries shipped in the jar (validation0.rq … validation9.rq) check mandatory properties on entities, attributes and verbs, ogit:history items, date formats and the validation-type/parameter pairing.
  3. References. de.arago.ogit.tools.OgitValidator checks every referenced attribute, parent and verb, each allowed edge's ends, scope, cardinality, labels, descriptions, creators and dates.

Because the directory check reads the canonical path, the root-namespace files in SGO/sgo and SGO/core pass only because the checkout itself is named OGIT (as on GitHub CI): /ogit/(.*)/node.ttl then matches. In a checkout under another name they fail with Directory mismatch. The layout is left as it is.

What was failing, and the fix

Stage Finding Fix
1 FiscalJurisdiction.ttl used ogit.Compliance: and ogit.FinancialMarket: undeclared declare both
1 12 NTO/WorkOrder/verbs/*.ttl capitalised (HasActivity.ttl declares hasActivity) rename the files
2 23 Healthcare/WorkOrder entities lacked dcterms:valid, 4 lacked dcterms:creator, Tenant lacked ogit:allowed add them
3 186 "attribute doesn't exist": WorkOrder and Healthcare declared attributes inline as a rdfs:Property / a ogit:Property, which OGIT does not read as attributes one file per attribute under NTO/{WorkOrder,Healthcare}/attributes/, as owl:DatatypeProperty + rdfs:subPropertyOf ogit:Attribute. IRIs unchanged. Six attributes defined two or three times with different wording now have one description and list every source column
3 13 WorkOrder attributes referenced but defined nowhere (menge, einzelpreis, versteckt, …) declared, each description checked against AdaWorldAPI/WoA models.py and the code that uses the column
3 ogit:created-at does not exist use the existing ogit:createdAt
3 Healthcare edges see below

Healthcare edges:

  • ogit:assesses was undeclared. It becomes ogit.Healthcare:assesses in NTO/Healthcare/verbs/, and its target AnatomicalStructure is declared.
  • loincRef is an attribute, as its own description says, so the edge to it becomes an optional attribute.
  • ogit:Tenant doesn't exist. OGIT records tenancy with the ogit:tenantId attribute.
  • The seven combo_* enumerations were subclasses of the undeclared ogit:Enumeration. They become entities and move to entities/.
  • The three Assessment subclasses move into their own files.

Accounting: eleven files added on 2026-05-28 described themselves as shortcuts over another program's internal field paths. Their descriptions now say what each term is.

  • Removed, because no entity referenced them: the verbs hasPickingType and hasProductCategory, and the attribute productCategoryComplete.
  • Renamed: commercialPartnerCountryCode is now counterpartyCountryCode; JournalEntry lists the new name.
  • Rewritten descriptions: fiscalCountryCode, iso3166Alpha2, isEuMember, vatRateStandard, FiscalJurisdiction, JournalEntry, hasFiscalCountry.
  • No code in lance-graph, OGAR or the consumer repositories reads the removed or renamed IRIs.

Also:

  • NTO/PTF entities declare the rdfs: and dcterms: they use.
  • src/lib.rs compiles on current rustc (E0700), still within rust-version 1.70.

An earlier commit on this branch moved SGO/sgo and SGO/core to SGO/ogit. It was based on a run in a checkout with another directory name, and is reverted.

tools/ogit_validate.py: the validator without Java

A Python 3.8+ port of the jar's three stages, with no packages. It prints the jar's messages on the same streams with the same exit status, and validate.sh is unchanged. Two findings from the bytecode decide the output order: rdf4j's memory store commits through a sorted TreeModel, so statements come back sorted rather than in file order, and the directory check reads the canonical path.

Known difference: stage 2 names one subject per failing query, and which one the jar names depends on rdf4j's grouping and on the argument order; no model of it reproduced the jar. The port names the first in store order, so the set of failing queries is the same but the subject can differ. --all lists every failing subject. rdf4j's wording for a generic Turtle syntax error is not reproduced either.

tools/test_ogit_validate.py runs 19 fixtures and compares the port with output recorded from the jar (and with the jar itself when Java is installed). Disabling the sort, the ogit/ removal in the directory check or the scope check makes it fail.

tools/import_check.py

A stricter check for importers, not wired into validate.sh or CI. It reports 0 errors on this branch. It also writes the verb catalogue (531 verbs: 519 owl:ObjectProperty, 12 verb-as-class) and the allowed-edge table (2,965 unique (source, verb, target) rows). The verb-as-class form (WorkOrder's 12, with ogit:from-to) is accepted, since OGAR documents it as a deliberate convention. 13 unit tests.

Not changed, reported for a decision

  • Attributes that duplicate root ogit: attributes:
    • WorkOrder: firstname/lastname/email/phone on User, vorname/nachname/telefon/plz/ort on Customer, createdAt (8 entities), updatedAt, url, titel, beschreibung;
    • Healthcare: firstname, lastname, country, priority.
  • ORM storage details carried as attributes: passwordHash, passwortEnc, notizenEnc, current (a sequence counter), Healthcare dbSource* provenance columns.

Re-pointing any of these at the shared terms changes IRIs that consumers read.

Checks run

  • ./validate.sh in a checkout named OGIT: Validation successful., exit 0. Before: aborts in stage 1.
  • ogit_validate.py and the jar over the whole repository give the same stdout, stderr and exit status on this branch in a checkout named OGIT (pass), in a checkout under another name (directory abort), on master (undeclared-prefix crash), with 8 injected stage 3 errors (same messages, same order) and with an attribute missing dcterms:creator (same stage 2 line).
  • python3 -I tools/test_ogit_validate.py: 6 tests over 19 fixtures pass.
  • python3 -I tools/test_import_check.py: 13 tests pass. python3 -I tools/import_check.py: 0 errors.
  • cargo test (this crate): passes.
  • OGAR, with OGIT_FORK_PATH at this branch and OGAR#364 applied: ogar-action-handler (with ontology-cache) and ogar-from-schema tests pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4

claude added 12 commits October 28, 2026 02:03
tools/import_check.py checks what an importer needs beyond validate.sh:
declared verbs and targets for every ogit:allowed and ogit:from-to edge,
one verb declaration shape, file names, duplicate declarations, and
undeclared prefixes. It can write the verb catalogue and the allowed-edge
table. It is not wired into validate.sh or CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
ogit.Compliance: and ogit.FinancialMarket: were used on lines 28-29 but not
declared, which made the validator stop on this file before reporting
anything else. Both targets exist (NTO/Compliance/entities/SanctionsEntry.ttl,
NTO/FinancialMarket/entities/Country.ttl).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
The validator requires a file to be named after the term it declares. The
twelve verbs are lowercase (ogit.WorkOrder:hasActivity); their files were
capitalised (HasActivity.ttl). Content is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
Dummy.ttl, HiroTopology.ttl and Test.ttl use both prefixes without
declaring them. The validator's parser fills them in from its defaults;
other Turtle readers reject the files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
They were subclasses of ogit:Enumeration, which OGIT does not declare, and
Medication's ogit:allowed list targets five of them, which the validator
rejects as missing entities. Their bodies are already entity-shaped
(ogit:id, ogit:parent, attribute lists, ogit:allowed); only the superclass
and the missing dcterms:valid change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
- LabValue: loincRef is an attribute (its own description says so), so it
  moves from an ogit:allowed edge into optional-attributes.
- Patient: ogit:Tenant does not exist; OGIT records tenancy with the
  ogit:tenantId attribute, so the edge becomes that attribute.
- Assessment: ogit:assesses was undeclared. It becomes ogit.Healthcare:assesses
  in NTO/Healthcare/verbs, and ogit.Healthcare:AnatomicalStructure, its target,
  is declared as an entity.
- The three Assessment subclasses move into their own files.
- The inline attribute definitions move into NTO/Healthcare/attributes/,
  one file per attribute, as owl:DatatypeProperty + rdfs:subPropertyOf
  ogit:Attribute. Their IRIs are unchanged.
- Entities gain the dcterms:valid line validation0.rq requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
- The inline rdfs:Property blocks move into NTO/WorkOrder/attributes/, one
  file per attribute, as owl:DatatypeProperty + rdfs:subPropertyOf
  ogit:Attribute. IRIs are unchanged. Six attributes were defined in two or
  three entity files with different wording; each now has one description
  and lists every source column.
- Thirteen attributes were referenced but defined nowhere (geraet, logbuch,
  intern, aktion, details, dateiname, anKunde, sortOrder, posTyp, menge,
  einheit, einzelpreis, versteckt); they are declared from their entities'
  descriptions.
- ogit:created-at does not exist; references now use the existing
  ogit:createdAt.
- Entities gain the dcterms:valid / dcterms:creator / ogit:allowed lines
  validation0.rq requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
WorkOrder's twelve verbs are rdfs:Class + rdfs:subClassOf ogit:Verb with an
ogit:from-to list, the convention OGAR documents as verb-as-class. C02 now
reports only a verb that has neither that shape nor owl:ObjectProperty +
rdfs:subPropertyOf ogit:Verb.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
Descriptions now state each column's type, default and the behaviour WoA's
code gives it (portal and DATEV filters, picture collection), and each
dcterms:source names the exact model column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
The validator requires every file to sit below a directory named after its
namespace: for ogit:creates the path must match /ogit/(.*)/creates.ttl. The
470 files in SGO/sgo and the 27 in SGO/core are all in the root ogit:
namespace, so with more than one input file the validator stopped at the
first of them ("creates.ttl should be under directory ogit"). That layout
has been rejected since the 2019 validator jar.

They move to SGO/ogit/{attributes,entities,verbs} and SGO/ogit/core. No file
content changes, and no file name collides. The READMEs, two NTO READMEs and
the comment in validate.sh follow the new paths.

./validate.sh now reports "Validation successful." over all 3675 files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
impl Iterator returned from by_branch captures the borrowed branch argument,
which current rustc rejects (E0700). Adding + '_ states that capture and still
builds on the crate's rust-version, 1.70.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
Since they are now subclasses of ogit:Entity, OGAR's reader (which takes
entities from entities/ directories) and lance-graph-ontology's reader
(which takes direct subclasses of ogit:Entity) only agree on them when they
sit with the other entities.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
AdaWorldAPI pushed a commit to AdaWorldAPI/OGAR that referenced this pull request Oct 28, 2026
The OGIT fork moves every root-namespace file from SGO/sgo and SGO/core
to SGO/ogit, because its validator requires a file to sit below a directory
named after its namespace (AdaWorldAPI/OGIT#11). The tests and sgo.rs read
SGO/ogit paths now.

ontology_cache's ONLY_IN_THE_TABLES is re-measured on that branch:
- The four old entries are gone. FiscalJurisdiction and the three NTO/PTF
  entities now declare their prefixes, so the cache reads them too.
- The three Healthcare Assessment subclasses are new. They now have their
  own files, so OGAR's reader takes them, while the cache takes only direct
  subclasses of ogit:Entity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
@AdaWorldAPI AdaWorldAPI changed the title Add an import-time consistency check; declare FiscalJurisdiction's prefixes Make validate.sh pass: fix fork additions, move SGO under SGO/ogit, add an import check Oct 28, 2026
The validator's directory check matches against the file's canonical
path, not the path relative to the repository. A root-namespace file
such as SGO/sgo/entities/Node.ttl needs `/ogit/` somewhere in that path,
and a checkout named OGIT (as on GitHub CI, /home/runner/work/OGIT/OGIT)
provides it. The move to SGO/ogit was based on a run in a checkout with
a different directory name; with the original layout, a checkout named
OGIT validates successfully.

This reverts commit 7725f73.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
tools/ogit_validate.py reproduces bin/ogit-validator.jar's three stages
(per-file prefix, name and directory checks; the ten validation queries;
the entity registry checks) with the jar's messages, streams and exit
status, so the ontology can be validated without Java.

The behaviour was read from the jar's bytecode. Two findings decide the
output order: rdf4j's memory store commits through a sorted TreeModel,
so statements come back sorted rather than in file order, and the
directory check matches against the file's canonical path.

tools/test_ogit_validate.py runs 18 fixtures and compares the port with
output recorded from the jar (and with the jar itself when Java is
installed). On the whole repository the port and the jar print the same
result in a checkout named OGIT, in a checkout under another name, and
on master.

import_check.py's parser now keeps language tags and the datatypes of
bare numbers and booleans, which the port needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
AdaWorldAPI pushed a commit to AdaWorldAPI/OGAR that referenced this pull request Oct 28, 2026
AdaWorldAPI/OGIT#11 no longer moves SGO/sgo and SGO/core. The validator's
directory check reads the canonical path, and a checkout named OGIT (or
ogit, as this repository's CI checks it out) already satisfies it, so
the move was not needed. This restores the paths e88c46d changed.

What stays is the re-measured ONLY_IN_THE_TABLES list: OGIT#11 still
declares the prefixes FiscalJurisdiction.ttl and the NTO/PTF entities
lacked, and still gives the three Healthcare Assessment subclasses their
own files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
@AdaWorldAPI AdaWorldAPI changed the title Make validate.sh pass: fix fork additions, move SGO under SGO/ogit, add an import check Make validate.sh pass, add a Python port of the validator and an import check Oct 28, 2026
AdaWorldAPI pushed a commit to AdaWorldAPI/lance-graph that referenced this pull request Oct 28, 2026
Takes #1831's STATUS_BOARD section as merged and adds a dated note on
AdaWorldAPI/OGIT#11 to the plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
Eleven NTO/Accounting files from 2026-05-28 described themselves as
shortcuts over another application's internal field paths, and named its
models. An ontology term should say what it is, not how one program
stores it.

- Removed, unreferenced by any entity: verbs hasPickingType and
  hasProductCategory, attribute productCategoryComplete.
- Renamed commercialPartnerCountryCode to counterpartyCountryCode
  (JournalEntry's attribute lists follow).
- Rewrote the descriptions of the remaining files (fiscalCountryCode,
  iso3166Alpha2, isEuMember, vatRateStandard, FiscalJurisdiction,
  JournalEntry, hasFiscalCountry) to describe the term itself.

No code in lance-graph, OGAR or the consumer repositories reads the
removed or renamed IRIs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
AdaWorldAPI pushed a commit to AdaWorldAPI/OGAR that referenced this pull request Oct 28, 2026
OGIT#11 removes three Accounting files that no entity referenced (two
verbs and one attribute), so the domain's inventory floor drops from 36
to 33. The floor still fails if Accounting shrinks further.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPWTwovjYJEUMuGPb4
@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review October 28, 2026 03:16
@AdaWorldAPI
AdaWorldAPI merged commit cb94c6f into master Oct 28, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T03:17:58.674917Z 0bf9a80 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

AdaWorldAPI added a commit to AdaWorldAPI/OGAR that referenced this pull request Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants