[ovhirup]

Fixes #35. Closes #37 and #38.

Problem

CONTRIBUTING.md said: "There is no separate test command — the workflow is the test suite." To check a change before pushing, you had to read .github/workflows/validate.yml, about 13.5 KB of inline bash and Python, and replay it step by step. In practice the first real run was in CI.

Change

• tools/validate.py runs everything the skill-lint job checked inline, with the same ::error:: messages:

  • manifests
  • plugin shape
  • SKILL.md structure
  • frontmatter YAML
  • revision agreement
  • agent assets
  • the Grok and Codex ports
  • the hero image

  Every category runs on each call, so one run shows every failing category, and the exit code is 1 if any failed. A category that was a bash step stops at its first problem, as the step did. Marker and revision checks stay exact whole-line matches (CONTRIBUTING rule 4). Without PyYAML it exits 2 with an install hint instead of skipping the YAML check.

• tools/test_validate.py copies the checkout and breaks it one way per test: a marker left only in prose, a frontmatter policy-rev that disagrees with the block, a port README revision that disagrees with its port, a missing asset, a plugin agents/ directory, a version mismatch, and the CLI exit code.

• validate.yml: the skill-lint job now runs python3 tools/validate.py in place of the inline steps. Job names are unchanged, so required status checks still match. The unit-test steps and the recipes job are untouched.

• CONTRIBUTING.md, docs/evals.md and the README layout tree now point to the command.

This edits a CI pipeline, so it is a change the policy says to review carefully.

Verification

• Parity with the old inline steps. I ran main's inline steps, extracted verbatim from validate.yml, and the new script on the same copy of the tree with 15 different breaks. The sorted ::error:: lines and the pass/fail result were identical in all 15 (ALL SAME). The breaks:
  • none (a clean tree)
  • a marker left only in prose
  • a frontmatter revision that disagrees with the block
  • a Grok README revision that disagrees with the port
  • a missing asset
  • a plugin agents/ directory
  • a version mismatch
  • a renamed Codex executor template section
  • LICENSE drift
  • a missing CHANGELOG heading
  • an unbalanced Grok code fence
  • a wrong revision in the evals file
  • a description over 1024 characters
  • a missing policy layer
  • an invalid marketplace JSON file
• python3 tools/validate.py: every category reports OK on this branch.
• cd tools && python3 -m unittest discover: 53 passed, up from 45, with none skipped.
• cd measured && python3 -m unittest test_measured: OK, and approve.py --self-check reports SOD: PASS.

Not done as written

#38 asks for a deliberately broken commit on a scratch branch, to show CI failing with the old message. I didn't push a broken branch to the shared remote. The 15-scenario parity run above covers the same claim locally, and this PR's own CI run shows the new step passing.

This branch is independent of #43 and #44. Whichever merges second may need a one-line rebase of the README tools line or docs/evals.md.

---

Generated by Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-02T08:02:10.094325Z	4b2bc33	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.