From 6329fbe4867940058053eebb89a22edb60c658f6 Mon Sep 17 00:00:00 2001 From: nettee Date: Mon, 14 Sep 2026 20:35:12 +0800 Subject: [PATCH 1/2] feat(issue-spec): support standalone file archives --- README.md | 16 +- docs/issue-spec-representation.md | 44 +++- e2e/tests/test_issue_dump_load.py | 181 +++++++++++++- lib/spec-manager.js | 224 +++++++++++++++++- scripts/ci/archive-old-specs.js | 70 ++++-- scripts/ci/test-archive-old-specs.js | 53 +++++ .../design.md | 20 ++ .../implementation.md | 25 ++ .../spec.md | 64 +++++ 9 files changed, 653 insertions(+), 44 deletions(-) create mode 100644 specs/change/20260914-standalone-file-issue-spec-v3/design.md create mode 100644 specs/change/20260914-standalone-file-issue-spec-v3/implementation.md create mode 100644 specs/change/20260914-standalone-file-issue-spec-v3/spec.md diff --git a/README.md b/README.md index 1b88a90..a1773b8 100644 --- a/README.md +++ b/README.md @@ -115,10 +115,24 @@ The `zest-dev` CLI manages spec files. Use it to inspect and update specs outsid | `zest-dev unset-active` | Unset active change spec | | `zest-dev update ` | Update spec status | | `zest-dev create-branch` | Create a git branch from the active change spec | -| `zest-dev dump [--dry-run]` | Archive a spec as an issue representation or GitHub issue | +| `zest-dev dump [--dry-run]` | Archive a directory Spec or standalone dated Markdown record as an issue representation or GitHub issue | | `zest-dev load [issue] [--from-file ]` | Reconstruct a spec from an issue representation or GitHub issue | | `zest-dev ralph` | Convert active Spec Progress items into Ralph tasks | +### Issue Spec Representation Compatibility + +Issue Spec Representation evolves without making existing archives unreadable: + +| Protocol | Represented source | `dump` behavior | `load` compatibility | Restored shape | +|----------|--------------------|-----------------|----------------------|----------------| +| V1 | Directory with `spec.md` | No longer emitted | Supported | `specs/change//` | +| V2 | Directory with one or more Markdown files; `spec.md` is optional | Emitted for directory Specs | Supported | `specs/change//` | +| V3 | Standalone `YYYYMMDD-slug.md` record | Emitted for standalone files | Supported | `specs/change/.md` | + +For directory Specs, `dump` accepts the existing Spec ID, directory/Main Spec path, or `active`. For standalone files, it accepts the direct path, filename, or an unambiguous ID without `.md`. If both `specs/change//` and `specs/change/.md` exist, the bare ID is ambiguous and fails; pass an explicit path to select one. `load` validates the protocol and refuses to overwrite either the target shape or a conflicting directory/file with the same logical ID. + +See [Issue Spec Representation](docs/issue-spec-representation.md) for the body/comment protocol and validation rules. + ### Status Transitions Valid status values: `new`, `designed`, `planned`, `implemented` diff --git a/docs/issue-spec-representation.md b/docs/issue-spec-representation.md index f2b8a84..31f429c 100644 --- a/docs/issue-spec-representation.md +++ b/docs/issue-spec-representation.md @@ -4,7 +4,7 @@ This document defines the forge-neutral issue representation used by `zest-dev d ## Purpose -An Issue Spec Representation stores one complete Markdown file snapshot of a Zest Dev Spec directory in one forge issue. It is a snapshot format, not a synchronization protocol. +An Issue Spec Representation stores either one complete Markdown snapshot of a Zest Dev Spec directory or one standalone dated Markdown record in one forge issue. It is a snapshot format, not a synchronization protocol. The representation is designed for GitHub and Forgejo issue primitives: - issue title @@ -16,7 +16,7 @@ Load correctness depends only on protocol headers and Markdown content in the is ## Versions -`dump` writes protocol version `2`. `load` accepts versions `1` and `2` so existing archives remain loadable. +`dump` writes protocol version `2` for Spec directories and version `3` for standalone Markdown files. `load` accepts versions `1`, `2`, and `3` so existing archives remain loadable. V1 and V2 always reconstruct directories; V3 always reconstructs a standalone file. Every protocol body/comment starts with a leading HTML comment header. A V2 issue body for a normal Spec begins like this: @@ -99,6 +99,31 @@ Comment order is not meaningful. Comments without a leading protocol header are Before writing any files, `load` verifies that the body payload and protocol comments match the manifest exactly. Missing, duplicate, or unlisted represented paths fail instead of producing a partial directory. +## V3 Standalone File + +V3 represents exactly one dated Markdown file that is a direct child of `specs/change`: + +```markdown + +# Historical change record +``` + +The V3 body header has exactly four fields: + +- `zest-dev-issue-spec` must be `3`. +- `kind` must be `standalone-file`. +- `spec-id` must be a valid dated Spec ID without `.md`. +- `filename` must be exactly `.md`. + +The complete file content starts after the header separator and may be empty. V3 has no protocol file comments because its one file is wholly represented in the issue body. Ordinary non-protocol issue comments are ignored during load. + +`dump` accepts the standalone path, its filename, or its ID without `.md`. A bare ID is accepted only when exactly one of `specs/change//` and `specs/change/.md` exists. When both exist, the ID is ambiguous and fails; an explicit path selects the intended source. + ## V1 Load Compatibility V1 archives do not contain a directory manifest. Their issue body header has `path: spec.md`, and the body payload is required to contain the Main Spec File: @@ -165,7 +190,7 @@ In V2, `body-path` must be one of the manifested paths. It describes storage loc ## Spec Identity And Local Write -The loaded Spec identity comes from the protocol header `spec-id`, not from `spec.md` frontmatter, title, labels, issue number, or URL. +The loaded Spec identity comes from the protocol header `spec-id`, not from `spec.md` frontmatter, title, labels, issue number, or URL. For V3, `filename` must agree with that identity. The `spec-id` must be a valid Spec directory name: @@ -173,7 +198,7 @@ The `spec-id` must be a valid Spec directory name: YYYYMMDD- ``` -`load` creates: +V1 and V2 `load` create: ```text specs/change// @@ -183,6 +208,14 @@ Every protocol comment must use the same version and `spec-id` as the issue body `load` does not change `specs/change/active`. For a directory without `spec.md`, successful output reports the Spec directory path rather than a nonexistent Main Spec File path. +V3 `load` creates the exact standalone target: + +```text +specs/change/.md +``` + +It fails if that target file or a same-ID target directory already exists. The file is written to a temporary sibling and renamed into place only after validation and a successful write. A loaded standalone historical record is never made active. + ## Local Representation Mode The same body/comment mapping can be represented locally as YAML: @@ -222,9 +255,10 @@ The protocol is fail-fast: - V2 missing, duplicate, or unlisted represented files fail - unexpected V2 body content without `body-path` fails - V1 body paths other than `spec.md` or empty `spec.md` content fail +- V3 kinds other than `standalone-file`, mismatched filenames, extra metadata fields, or protocol comments fail - invalid file paths fail - invalid UTF-8 Markdown content fails -- existing target Spec directories fail +- existing target shapes or conflicting same-ID directory/file shapes fail - unsupported forge transports fail - failed remote issue or comment operations fail diff --git a/e2e/tests/test_issue_dump_load.py b/e2e/tests/test_issue_dump_load.py index d70653a..25394c5 100644 --- a/e2e/tests/test_issue_dump_load.py +++ b/e2e/tests/test_issue_dump_load.py @@ -1,5 +1,6 @@ import json import os +import shutil import stat from pathlib import Path @@ -116,6 +117,145 @@ def test_v2_directory_manifest_round_trips_without_main_file_and_loads_v1(cli): } +def test_v3_standalone_file_round_trips_by_path_and_unambiguous_id(cli): + spec_id = "20260101-legacy-record" + filename = f"{spec_id}.md" + source_path = cli.project_dir / "specs" / "change" / filename + source_path.parent.mkdir(parents=True) + source_bytes = "# Historical change record\n\n你好，世界。\n".encode() + source_path.write_bytes(source_bytes) + + by_path = cli.yaml("dump", str(source_path), "--dry-run") + by_id = cli.yaml("dump", spec_id, "--dry-run") + assert by_path == by_id + issue = by_path["issue"] + assert protocol_header(issue["body"]) == { + "zest-dev-issue-spec": 3, + "kind": "standalone-file", + "spec-id": spec_id, + "filename": filename, + } + assert issue["body"].endswith(source_bytes.decode()) + assert issue["comments"] == [] + + source_path.rename(source_path.with_suffix(".source")) + dump_path = cli.project_dir / "standalone-v3.yml" + load_issue = {**issue, "comments": ["Archive discussion without protocol metadata."]} + dump_path.write_text(yaml.safe_dump(load_issue, sort_keys=False), encoding="utf-8") + loaded = cli.yaml("load", "--from-file", str(dump_path)) + + assert loaded["spec"] == { + "id": spec_id, + "path": f"specs/change/{filename}", + "active": False, + "status": "new", + } + assert source_path.read_bytes() == source_bytes + + +def test_v3_standalone_file_fails_fast_for_ambiguity_collisions_and_corruption(cli): + spec_id = "20260102-ambiguous-record" + filename = f"{spec_id}.md" + specs_dir = cli.project_dir / "specs" / "change" + standalone_path = specs_dir / filename + directory_path = specs_dir / spec_id + directory_path.mkdir(parents=True) + (directory_path / "notes.md").write_text("# Directory\n", encoding="utf-8") + standalone_path.write_text("# Standalone\n", encoding="utf-8") + + assert "Ambiguous Spec identifier" in cli.fail("dump", spec_id, "--dry-run") + assert cli.yaml("dump", str(standalone_path), "--dry-run")["ok"] is True + assert cli.yaml("dump", str(directory_path), "--dry-run")["ok"] is True + + shutil.rmtree(directory_path) + dumped = cli.yaml("dump", spec_id, "--dry-run")["issue"] + dump_path = cli.project_dir / "standalone-collision.yml" + dump_path.write_text(yaml.safe_dump(dumped, sort_keys=False), encoding="utf-8") + assert "Target standalone Spec file already exists" in cli.fail( + "load", "--from-file", str(dump_path) + ) + + standalone_path.unlink() + directory_path.mkdir() + assert "Conflicting target Spec directory already exists" in cli.fail( + "load", "--from-file", str(dump_path) + ) + + shutil.rmtree(directory_path) + invalid_cases = { + "wrong-kind": protocol_document( + { + "zest-dev-issue-spec": 3, + "kind": "directory", + "spec-id": spec_id, + "filename": filename, + }, + "# Body\n", + ), + "wrong-filename": protocol_document( + { + "zest-dev-issue-spec": 3, + "kind": "standalone-file", + "spec-id": spec_id, + "filename": "20260102-other.md", + }, + "# Body\n", + ), + "extra-metadata": protocol_document( + { + "zest-dev-issue-spec": 3, + "kind": "standalone-file", + "spec-id": spec_id, + "filename": filename, + "files": [filename], + }, + "# Body\n", + ), + } + for name, body in invalid_cases.items(): + invalid_path = cli.project_dir / f"{name}.yml" + invalid_path.write_text( + yaml.safe_dump({"body": body, "comments": []}, sort_keys=False), + encoding="utf-8", + ) + assert cli.run("load", "--from-file", str(invalid_path)).returncode != 0 + + protocol_comment_path = cli.project_dir / "protocol-comment.yml" + protocol_comment_path.write_text( + yaml.safe_dump( + { + "body": protocol_document( + { + "zest-dev-issue-spec": 3, + "kind": "standalone-file", + "spec-id": spec_id, + "filename": filename, + }, + "# Body\n", + ), + "comments": [ + protocol_document( + {"zest-dev-issue-spec": 3, "spec-id": spec_id}, + "# Unexpected protocol payload\n", + ) + ], + }, + sort_keys=False, + ), + encoding="utf-8", + ) + assert "must not contain protocol comments" in cli.fail( + "load", "--from-file", str(protocol_comment_path) + ) + + +def test_dump_rejects_invalid_utf8_standalone_file(cli): + path = cli.project_dir / "specs" / "change" / "20260103-invalid-utf8.md" + path.parent.mkdir(parents=True) + path.write_bytes(b"\xff") + assert "Invalid UTF-8 Markdown file" in cli.fail("dump", str(path), "--dry-run") + + def test_dump_and_load_round_trip_yaml_sensitive_markdown_paths(cli): created = cli.yaml("create", "yaml-sensitive-paths")["spec"] source_id = created["id"] @@ -319,13 +459,19 @@ def test_v2_manifest_fails_fast_for_incomplete_or_invalid_representations(cli): assert expected_error in cli.fail("load", "--from-file", str(path)) -@pytest.mark.parametrize("with_spec_md", [True, False]) -def test_github_transport_uses_gh_and_reports_comment_failure(cli, with_spec_md): +@pytest.mark.parametrize("source_shape", ["directory-with-main", "directory-without-main", "standalone"]) +def test_github_transport_uses_gh_and_reports_comment_failure(cli, source_shape): created = cli.yaml("create", "github-dump-source")["spec"] spec_dir = cli.project_dir / "specs" / "change" / created["id"] (spec_dir / "notes.md").write_text("# Notes\n", encoding="utf-8") - if not with_spec_md: + spec_identifier = created["id"] + standalone_path = cli.project_dir / "specs" / "change" / f"{created['id']}.md" + if source_shape == "directory-without-main": (spec_dir / "spec.md").unlink() + elif source_shape == "standalone": + shutil.rmtree(spec_dir) + standalone_path.write_text("# Standalone archive\n", encoding="utf-8") + spec_identifier = str(standalone_path) fake_bin = cli.project_dir / "fake-bin" fake_bin.mkdir() @@ -373,35 +519,44 @@ def test_github_transport_uses_gh_and_reports_comment_failure(cli, with_spec_md) fake_gh.chmod(fake_gh.stat().st_mode | stat.S_IXUSR) env = {"PATH": f"{fake_bin}{os.pathsep}{os.environ['PATH']}", "GH_LOG": str(log_path)} - dumped = cli.yaml("dump", created["id"], env=env) + dumped = cli.yaml("dump", spec_identifier, env=env) assert dumped["ok"] is True assert dumped["issue"]["url"] == "https://github.com/nettee/zest-dev/issues/123" assert dumped["issue"]["closed"] is True log_entries = [yaml.safe_load(line) for line in log_path.read_text(encoding="utf-8").splitlines()] assert log_entries[1]["args"][:2] == ["issue", "create"] assert "--label" in log_entries[1]["args"] - assert log_entries[2]["args"][:2] == ["issue", "comment"] + if source_shape != "standalone": + assert log_entries[2]["args"][:2] == ["issue", "comment"] assert log_entries[-1]["args"] == ["issue", "close", "https://github.com/nettee/zest-dev/issues/123"] fail_env = {**env, "FAIL_COMMENT": "1"} - assert "created issue before failure: https://github.com/nettee/zest-dev/issues/123" in cli.fail( - "dump", created["id"], env=fail_env - ) + if source_shape != "standalone": + assert "created issue before failure: https://github.com/nettee/zest-dev/issues/123" in cli.fail( + "dump", spec_identifier, env=fail_env + ) close_fail_env = {**env, "FAIL_CLOSE": "1"} assert "created issue before failure: https://github.com/nettee/zest-dev/issues/123" in cli.fail( - "dump", created["id"], env=close_fail_env + "dump", spec_identifier, env=close_fail_env ) - dry_run = cli.yaml("dump", created["id"], "--dry-run") + dry_run = cli.yaml("dump", spec_identifier, "--dry-run") body_path = cli.project_dir / "issue-body.md" comments_path = cli.project_dir / "issue-comments.yml" body_path.write_text(dry_run["issue"]["body"], encoding="utf-8") comments_path.write_text(json.dumps(dry_run["issue"]["comments"]), encoding="utf-8") load_env = {**env, "ISSUE_BODY": str(body_path), "ISSUE_COMMENTS": str(comments_path)} - source_files = markdown_files(spec_dir) - spec_dir.rename(spec_dir.with_name(f"{created['id']}.source")) + if source_shape == "standalone": + source_bytes = standalone_path.read_bytes() + standalone_path.rename(standalone_path.with_suffix(".source")) + else: + source_files = markdown_files(spec_dir) + spec_dir.rename(spec_dir.with_name(f"{created['id']}.source")) loaded = cli.yaml("load", "123", env=load_env) assert loaded["ok"] is True assert loaded["source"] == {"type": "github", "issue": "123"} - assert markdown_files(cli.project_dir / "specs" / "change" / created["id"]) == source_files + if source_shape == "standalone": + assert standalone_path.read_bytes() == source_bytes + else: + assert markdown_files(cli.project_dir / "specs" / "change" / created["id"]) == source_files diff --git a/lib/spec-manager.js b/lib/spec-manager.js index 5e57821..c5c98a6 100644 --- a/lib/spec-manager.js +++ b/lib/spec-manager.js @@ -18,8 +18,13 @@ const STATUS_ORDER = { }; const ISSUE_SPEC_PROTOCOL_V1 = 1; const ISSUE_SPEC_PROTOCOL_V2 = 2; +const ISSUE_SPEC_PROTOCOL_V3 = 3; const ISSUE_SPEC_WRITE_PROTOCOL_VERSION = ISSUE_SPEC_PROTOCOL_V2; -const SUPPORTED_ISSUE_SPEC_PROTOCOL_VERSIONS = [ISSUE_SPEC_PROTOCOL_V1, ISSUE_SPEC_PROTOCOL_V2]; +const SUPPORTED_ISSUE_SPEC_PROTOCOL_VERSIONS = [ + ISSUE_SPEC_PROTOCOL_V1, + ISSUE_SPEC_PROTOCOL_V2, + ISSUE_SPEC_PROTOCOL_V3 +]; const ISSUE_SPEC_LABELS = ['spec:change', 'archive']; function pathExistsIncludingDanglingSymlink(filePath) { @@ -180,6 +185,96 @@ function normalizeSpecIdentifier(specIdentifier) { return basename; } +function pathEntryType(entryPath) { + try { + const stat = fs.lstatSync(entryPath); + if (stat.isFile()) return 'file'; + if (stat.isDirectory()) return 'directory'; + return 'unsupported'; + } catch (error) { + if (error && error.code === 'ENOENT') return null; + throw error; + } +} + +function directoryDumpSource(specIdentifier) { + const spec = getSpec(specIdentifier); + validateSpecId(spec.id); + return { + kind: 'directory', + id: spec.id, + path: path.join(SPECS_DIR, spec.id) + }; +} + +function standaloneDumpSource(specIdentifier) { + const inputPath = String(specIdentifier); + const resolvedPath = path.resolve(inputPath.includes('/') || inputPath.includes('\\') + ? inputPath + : path.join(SPECS_DIR, inputPath)); + const specsDirPath = path.resolve(SPECS_DIR); + const filename = path.basename(resolvedPath); + + if (path.dirname(resolvedPath) !== specsDirPath) { + throw new Error(`Standalone Spec file must be a direct child of ${SPECS_DIR}: ${inputPath}`); + } + if (!/^\d{8}-[a-z0-9][a-z0-9-]*\.md$/.test(filename)) { + throw new Error(`Invalid standalone Spec filename "${filename}"`); + } + + const entryType = pathEntryType(resolvedPath); + if (entryType === null) { + throw new Error(`Standalone Spec file not found: ${inputPath}`); + } + if (entryType !== 'file') { + throw new Error(`Standalone Spec path is not a regular file: ${inputPath}`); + } + + const id = filename.slice(0, -'.md'.length); + validateSpecId(id); + return { kind: 'standalone-file', id, path: resolvedPath, filename }; +} + +function resolveDumpSource(specIdentifier) { + if (specIdentifier === 'active' || specIdentifier === 'current') { + return directoryDumpSource(specIdentifier); + } + + const input = String(specIdentifier); + const basename = path.basename(input); + if (basename === 'spec.md' || basename === 'README.md') { + return directoryDumpSource(input); + } + if (basename.endsWith('.md')) { + return standaloneDumpSource(input); + } + + const isBareIdentifier = input === basename; + if (!isBareIdentifier) { + return directoryDumpSource(input); + } + + validateSpecId(input); + const directoryPath = path.join(SPECS_DIR, input); + const standalonePath = path.join(SPECS_DIR, `${input}.md`); + const directoryType = pathEntryType(directoryPath); + const standaloneType = pathEntryType(standalonePath); + + if (directoryType && directoryType !== 'directory') { + throw new Error(`Spec directory path has unsupported file type: ${directoryPath}`); + } + if (standaloneType && standaloneType !== 'file') { + throw new Error(`Standalone Spec path is not a regular file: ${standalonePath}`); + } + if (directoryType && standaloneType) { + throw new Error(`Ambiguous Spec identifier "${input}": both ${directoryPath} and ${standalonePath} exist`); + } + if (standaloneType) { + return standaloneDumpSource(`${input}.md`); + } + return directoryDumpSource(input); +} + function normalizeIssueSpecPath(relativePath) { if (typeof relativePath !== 'string' || relativePath.trim() === '') { throw new Error('Invalid Issue Spec path: path is required'); @@ -270,6 +365,18 @@ function renderIssueSpecManifest(specId, markdownFiles, bodyPath, bodyContent) { return renderIssueSpecDocument(metadata, bodyContent); } +function renderStandaloneIssueSpec(source, content) { + return renderIssueSpecDocument( + { + 'zest-dev-issue-spec': ISSUE_SPEC_PROTOCOL_V3, + kind: 'standalone-file', + 'spec-id': source.id, + filename: source.filename + }, + content + ); +} + function parseIssueSpecDocument(text, location, { requireProtocol = true } = {}) { if (typeof text !== 'string') { throw new Error(`${location} must be a string`); @@ -326,10 +433,18 @@ function parseIssueSpecFile(text, location, { requireProtocol = true } = {}) { } function specToIssueRepresentation(specIdentifier) { - const spec = getSpec(specIdentifier); - validateSpecId(spec.id); + const source = resolveDumpSource(specIdentifier); + if (source.kind === 'standalone-file') { + const content = readMarkdownFile(path.dirname(source.path), source.filename); + return { + title: `[archive] ${source.id}`, + labels: [...ISSUE_SPEC_LABELS], + body: renderStandaloneIssueSpec(source, content), + comments: [] + }; + } - const specDirPath = path.join(SPECS_DIR, spec.id); + const specDirPath = source.path; const markdownFiles = walkSpecDirectory(specDirPath).sort(); if (markdownFiles.length === 0) { throw new Error('Issue Spec Representation requires at least one Markdown file'); @@ -343,7 +458,7 @@ function specToIssueRepresentation(specIdentifier) { ? readMarkdownFile(specDirPath, bodyPath) : ''; const body = renderIssueSpecManifest( - spec.id, + source.id, markdownFiles, bodyPath, bodyContent @@ -352,14 +467,14 @@ function specToIssueRepresentation(specIdentifier) { .filter(relativePath => relativePath !== bodyPath) .map(relativePath => { return renderIssueSpecFile( - spec.id, + source.id, relativePath, readMarkdownFile(specDirPath, relativePath) ); }); return { - title: `[archive] ${spec.id}`, + title: `[archive] ${source.id}`, labels: [...ISSUE_SPEC_LABELS], body, comments @@ -401,6 +516,7 @@ function issueRepresentationV1ToFiles(bodyFile, comments) { } return { + kind: 'directory', specId: bodyFile.specId, files: [...files.entries()].map(([relativePath, content]) => ({ path: relativePath, content })) }; @@ -467,11 +583,51 @@ function issueRepresentationV2ToFiles(bodyDocument, comments) { } return { + kind: 'directory', specId: bodyDocument.specId, files: manifestPaths.map(relativePath => ({ path: relativePath, content: files.get(relativePath) })) }; } +function issueRepresentationV3ToStandalone(bodyDocument, comments) { + const metadataKeys = Object.keys(bodyDocument.metadata).sort(); + const expectedKeys = ['filename', 'kind', 'spec-id', 'zest-dev-issue-spec']; + if (metadataKeys.length !== expectedKeys.length || metadataKeys.some((key, index) => key !== expectedKeys[index])) { + throw new Error('Invalid Issue Spec V3 metadata fields'); + } + if (bodyDocument.metadata.kind !== 'standalone-file') { + throw new Error('Issue Spec V3 kind must be standalone-file'); + } + + const filename = bodyDocument.metadata.filename; + const expectedFilename = `${bodyDocument.specId}.md`; + if (filename !== expectedFilename) { + throw new Error(`Issue Spec V3 filename must be ${expectedFilename}`); + } + + for (const [index, comment] of comments.entries()) { + const commentBody = typeof comment === 'string' ? comment : comment && comment.body; + const parsed = parseIssueSpecDocument(commentBody, `issue comment ${index + 1}`, { + requireProtocol: false + }); + if (parsed) { + throw new Error(`Issue Spec V3 must not contain protocol comments (issue comment ${index + 1})`); + } + } + + const encoded = Buffer.from(bodyDocument.content, 'utf-8'); + if (encoded.toString('utf-8') !== bodyDocument.content) { + throw new Error('Invalid UTF-8 Markdown content in Issue Spec V3 body'); + } + + return { + kind: 'standalone-file', + specId: bodyDocument.specId, + filename, + content: bodyDocument.content + }; +} + function issueRepresentationToFiles(issueRepresentation) { if (!issueRepresentation || typeof issueRepresentation !== 'object') { throw new Error('Issue Spec Representation must be an object'); @@ -492,17 +648,29 @@ function issueRepresentationToFiles(issueRepresentation) { }; return issueRepresentationV1ToFiles(bodyFile, comments); } + if (bodyDocument.version === ISSUE_SPEC_PROTOCOL_V3) { + return issueRepresentationV3ToStandalone(bodyDocument, comments); + } return issueRepresentationV2ToFiles(bodyDocument, comments); } function writeIssueRepresentation(issueRepresentation, source) { - const { specId, files } = issueRepresentationToFiles(issueRepresentation); + const parsed = issueRepresentationToFiles(issueRepresentation); + if (parsed.kind === 'standalone-file') { + return writeStandaloneIssueRepresentation(parsed, source); + } + + const { specId, files } = parsed; const specDirPath = path.join(SPECS_DIR, specId); + const conflictingStandalonePath = path.join(SPECS_DIR, `${specId}.md`); const tempDirPath = path.join(SPECS_DIR, `.${specId}.tmp-${process.pid}-${Date.now()}`); - if (fs.existsSync(specDirPath)) { + if (pathExistsIncludingDanglingSymlink(specDirPath)) { throw new Error(`Target Spec directory already exists: ${specDirPath}`); } + if (pathExistsIncludingDanglingSymlink(conflictingStandalonePath)) { + throw new Error(`Conflicting standalone Spec file already exists: ${conflictingStandalonePath}`); + } try { fs.mkdirSync(tempDirPath, { recursive: true }); @@ -537,6 +705,44 @@ function writeIssueRepresentation(issueRepresentation, source) { }; } +function writeStandaloneIssueRepresentation(representation, source) { + const targetPath = path.join(SPECS_DIR, representation.filename); + const conflictingDirectoryPath = path.join(SPECS_DIR, representation.specId); + const tempPath = path.join( + SPECS_DIR, + `.${representation.filename}.tmp-${process.pid}-${Date.now()}` + ); + + if (pathExistsIncludingDanglingSymlink(targetPath)) { + throw new Error(`Target standalone Spec file already exists: ${targetPath}`); + } + if (pathExistsIncludingDanglingSymlink(conflictingDirectoryPath)) { + throw new Error(`Conflicting target Spec directory already exists: ${conflictingDirectoryPath}`); + } + + try { + fs.mkdirSync(SPECS_DIR, { recursive: true }); + fs.writeFileSync(tempPath, representation.content, 'utf-8'); + fs.renameSync(tempPath, targetPath); + } catch (error) { + if (pathExistsIncludingDanglingSymlink(tempPath)) { + fs.unlinkSync(tempPath); + } + throw error; + } + + return { + ok: true, + spec: { + id: representation.specId, + path: targetPath, + active: false, + status: 'new' + }, + source + }; +} + function loadIssueRepresentationFromFile(filePath) { if (!filePath) { throw new Error('Missing --from-file path'); diff --git a/scripts/ci/archive-old-specs.js b/scripts/ci/archive-old-specs.js index 4f8bd9c..27b76b7 100644 --- a/scripts/ci/archive-old-specs.js +++ b/scripts/ci/archive-old-specs.js @@ -38,22 +38,58 @@ function cutoffTimestamp(now = new Date(), maxAgeDays = 10) { return Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), now.getUTCDate()) - maxAgeDays * DAY_MS; } +function archiveSpecId(specIdentifier) { + return specIdentifier.endsWith('.md') ? specIdentifier.slice(0, -3) : specIdentifier; +} + +function archiveSourcePath(specsDir, specIdentifier) { + return path.join(specsDir, specIdentifier); +} + +function dumpIdentifier(specsDir, specIdentifier) { + return specIdentifier.endsWith('.md') + ? archiveSourcePath(specsDir, specIdentifier) + : specIdentifier; +} + +function removeArchivedSource(sourcePath) { + const stat = fs.lstatSync(sourcePath); + if (stat.isDirectory()) { + fs.rmSync(sourcePath, { recursive: true, force: false }); + return; + } + fs.unlinkSync(sourcePath); +} + function listEarliestSpecs({ specsDir = DEFAULT_SPECS_DIR, limit = 10 } = {}) { if (!fs.existsSync(specsDir)) { throw new Error(`Specs directory not found: ${specsDir}`); } - return fs.readdirSync(specsDir, { withFileTypes: true }) - .filter(entry => entry.isDirectory() && SPEC_ID_PATTERN.test(entry.name)) + const candidates = fs.readdirSync(specsDir, { withFileTypes: true }) + .filter(entry => ( + (entry.isDirectory() && SPEC_ID_PATTERN.test(entry.name)) || + (entry.isFile() && entry.name.endsWith('.md') && SPEC_ID_PATTERN.test(archiveSpecId(entry.name))) + )) .map(entry => entry.name) - .sort() - .slice(0, limit); + .sort((left, right) => archiveSpecId(left).localeCompare(archiveSpecId(right))); + + const seenIds = new Set(); + for (const candidate of candidates) { + const specId = archiveSpecId(candidate); + if (seenIds.has(specId)) { + throw new Error(`Ambiguous archive Spec ID: both ${specId} and ${specId}.md exist in ${specsDir}`); + } + seenIds.add(specId); + } + + return candidates.slice(0, limit); } function selectSpecsToArchive({ specsDir = DEFAULT_SPECS_DIR, now = new Date(), limit = 10, maxAgeDays = 10 } = {}) { const cutoff = cutoffTimestamp(now, maxAgeDays); return listEarliestSpecs({ specsDir, limit }) - .filter(specId => parseUtcDatePrefix(specId) < cutoff); + .filter(specIdentifier => parseUtcDatePrefix(archiveSpecId(specIdentifier)) < cutoff); } function findArchiveIssue(specId, { runner = execFileSync } = {}) { @@ -111,40 +147,42 @@ function commandErrorDetails(error) { return details[0] || 'unknown error'; } -function preflightSpecs(specIds, { specsDir = DEFAULT_SPECS_DIR, runner = execFileSync } = {}) { - // Date-named directories are candidates by contract; invalid candidates are +function preflightSpecs(specIdentifiers, { specsDir = DEFAULT_SPECS_DIR, runner = execFileSync } = {}) { + // Date-named directories and standalone Markdown files are candidates by contract; invalid candidates are // rejected here instead of being silently skipped or partially archived. - for (const specId of specIds) { + for (const specIdentifier of specIdentifiers) { try { - runner('zest-dev', ['dump', specId, '--dry-run'], { encoding: 'utf8' }); + runner('zest-dev', ['dump', dumpIdentifier(specsDir, specIdentifier), '--dry-run'], { encoding: 'utf8' }); } catch (error) { - const candidatePath = path.join(specsDir, specId); + const candidatePath = archiveSourcePath(specsDir, specIdentifier); throw new Error(`Archive preflight failed for ${candidatePath}: ${commandErrorDetails(error)}`); } } } function archiveSpecs({ specsDir = DEFAULT_SPECS_DIR, now = new Date(), limit = 10, maxAgeDays = 10, runner = execFileSync, postDumpIssueLookupAttempts = POST_DUMP_ISSUE_LOOKUP_ATTEMPTS, postDumpIssueLookupDelayMs = POST_DUMP_ISSUE_LOOKUP_DELAY_MS, sleep = sleepMs } = {}) { - const specIds = selectSpecsToArchive({ specsDir, now, limit, maxAgeDays }); - preflightSpecs(specIds, { specsDir, runner }); + const specIdentifiers = selectSpecsToArchive({ specsDir, now, limit, maxAgeDays }); + preflightSpecs(specIdentifiers, { specsDir, runner }); const archived = []; const skippedExistingIssue = []; const associatedIssues = []; - for (const specId of specIds) { + for (const specIdentifier of specIdentifiers) { + const specId = archiveSpecId(specIdentifier); + const sourcePath = archiveSourcePath(specsDir, specIdentifier); const existingIssue = findArchiveIssue(specId, { runner }); if (existingIssue) { - fs.rmSync(path.join(specsDir, specId), { recursive: true, force: false }); + removeArchivedSource(sourcePath); skippedExistingIssue.push(specId); associatedIssues.push({ specId, issueNumber: existingIssue.number }); continue; } - runner('zest-dev', ['dump', specId], { stdio: 'inherit' }); + runner('zest-dev', ['dump', dumpIdentifier(specsDir, specIdentifier)], { stdio: 'inherit' }); const archiveIssue = findArchiveIssueWithRetry(specId, { runner, attempts: postDumpIssueLookupAttempts, delayMs: postDumpIssueLookupDelayMs, sleep }); if (!archiveIssue) throw new Error(`Archive issue was not created for ${specId}`); - fs.rmSync(path.join(specsDir, specId), { recursive: true, force: false }); + removeArchivedSource(sourcePath); archived.push(specId); associatedIssues.push({ specId, issueNumber: archiveIssue.number }); } diff --git a/scripts/ci/test-archive-old-specs.js b/scripts/ci/test-archive-old-specs.js index 377b25c..2247356 100644 --- a/scripts/ci/test-archive-old-specs.js +++ b/scripts/ci/test-archive-old-specs.js @@ -30,6 +30,12 @@ function makeSpec(specsDir, specId) { fs.writeFileSync(path.join(dir, 'spec.md'), '# Test\n'); } +function makeStandaloneSpec(specsDir, specId) { + const filePath = path.join(specsDir, `${specId}.md`); + fs.writeFileSync(filePath, '# Historical record\n'); + return filePath; +} + function makeInvalidSpecDirectory(specsDir, specId) { const dir = path.join(specsDir, specId); fs.mkdirSync(dir, { recursive: true }); @@ -76,6 +82,51 @@ function testSelectsOnlyMoreThanTenDaysOld() { ); } +function testListsAndArchivesStandaloneSpecsByExactPath() { + const { specsDir } = fixture(); + const specId = '20260601-standalone'; + const sourcePath = makeStandaloneSpec(specsDir, specId); + const calls = []; + let dumped = false; + + assert.deepStrictEqual(listEarliestSpecs({ specsDir }), [`${specId}.md`]); + const result = archiveSpecs({ + specsDir, + now: new Date('2026-07-04T00:00:00Z'), + runner: (cmd, args) => { + calls.push({ cmd, args }); + if (cmd === 'zest-dev' && args[0] === 'dump') { + assert.strictEqual(args[1], sourcePath); + if (!args.includes('--dry-run')) dumped = true; + return ''; + } + if (cmd === 'gh' && args[0] === 'issue' && args[1] === 'list') { + return dumped ? '[{"number":456}]' : '[]'; + } + throw new Error(`unexpected command: ${cmd} ${args.join(' ')}`); + } + }); + + assert.deepStrictEqual(result, { + archived: [specId], + skippedExistingIssue: [], + associatedIssues: [{ specId, issueNumber: 456 }] + }); + assert.strictEqual(fs.existsSync(sourcePath), false); + assert.strictEqual(calls.filter(call => call.cmd === 'zest-dev').length, 2); +} + +function testRejectsAmbiguousDirectoryAndStandaloneId() { + const { specsDir } = fixture(); + const specId = '20260601-ambiguous'; + makeSpec(specsDir, specId); + makeStandaloneSpec(specsDir, specId); + assert.throws( + () => listEarliestSpecs({ specsDir }), + /Ambiguous archive Spec ID/ + ); +} + function testDeletesOnlyAfterSuccessfulDump() { const { specsDir } = fixture(); makeSpec(specsDir, '20260601-ok'); @@ -238,6 +289,8 @@ function testRunsGlobalZestDevDump() { function main() { testListsEarliestTenAndIgnoresActiveSymlinkEntry(); testSelectsOnlyMoreThanTenDaysOld(); + testListsAndArchivesStandaloneSpecsByExactPath(); + testRejectsAmbiguousDirectoryAndStandaloneId(); testDeletesOnlyAfterSuccessfulDump(); testPreflightRejectsMixedValidAndInvalidBatchWithoutSideEffects(); testExistingArchiveIssueDeletesWithoutDumpingAgain(); diff --git a/specs/change/20260914-standalone-file-issue-spec-v3/design.md b/specs/change/20260914-standalone-file-issue-spec-v3/design.md new file mode 100644 index 0000000..4465c9a --- /dev/null +++ b/specs/change/20260914-standalone-file-issue-spec-v3/design.md @@ -0,0 +1,20 @@ +# Design Record + +## Research Findings + +- Issue #151 requires lossless local and remote dump/load for `specs/change/YYYYMMDD-slug.md`, exact filename/content preservation, explicit directory/file protocol distinction, and visible failure for ambiguity, collisions, corruption, or data-loss risk. Source: https://github.com/nettee/zest-dev/issues/151. +- Spec discovery accepts only dated directories; general identifier normalization preserves `.md` for standalone paths, so both an explicit file path and its extensionless ID fail lookup. Source: `lib/spec-manager.js:37-46,163-181` and reproduced public CLI behavior. +- V2 serialization walks `specs/change/` as a directory, while loading always creates that directory and writes manifested relative files beneath it. Source: `lib/spec-manager.js:328-367,409-537`. +- V1 requires `spec.md`; V2 adds a complete directory manifest and supports directories without `spec.md`. Existing focused E2E coverage passes. Source: `docs/issue-spec-representation.md`; `e2e/tests/test_issue_dump_load.py`. +- Archive candidate discovery filters for directories and later removes `specs/change/`, so standalone files are invisible to automation. Source: `scripts/ci/archive-old-specs.js:41-57,127-149`. +- Inference: changing global Spec discovery would expose standalone historical records to lifecycle commands that require directory structure. A dump-specific source resolver keeps this compatibility boundary narrow. + +## Design Decisions + +- Directory dump remains V2 and V1/V2 load behavior remains unchanged. Standalone dump emits V3, and V3 is accepted only as a tagged standalone representation. Rationale: this is the smallest protocol extension and avoids reinterpreting existing directory archives. Premises: V1/V2 are already documented directory protocols and issue #151 requires preserving V2 behavior. +- A V3 issue body includes `zest-dev-issue-spec: 3`, `kind: standalone-file`, `spec-id`, exact `filename`, and the file content; protocol comments are not allowed. Rationale: a single body is complete and human-readable, while required shape metadata prevents load-time guessing. +- Dump source resolution accepts an explicit path/filename or a bare dated ID. A bare ID resolves only when exactly one of `/` and `.md` exists; both or neither fail. Explicit sources must be direct children of `specs/change`, match the dated filename/ID grammar, and have the expected filesystem type. Rationale: path inputs can disambiguate deliberate access, while bare IDs cannot silently choose one source. +- Parsing produces either directory files or one standalone target. V3 validates exact allowed metadata, matching `filename === .md`, absence of protocol comments, valid UTF-8, and non-conflicting target paths before writing through a temporary-file rename boundary. Both the target file and same-ID directory, including dangling symlink entries, are collisions. Rationale: issue #151 prioritizes losslessness and visible failure. +- Archive automation models candidates with logical ID, kind, and exact path. It includes dated directories and dated `.md` files, orders/limits by logical ID, rejects duplicate IDs before preflight, dumps explicit standalone paths, and removes only the resolved source after confirmed archival. Rationale: CLI support alone does not make directory-only automation complete. +- Local and fake-GitHub paths share the same representation builder/parser. Public-CLI E2E tests cover byte-for-byte content, explicit and bare identifiers, remote transport, corrupt V3 input, collisions, invalid UTF-8, and V1/V2 regression behavior; archive script tests cover discovery, ambiguity, and exact deletion. +- Planned file changes: `lib/spec-manager.js` for source/protocol/write dispatch; `e2e/tests/test_issue_dump_load.py` for CLI behavior; `scripts/ci/archive-old-specs.js` and its test for automation; `docs/issue-spec-representation.md` and `README.md` for protocol and compatibility documentation. diff --git a/specs/change/20260914-standalone-file-issue-spec-v3/implementation.md b/specs/change/20260914-standalone-file-issue-spec-v3/implementation.md new file mode 100644 index 0000000..0dc1a56 --- /dev/null +++ b/specs/change/20260914-standalone-file-issue-spec-v3/implementation.md @@ -0,0 +1,25 @@ +# Implementation + + + +## Outcome + +- Standalone dated Markdown records now dump as explicit V3 representations and load back to their exact filename/content through local or GitHub transport, while directory Specs continue to use V2 and V1/V2 loading remains compatible. +- Archive automation now discovers, preflights, archives, and removes standalone files by exact path, with fail-fast ambiguity and collision handling. +- README and protocol documentation describe supported identifiers and the V1/V2/V3 evolution and compatibility matrix. + +## Deviations + +None found. + +## Verification + +- EAG: `cd e2e && env UV_CACHE_DIR=/tmp/zest-dev-uv-cache uv run pytest -q tests/test_issue_dump_load.py -k 'standalone or v2_directory_manifest'` — 6 passed, 6 deselected. +- Local E2E: `env UV_CACHE_DIR=/tmp/zest-dev-uv-cache pnpm test:local` — 30 passed, 1 package-only test skipped. +- Package E2E: `env UV_CACHE_DIR=/tmp/zest-dev-uv-cache pnpm test:package` — 31 passed. +- Archive automation: `node scripts/ci/test-archive-old-specs.js` — passed. +- Diff hygiene: `git diff --check` — passed. + +## Spec Retrospective + +None. diff --git a/specs/change/20260914-standalone-file-issue-spec-v3/spec.md b/specs/change/20260914-standalone-file-issue-spec-v3/spec.md new file mode 100644 index 0000000..ec95886 --- /dev/null +++ b/specs/change/20260914-standalone-file-issue-spec-v3/spec.md @@ -0,0 +1,64 @@ +--- +id: 20260914-standalone-file-issue-spec-v3 +name: Standalone File Issue Spec V3 +status: implemented +created: '2026-09-14' +--- + +## Overview + +`zest-dev dump` and `load` preserve directory-based Specs through Issue Spec Representation V2, but repositories that adopted directory Specs incrementally may still contain standalone dated Markdown records such as `specs/change/20260101-legacy-record.md`. Those records cannot currently use local or GitHub archival because Spec discovery, serialization, loading, and archive automation assume a directory source. + +Add a V3 standalone-file representation that preserves the exact filename and UTF-8 Markdown content. Directory dumps remain V2, V1/V2 loading remains compatible, and V3 loading reconstructs the original standalone file. Explicit paths and unambiguous IDs are supported; ambiguous sources, collisions, invalid paths, corrupt protocol data, and data-loss risks fail visibly. README and protocol documentation describe the V1/V2/V3 evolution and compatibility. + +## Design + +### Summary + +Resolve dump inputs into an explicit directory or standalone-file source without changing the directory-oriented Spec lifecycle. Continue emitting V2 for directories and introduce a tagged V3 body representation for standalone files. Load dispatches by protocol version, validates the complete representation before writing, and atomically reconstructs the matching directory or file shape. Archive automation discovers both shapes and rejects duplicate logical IDs before mutation. + +See [design.md](./design.md) for the Design Record. + +### E2E Acceptance Gate (EAG) + +A standalone dated Markdown file can be dumped by explicit path and unambiguous ID, round-tripped byte-for-byte through local and fake-GitHub V3 transport, and restored as the original file while existing V1/V2 directory behavior remains green. Verification: `cd e2e && env UV_CACHE_DIR=/tmp/zest-dev-uv-cache uv run pytest -q tests/test_issue_dump_load.py -k 'standalone or v2_directory_manifest'`. + +## Plan + +### Ticket 1 (AFK): Add V3 standalone dump/load + +Goal: Round-trip standalone dated Markdown records without changing directory Spec behavior. +Scope: Add source resolution, V3 rendering/parsing, strict validation, collision handling, atomic file writes, and local/fake-GitHub E2E coverage. +Depends on: None + +### Ticket 2 (AFK): Extend archive automation + +Goal: Let old-spec automation archive standalone files safely. +Scope: Discover and sort directory/file candidates, reject duplicate logical IDs, pass resolvable identifiers to dump, delete the exact archived source, and extend script tests. +Depends on: Ticket 1 + +### Ticket 3 (AFK): EAG Validation + +Goal: Validate the completed V3 workflow and V1/V2 compatibility. +Scope: Run the Spec EAG and relevant archive tests against the public CLI behavior. +Depends on: Ticket 2 + +### Ticket 4 (AFK): Documentation Sync + +Goal: Make supported identifiers and protocol compatibility reviewable. +Scope: Document V3 protocol details and add a V1/V2/V3 dump/load evolution and compatibility table to README. +Depends on: Ticket 3 + +## Progress + +- [x] Ticket 1 (AFK): Add V3 standalone dump/load +- [x] Ticket 2 (AFK): Extend archive automation +- [x] Ticket 3 (AFK): EAG Validation +- [x] Ticket 4 (AFK): Documentation Sync +## Implementation + +See [implementation.md](./implementation.md). + +## Deferred Follow-Ups (DFU) + +None. From 36b99591fe2afcfc2e395f2a9b5c630474c3a304 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 14 Sep 2026 12:36:00 +0000 Subject: [PATCH 2/2] ci: auto bump patch version --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index c099efc..70b8f46 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "zest-dev", - "version": "1.0.14", + "version": "1.0.15", "description": "A lightweight, human-interactive development workflow for AI-assisted coding", "author": { "name": "nettee",