Repository navigation
[Benchmark] add mkfifo - #42
Merged
Merged
Conversation
Contributor
Author
Contributor
|
현재 Mode 파싱 부분이 매우 긴 문자열이 들어왔을 때 stack overflow가 발생할 수 있는 구현 구조입니다. stack overflow를 발생시키는 mode 예시: 위 케이스를 테스트 케이스에도 추가해주세요 |
Problem: TODOLIST lists mkfifo as open for contribution (issue prosyslab#14). Approach: Scope follows GNU src/mkfifo.c and gnulib modechange.c. MODE is specified as abstract syntax with Render and Meaning; the Core parser is proved to invert Render, so "invalid mode" is exact and a valid MODE has one meaning. Each operand is one CreateSpecialNode call and, with -m, one non-following SetFileMode call (GNU lchmod). -Z/--context are left out. Boundaries: new files under bench/utils/mkfifo only; no shared IO change. Validation: make -C bench/utils/mkfifo verify -> 85 declarations, 946 conditions, 0 failed (DAFNY_VERIFICATION_OUTCOME=verified). A local differential smoke run of 36 cases against the pinned GNU build matched. Limitations: work in progress. Tests.py and the fuzzer generator are not written yet; make check and the fuzzing campaigns have not been run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: the mkfifo scaffold had no runtime cases, and the ambiguous empty
long option ("--=", "--=x") did not list GNU's possibilities.
Approach: port tests/misc/mknod.sh (octal and symbolic -m under umask 777)
and add cases for default modes, MODE expressions under several umasks,
invalid and special-bit MODEs, existing names, path errors, an unwritable
directory, partial effects, quoting, option forms, information requests,
parse errors and /dev/full streams. Fix the ambiguous-option text in Spec.
Boundaries: bench/utils/mkfifo only (Tests.py and MkfifoSpec.dfy).
Validation: make -C bench/utils/mkfifo test -> 145 passed, 6 deselected;
dafny-benchmark verify MkfifoSpec.dfy (isolated, 30 s) -> 0 errors.
Limitations: the fuzzer generator, make check and the fuzzing campaigns
are still to come. A failed permission change after a successful creation
cannot be triggered by these fixtures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: the fuzzer had no mkfifo generator, and its filesystem snapshot rejected FIFO nodes as unsupported, so no mkfifo case could be compared. Approach: add generators/mkfifo.rs (MODE in four spellings from chmod's mode generator plus mkfifo edge cases, new/existing/nested operands, options after operands, malformed options and information requests, and 29 deterministic scenarios) and register it with -m/--mode as value flags. Record FIFOs in fs_snapshot.rs from metadata only, without opening them; character devices are still rejected. Add a regression test. Boundaries: shared fuzzer snapshot code gains one node kind; no change to comparison rules or other utilities' generators. Validation: cargo test --release -> 500 passed; cargo fmt --check clean. run.py fuzz mkfifo --seeds 1,7,19 --iterations 1000 -> each seed match=1000 mismatch=0 timeout=0 other_errors=0, option pairs 6/6. Limitations: the generator does not create pre-existing FIFOs in its fixtures; Tests.py covers that case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: benchmark.yaml was still the scaffold placeholder, so make check stopped at "source details are incomplete", and mkfifo.md left the frame and the evidence section as TODO. Approach: record GNU coreutils mkfifo, its URL and GPL-3.0-or-later license (checked in the src/mkfifo.c header) as verified source details; state the RunCore frame; summarize the measured tests, fuzzing, verification and final-gate results without evaluator inputs or reference answers. Boundaries: bench/utils/mkfifo/benchmark.yaml and mkfifo.md only. Validation: python3 -m benchmarks validate mkfifo -> mkfifo: valid; make check TASK=mkfifo (on da69888 plus this benchmark.yaml) -> mkfifo: checks passed. Limitations: the PR evidence will be rerun on the final commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: the specification relied on the RunCore frame for standard output and did not state that a normal run leaves it unchanged; review on earlier PRs asked specifications to state what does not change. The scope document also lacked the natural-language description from the GNU manual. Approach: add `io.stdout() == old(io.stdout())` to MkfifoSpec.Spec and carry it through MkfifoProof.Complete and MkfifoCore.RunCore; summarize coreutils.texi "mkfifo invocation" (lines 10848-10886) at the top of mkfifo.md and reword the frame sentence. Boundaries: bench/utils/mkfifo only. Validation: dafny-benchmark verify (isolated, 30 s) on MkfifoSpec.dfy, the Complete lemma, MkfifoCore.RunCore and Mkfifo.RunCore -> 0 errors; python3 -m benchmarks validate mkfifo -> mkfifo: valid. A mutation check of 16 incorrect implementations (controls passing) failed verification in every case, including one that writes to standard output. Limitations: the full evidence is rerun on this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: the evidence section of mkfifo.md named commit da69888 and a verification count of 946 conditions. Adding the unchanged-stdout clause in 8f55d4a raised the count to 955, so the document no longer matched the commit it ships with, and any later revision would make it stale again. Approach: point to the pull request for the tested commit, artifact hashes, fuzzer image and full outputs; keep the base revision, tool versions, commands and the results that do not depend on the revision; state that every declaration verifies instead of giving counts. Boundaries: bench/utils/mkfifo/mkfifo.md only. Validation: python3 -m benchmarks validate mkfifo -> mkfifo: valid. Limitations: none; the PR evidence is rerun on this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: Review of prosyslab#42: the MODE parser and Meaning recursed once per character, so a long MODE overflowed the stack. A MODE of 50,000 characters (`0...0600` or `u...u+r`) crashed with "Stack overflow", and 20,000 characters took 12 s because every step copied the rest of the string. Approach: The recursive functions stay as the definitions the proofs use. ParseMode gets a `by method` body that walks indices of the string, and DecideMode calls ComputeMeaning instead of Spec.Meaning; each loop is proved to return the value of the function it replaces. Octal digits stop at the first value above 07777, as mode_compile does. Small helper methods hide the other spec definitions so that each step verifies within the time limit. mkfifo.md describes this and the new base revision. Boundaries: MkfifoModeCore.dfy, one call in MkfifoCore.dfy and mkfifo.md. The specification and the proofs are unchanged. Validation: make -C bench/utils/mkfifo verify -> 112 declarations, 1,542 conditions, 0 failed; make check TASK=mkfifo passes. A MODE of 100,000 characters now finishes in 0.3 s. Three incorrect variants of the new loops (trailing comma accepted, 07777 limit off by one, last clause skipped) each fail verification. Limitations: The recursive functions are still compiled, but no run-time path calls them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: Review of prosyslab#42 asked for the long MODE cases as tests. Approach: Eight GNU parity cases with MODE strings of about 100,000 characters, below Linux's 128 KiB limit for one argument: the two cases from the review, long runs of permission letters, actions and clauses, a digit run after an operator, and two digit runs above 07777 that must be rejected. Boundaries: bench/utils/mkfifo/Tests.py only. Validation: make -C bench/utils/mkfifo test -> 153 passed. Limitations: none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
리뷰 감사합니다. 반영했습니다.
검증·테스트·퍼징 결과는 PR 본문에 갱신했습니다. |
duncan020313
pushed a commit
that referenced
this pull request
Oct 30, 2026
Problem: Review of #42: the MODE parser and Meaning recursed once per character, so a long MODE overflowed the stack. A MODE of 50,000 characters (`0...0600` or `u...u+r`) crashed with "Stack overflow", and 20,000 characters took 12 s because every step copied the rest of the string. Approach: The recursive functions stay as the definitions the proofs use. ParseMode gets a `by method` body that walks indices of the string, and DecideMode calls ComputeMeaning instead of Spec.Meaning; each loop is proved to return the value of the function it replaces. Octal digits stop at the first value above 07777, as mode_compile does. Small helper methods hide the other spec definitions so that each step verifies within the time limit. mkfifo.md describes this and the new base revision. Boundaries: MkfifoModeCore.dfy, one call in MkfifoCore.dfy and mkfifo.md. The specification and the proofs are unchanged. Validation: make -C bench/utils/mkfifo verify -> 112 declarations, 1,542 conditions, 0 failed; make check TASK=mkfifo passes. A MODE of 100,000 characters now finishes in 0.3 s. Three incorrect variants of the new loops (trailing comma accepted, 07777 limit off by one, last clause skipped) each fail verification. Limitations: The recursive functions are still compiled, but no run-time path calls them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change and scope
Adds
mkfifo(issue #14): one FIFO per operand in command-line order, with-m/--mode(octal and symbolic MODE),--help,--versionand GNU parsing diagnostics. A failed operand does not stop later operands.MODE is specified as abstract syntax with
RenderandMeaning; the specification contains no parsing algorithm. The Core parser is proved to invertRender, which makes theinvalid modedecision exact and shows that a valid MODE has one meaning. With-m, each successful creation is followed by one non-followingSetFileModecall, which is GNU'slchmodaftermkfifo. (mkdir leaves out-mbecause chmod after creation adds a failure point that GNU does not have. For mkfifo that second call is GNU's own behavior.)Example (umask 022):
mkfifo -m =rw,+x pipecreatespipewith mode0755;mkfifo -m +644,u+x pipegives0766;mkfifo -m u+s pipereportsmkfifo: mode must specify only file permission bitsand creates nothing.mkfifo, coreutils.62c4c9ffef2464255c6234e76005615745ebbdde(branch head). All results below come from one run on basea832a0cwith the changes of the last two commits applied but not yet committed;git diff a832a0c 62c4c9fis byte-identical to the tested diff (sha2566c66e4e339de2074444f56a2bda299690743364821f6588317bb2c35aacd01bd)./workspace/dafnyutilsin the contributor container (Ubuntu 24.04, Linux x86-64); Dafny 4.11.1+10dca3ba4a2909c11299176440401fc31413b53d, .NET SDK 8.0.132, Python 3.12.3, pytest 9.1.1, Ruff 0.15.12, Rust/Cargo 1.94.0, Z3 4.16.0.bench/utils/mkfifo/tools/coreutils_fuzzer/src/fuzz/input/generators/mkfifo.rs, registered ingenerators/mod.rsandutils/capabilities.rsfuzz/comparison/fs_snapshot.rsnow records a FIFO node from its metadata (kindfifo, no contents) instead of failing with "unsupported filesystem node". It never opens the FIFO. Character devices are still rejected. A regression test is added insrc/tests/fuzzer.rs. Please review this part separately.src/mkfifo.cat2cf491412c199e2211880ec3f4ba387026638a33, gnuliblib/modechange.cat83730551b451b43577531986a5c4300e0166c42f, https://www.gnu.org/software/coreutils/, GPL-3.0-or-later; portedtests/misc/mknod.sh(mkfifo -m 734,mkfifo --mode='ug+rw,o+r'underumask 777).bench/utils/mkfifo/mkfifo.md; base revision3c704ee78884a4e764354431bb72b27771c02980. Trusted calls:CreateSpecialNode,SetFileMode,GetUmask, quoting, errno text and stream writes.Changes after review
Review comment of 2026-10-27: a long MODE could overflow the stack in the parser, and
0…0600andu…u+rshould become test cases.Meaningrecursed once per character. A 50,000-character MODE crashed withStack overflow, and 20,000 characters took 12 s because every step copied the rest of the string.ParseMode(itsby methodbody) andComputeMeaningwalk string indices, and each loop is proved to return the value of the function it replaces, so the specification and the existing proofs are unchanged. Octal digits stop at the first value above 07777, asmode_compiledoes. The compiled C# calls none of the recursive functions.-mand--mode=) match GNU in output, exit status and resulting mode; the fuzzer with five more seeds (2, 3, 5, 11, 13; 1,000 iterations each) found no mismatch; and the changed modules (MkfifoModeCore.dfy,MkfifoCore.dfy) verify with five other Z3 seeds (1, 7, 100, 2026, 12345) under both the repository's isolated-assertion settings and the per-method settings ofmake check, with no failure or timeout.mainat3c704ee(tsort). The only conflict was the utility list inutils/capabilities.rs, which now contains both.Soundness and completeness
Following the review guidance on #38 (soundness: the specification accepts a correct execution trace; completeness: it rejects an incorrect one):
Soundness. The verified Core implementation matches the pinned GNU build on 153 runtime cases and on 3,000 generated fuzzing cases (sections 1 and 2), so GNU's observed traces satisfy the specification on all of them.
Completeness. For each operand the specification fixes the exact native requests: one
CreateSpecialNodewith0666(no-m) or the MODE bits (-m), and, only after a successful creation with-m, one non-followingSetFileMode. It also fixes operand order, every diagnostic byte and its order, the exit status, and the MODE decision before any operand. It states what does not change: the filesystem after a rejected MODE, the filesystem between operands, and standard output. Help, version, parse errors and a missing operand are early exits of the shared runner, which never callRunCore.Mutation check. Each incorrect implementation below changes one place in the reference Core (the specification is unchanged); unchanged controls verify. Every mutant fails verification at a specification obligation. This shows that the reference proof cannot be completed for these behaviors; it is not a separate proof that no other proof exists.
-mOperandStep(no-m: one creation only)0666, then chmod to MODEOperandStep(creation mode is the MODE bits)OperandStep(SetFileMode(path, false, …))OperandStep(failure ends the operand)OperandStep(one creation request)OutcomeDiagnosticOperandResultsOperandResultsPlanRunexit statusOrderedDiagnosticsModePlanOf(Meaning)ModePlanOf(OnlyPermissionBits)PlanRun(filesystem unchanged)Spec(io.stdout() == old(io.stdout()))u+r,ModeInvalid(parser round trip)u+r,ParseModeby method(equals the parser function)010000OctalUpTo(equalsOctalValue(ds) <= 07777)ComputeMeaning(equalsMeaning)Left open on purpose. Only the native observations: the resulting FIFO's mode, ownership and timestamps come from the trusted
CreateSpecialNodeandSetFileModeresults, as formkdir. A failed permission change after a successful creation is specified and proved but cannot be produced by the fixtures.Issues raised on #39 and #40 are covered:
--=lists GNU's possibilities,-éreports only its first byte, both diagnostics use GNU'squoteaf, the scope document starts with the manual's description, and the branch has no merge commit.AI usage
I investigated how GNU mkfifo works and when it fails (permission bits, umask, symbolic MODE), decided the scope (
-mincluded,-Z/--contextleft out) and how MODE is specified, then directed the implementation. Claude assisted with writing and reviewing the specification, code, proofs, tests and fuzzer generator based on my instructions. I reviewed the specification, the scope document and the shared fuzzer change against GNUsrc/mkfifo.c.Setup note for maintainers
On a host whose user ID is not 1000,
.devcontainer/post-create.shfails withmkdir: cannot create directory '/workspace/tmp/update-z3-rebuild': Permission denied: the image creates/workspace/tmpfor UID 1000, andupdateRemoteUserUIDthen changesvscodeto the host UID. Runningchown vscode:vscode /workspace/tmpas root inside the container and rerunning the script worked. This PR does not change the setup files.Options left out due to IO.dfy
-Z,--context[=CTX]selabel_open/defaultcon,setfscreatecon,smack_set_label_for_self). Same decision asmkdir.1. Test results
make -C bench/utils/mkfifo testdafny_verifycases are deselected here and run bymake check, below).2. Fuzzing results
python3 tools/coreutils_fuzzer/run.py fuzz mkfifo --seeds 1,7,19 --iterations 10001,7,19; requested iterations per seed:1000._build/coreutils/src/mkfifo(native, sha2562eda4b35ab2839d68f98803caf2263376929c0e3cb828fe6c8bc821693c94be1),_build/bench/mkfifo_bench.dll(dotnet DLL, sha256958d7e867d9b95c9638d9c232654e644bd692bf8c2d1afb044cd17c53010b751).62c4c9f;dafnyutils-coreutils-fuzzer:latestsha256:c312109aeaec50957a8240aa7c52d6bf8b564f18d926afdcf6a672f552f2c6d5.fs:file-addedcounts only regular files (node_is_fileinfuzz/semantic.rs), so a run that creates a FIFO falls in no filesystem bucket. The other missing buckets are changes mkfifo never makes: it does not modify, remove or retime existing entries (an existing path fails withFile exists), and it never reads standard input. The filesystem comparison itself does include the new FIFO nodes and their modes. This PR leavessemantic.rsunchanged.Full standard output:
3. Verification results
make -C bench/utils/mkfifo verify, 0Mkfifo.dfy,MkfifoCore.dfy,MkfifoProof.dfyand the files they include (MkfifoSpec.dfy,MkfifoSchema.dfy,MkfifoModeCore.dfy,MkfifoModeProof.dfy,CliSupport.dfy); repository limits: isolated assertions, 30-second condition limit, 100-second declaration limit. Termination of MODE parsing and the operand loop is proved; CLI termination keeps the shared runner's boundary.python3 -m benchmarks validate mkfifo→mkfifo: valid(exit 0);make check TASK=mkfifo→mkfifo: checks passed(exit 0; it rebuilds, reruns the 153 runtime cases and the verification, and runs the 6dafny_verifycases).make -C bench/utils/mkfifo verify(first and last lines; one block per declaration in between):python3 -m benchmarks validate mkfifo:make check TASK=mkfifo(last lines):🤖 Generated with Claude Code