Repository navigation
Round5 fixes - #35
Merged
Merged
Round5 fixes#35
Conversation
The temporary output file added in 5158400 was created 0600 and renamed over the output, so every output became 0600, a symlinked --output was replaced by a regular file, and an interrupted run left the temporary file behind for later runs to hash. The temporary file is now created next to the file a symlinked output points to, with the permissions of the file it replaces or the umask default, and removed on SIGINT or SIGTERM. Files to skip are also matched by identity, so the output or audit file reached through a symlinked path is not hashed into the results. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Relative audit paths also resolve against the working directory, so auditing by absolute path matches as auditing by relative path does - The audit file reached through a symlinked path is still recognised - A path listed twice in an audit file is used once with a warning, rather than reported missing although the file exists - Unreadable new files and files in unreadable directories are listed as could not be read - Paths supplied explicitly, as arguments or in --input, which are broken symlinks or special files are errors, while found when walking they are skipped with a warning - --exclude-dir matches the end of the path for symlinked directories - A SQLite output name containing ? or # is written correctly - An empty --hash, and a failed write of --hashes, are errors - A SQLite file which is not a hashit database says so - rhash ED2K BSD style lines are accepted - Flag errors are printed in the same ERROR format - Help text for --output and --hash Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Corrects the claim that --threads 1 gives a stable order, and documents output permissions, symlinks and hard links, ignore rules inside symlinked directories, explicit paths, unreadable files, concatenated hashdeep files, sum manifests written by redirect, hashit sum files using other hashes, and the ed2k variant. Adds tests for each fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Regressions from the previous commits on this branch: - A record resolved through the audit file's directory could take the path of a different record. Each record's own resolution now takes priority, and the working directory is no longer a record base; instead absolute paths on disk are also tried relative to it - --overwrite failed on a read only output, as its mode was applied to the temporary file before writing. It is now applied before renaming - A hard link to the old output file in the input was silently skipped. The old output is now only skipped by path - SIGHUP and SIGQUIT now also remove the temporary output - A symlink loop as --output is refused rather than replaced - SQLite -wal and -shm files reached through a symlinked path are skipped by their real path - The temporary output name is guarded by a lock for the signal handler - A dangling symlink as --output says what it is Older bugs: - Names starting with two dots, such as ..notes, were treated as being outside the directory, breaking SQLite audits from elsewhere - Two files on disk matching the same record were both counted. The file matching the record's own path claims it, then the first in sort order, and the other is new - A sum file listing a path twice kept the last hash silently, and the same file listed by relative and absolute path counted twice - SQLite file URIs on Windows were built as file://C:/ which reads the drive as a host Also warns when --format is ignored by an audit, or an output or audit file supplied as an input is skipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Writing output to a file such as notes deleted unrelated files named notes-journal, notes-wal or notes-shm, and left them out of the results, for every format. The clean up of a previous database's sidecar files was applied regardless of format since 5158400. Sidecar names are now only skipped for SQLite output and audit databases, and only removed when replacing an existing SQLite database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Audit matching: - Paths are only compared as absolute paths. Records and files on disk were also matched by their bare relative path, which only refers to the same file when run from the same directory, so auditing from a subdirectory compared files against the wrong records, and overlapping paths gave a different result between runs. Records are resolved against the invoked from directory, then the audit file's directory, then the working directory, in that priority, and SQLite audits look up the path relative to invoked_from - -f hashdeep with a sha1 or other audit file no longer replaces the audit's hashes with md5 and sha256 - The file reported as moved is the first in path order - Unreadable files are always listed, including when another file claims their record - SQLite databases record which hashes they hold rather than an audit guessing from the first row Output: - A symlinked --output is resolved by the operating system, as resolving it by hand replaced the wrong file for links through a symlinked directory. Dangling and looping links are refused - The output or audit file supplied as an input is an error, so the existing output is kept rather than replaced with empty results - A symlink to the existing output is not hashed, while hard links are - SQLite sidecar names are only skipped and removed for SQLite - A mode of 000 is kept, and SIGPIPE also removes the temporary file - Symlink loops found while walking are skipped with a warning - A single warning for duplicate paths, and none for -f text with -a Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # processor/processor.go # test-all.sh
…nd signals - SQLite audits match paths the same way as hashdeep files, resolving stored paths against invoked from, the database's directory and the working directory through a temporary index, so copied trees and ../ paths audit - the audit or output file matched by a wildcard is skipped with a warning, an existing output is only kept when nothing else was hashed - a new output through a symlinked directory is created where it resolves - the working directory is looked up once rather than for every path - lookup priority is indexed by directory so record order does not matter - the audit file through a symlink matches its own entry and uses its real directory as well as the link's - the output reached through a symlinked directory is not hashed - an audit file can be read from a pipe, a SQLite one is an error - signals are always handled, a broken pipe always exits 141 - remove the retired Go Report Card badge, document remaining edge cases Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
modernc.org/sqlite does not support illumos or Solaris, and github.com/gosuri/uilive uses syscall.SYS_IOCTL which they lack. Both are now only built on other platforms. On illumos and Solaris the sqlite format and auditing a SQLite database are errors, and --progress is ignored with a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Leave signals ignored at startup ignored, so nohup and trap '' work again - Audit when the working directory has been deleted - Look SQLite records up directly, indexing only unusual path spellings, restoring audit speed on large databases - Keep non UTF-8 names when a record is claimed by a better match - Resolve Windows drive relative paths such as D:a and -o C:out.txt - Do not replace an output file created while running without --overwrite - Discard a SQLite database which could not be written in full, and remove its temporary files - Correct README wording on signals and kept output files - Build for illumos in test-all.sh, and add round 9 checks - Use native separators in tests so they pass on Windows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pr-insights
Bot
added
M/size
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.