[common] Close the file-index stream when reader construction fails - #9636
Open
LuciferYang wants to merge 1 commit into
Open
[common] Close the file-index stream when reader construction fails#9636LuciferYang wants to merge 1 commit into
LuciferYang wants to merge 1 commit into
Conversation
FileIndexFormat.Reader's constructor closed the stream only on IOException, while an unrecognized header throws RuntimeException. A throwing constructor never assigns the caller's try-with-resources resource, so FileIndexProcessor and FileIndexesTable leaked the stream they had just opened. Catch RuntimeException there too, which is where the stream reference lives, and drop the call-site workaround apache#9462 added to FileIndexPredicate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
close #9635
FileIndexFormat.Reader's constructor closed the stream it was handed only for anIOException, while a header it does not recognize is reported withthrow new RuntimeException("This file is not file index file."). Every call site opens the stream inside the try-with-resources head:A constructor that throws never assigns that resource, so the stream stays open. #9462 fixed this for
FileIndexPredicateby catchingRuntimeExceptionat the call site;FileIndexProcessorandFileIndexesTablestill leak, one per failed read, andFileIndexProcessorruns per data file during compaction andrewrite-file-index.The
catchin the constructor now coversRuntimeExceptiontoo, which is where the stream reference lives, so all three call sites are covered and the next one does not have to remember. The call-site workaround inFileIndexPredicategoes away with it.One behavior change worth knowing about: a bad magic or version now arrives wrapped,
RuntimeException("Exception happens while construct file index reader.")with the original as its cause, rather than bare.FileIndexesTable.fileIndexReadExceptiontestsgetCause() instanceof IOExceptionand is unaffected; the only place that asserted the old message wasFileIndexPredicateCloseTest, updated here.Tests
FileIndexFormatFormatTest.testCreateReaderClosesStreamOnBadMagichandscreateReadera stream over bytes that are not an index file, withclose()instrumented, and asserts the stream was closed before it asserts anything about the exception. Ordering it that way matters: the close is what the fix is about, and asserting the message first would let a change that only wraps the exception pass.FileIndexPredicateCloseTestkeeps covering the same thing one layer up, with its message expectation updated.Against the unfixed constructor the new test fails on the close assertion.
mvn -pl paimon-common -Dtest=FileIndexFormatFormatTest,FileIndexPredicateCloseTest teston JDK 8: 5 tests, 0 failures.spotless:checkandcheckstyle:checkon paimon-common are clean.