[common] Fix JSON round-trip of temporal and decimal predicate literals - #9640
Open
LuciferYang wants to merge 1 commit into
Open
[common] Fix JSON round-trip of temporal and decimal predicate literals#9640LuciferYang wants to merge 1 commit into
LuciferYang wants to merge 1 commit into
Conversation
Jackson's JavaTimeModule wrote LocalDate/LocalTime/LocalDateTime/Instant literals as arrays and BigDecimal as a JSON number, and PredicateBuilder.convertJavaObject rejects the ArrayList and Double it gets back, so DATE, TIME, TIMESTAMP, TIMESTAMP_LTZ and DECIMAL literals could not survive the predicate JSON that the REST catalog sends and reads. Write them as ISO-8601 / plain strings and parse them back.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
close #9639
LeafPredicatehanded its literals to Jackson as whateverPredicateBuilder.convertToJavaObjectproduced.JsonSerdeUtilregistersJavaTimeModulewithout disablingWRITE_DATES_AS_TIMESTAMPS, so aLocalDatewas written as[2026,1,15]and aBigDecimalas a JSON number; reading those back yields anArrayListand aDouble, andPredicateBuilder.convertJavaObjectrejects both. DATE, TIME, TIMESTAMP, TIMESTAMP_LTZ and DECIMAL literals therefore could not round-trip at all: the writer's output is exactly what the reader refuses.Those five now go out as strings, ISO-8601 for the temporal types and
toPlainStringfor decimals, and come back through the matching parse beforeconvertJavaObjectsees them. Everything else is untouched.This changes the wire representation of those literals. Nothing that worked before breaks, because reading them back never worked; but a non-Paimon REST server that parses the filter JSON itself would see a string where it previously saw an array or a number, so it is worth calling out in release notes.
The reader deliberately does not also accept the old array and number forms. I had that in an earlier version and dropped it: those shapes have never been readable, so there is no stored or in-flight predicate in that form that used to work, and reconstructing them means guessing at things like which trailing fields Jackson omitted and how much precision a double kept. If a concrete compatibility case turns up, that is worth adding on its own terms.
Tests
PredicateJsonSerdeTest.testTemporalAndDecimalLiteralsRoundTripbuilds a predicate over a row with all five affected types, including a DECIMAL(20,3) with more significant digits than a double can hold, serializes it and asserts the parsed predicate equals the original.Against the unfixed code it fails with
UnsupportedOperationException: Unexpected date literal of class java.util.ArrayList.mvn -pl paimon-common -Dtest=PredicateJsonSerdeTest teston JDK 8: 91 tests, 0 failures.spotless:checkandcheckstyle:checkon paimon-common are clean.