Skip to content

[common] Run the z-order sign flip at the width of the bit pattern - #9628

Open
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:fix/zorder-float-double-shift
Open

[common] Run the z-order sign flip at the width of the bit pattern#9628
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:fix/zorder-float-double-shift

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

Purpose

close #9627

ZOrderByteUtils.doubleToOrderedBytes built its sign-magnitude mask from a 31-bit shift of a 64-bit value:

long lval = Double.doubleToLongBits(val);
lval ^= ((lval >> (Integer.SIZE - 1)) | Long.MIN_VALUE);

The shift is meant to be a sign extension, all ones for a negative pattern and all zeros for a positive one. At 31 it instead copies bits 31 through 61 into the low half of the mask, so any value bit lining up with a set bit there gets inverted. Negatives lose most of their ordering: -2.5 encodes to 0x3ffbffff80080000 while the strictly smaller -2.5000000000000004 encodes to 0x3ffbffff80080001. Positives are affected wherever two values differ at one of those positions: 2.5 encodes to 0xc004000080080000 and the larger 2.5000000002328306 to 0xc004000080000000. floatToOrderedBytes widens to double and ran the same line, so both types were wrong.

Both now shift by Long.SIZE - 1. The same transform is written correctly in SortUtil, where putFloatNormalizedKey shifts an int by Integer.SIZE - 1 and putDoubleNormalizedKey shifts a long by Long.SIZE - 1; the z-order copy had kept the int-sized constant on a long.

The float path keeps widening to double instead of encoding the 32-bit pattern into half the buffer. Widening is exact and order-preserving, and it keeps all eight bytes carrying information, which matters because interleaveBits consumes bytes most-significant first and Spark truncates the interleaved output to a maximum size.

Nothing on disk changes, since a z-value only exists while sorting.

Tests

TestZOrderByteUtil.testFloatDoubleNegativeOrdering walks an ascending array from -MAX_VALUE to MAX_VALUE for both types and asserts the encodings ascend with it. It then walks a thousand adjacent bit patterns descending from -3.0f and from just below -2.5d, asserting each encoding is strictly lower than the one before, which is where the old mask fails: the arrays on their own are too sparse to catch it.

The existing testFloatOrdering and testDoubleOrdering only used random.nextFloat() and random.nextDouble(), both in [0, 1), which is why this survived.

mvn -pl paimon-common -Dtest=TestZOrderByteUtil test on JDK 8: 15 tests, 0 failures. spotless:check and checkstyle:check on paimon-common are clean.

doubleToOrderedBytes built its sign-magnitude mask with
(lval >> (Integer.SIZE - 1)), a 31-bit shift of a 64-bit value, so for
a negative double only part of the magnitude was inverted and adjacent
negatives compared out of order. floatToOrderedBytes widens to double
and ran the same line.

Shift by Long.SIZE - 1 in both. SortUtil already does this correctly
for its own float and double normalized keys.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Z-order encoding of negative FLOAT and DOUBLE values is not order-preserving

1 participant