Search before asking
Paimon version
master, 475be566f (2.1-SNAPSHOT).
Compute Engine
Flink. SupportsRowLevelOperationFlinkTableSink is the only production user of this visitor.
Minimal reproduce step
This one is a contract problem rather than something I could trigger from SQL, and I want to be upfront about that.
OnlyPartitionKeyEqualVisitor decides whether a DELETE can be executed by dropping partitions. It collects the equality literals into a map, one entry per partition key:
public Boolean visitEqual(FieldRef fieldRef, Object literal) {
boolean contains = partitionKeys.contains(fieldRef.name());
if (contains) {
partitions.put(fieldRef.name(), literal.toString());
return true;
}
return false;
}
Two equality predicates on the same key therefore overwrite each other, and the conjunction still reports "droppable". Feeding it pt = 'a' AND pt = 'b':
OnlyPartitionKeyEqualVisitor visitor = new OnlyPartitionKeyEqualVisitor(Arrays.asList("pt", "dt"));
Predicate contradiction = PredicateBuilder.and(equal(pt, "a"), equal(pt, "b"));
contradiction.visit(visitor); // true
visitor.partitions(); // {pt=b}
The Flink sink reads that as a partition drop of pt = 'b' and deletes every row in that partition, while the predicate itself matches nothing.
What I could not do is get such a predicate to the sink from SQL. DELETE FROM t WHERE pt = 'a' AND pt = 'b' is folded to FALSE by Calcite's simplification before applyDeleteFilters runs, and PredicateBuilder.and does not merge equality predicates itself, so the visitor only sees two Equal leaves if something hands them over unfolded. So today the guarantee that this cannot happen lives in the planner, not in Paimon.
What doesn't meet your expectations?
The visitor reduces a conjunction of constraints to a last-write-wins map, which loses the information that the constraints conflict. For a decision whose consequence is dropping a whole partition, that seems worth deciding inside Paimon rather than relying on an external optimizer to never pass a contradiction through. A future engine version, another engine binding, or a caller that builds the predicate directly would not have that protection.
Anything else?
The same class already returns false for everything it does not understand (visitIsNull, visitIn, ranges, OR), so a conservative answer for a contradiction fits how it is written.
Are you willing to submit a PR?
Search before asking
Paimon version
master,
475be566f(2.1-SNAPSHOT).Compute Engine
Flink.
SupportsRowLevelOperationFlinkTableSinkis the only production user of this visitor.Minimal reproduce step
This one is a contract problem rather than something I could trigger from SQL, and I want to be upfront about that.
OnlyPartitionKeyEqualVisitordecides whether a DELETE can be executed by dropping partitions. It collects the equality literals into a map, one entry per partition key:Two equality predicates on the same key therefore overwrite each other, and the conjunction still reports "droppable". Feeding it
pt = 'a' AND pt = 'b':The Flink sink reads that as a partition drop of
pt = 'b'and deletes every row in that partition, while the predicate itself matches nothing.What I could not do is get such a predicate to the sink from SQL.
DELETE FROM t WHERE pt = 'a' AND pt = 'b'is folded to FALSE by Calcite's simplification beforeapplyDeleteFiltersruns, andPredicateBuilder.anddoes not merge equality predicates itself, so the visitor only sees twoEqualleaves if something hands them over unfolded. So today the guarantee that this cannot happen lives in the planner, not in Paimon.What doesn't meet your expectations?
The visitor reduces a conjunction of constraints to a last-write-wins map, which loses the information that the constraints conflict. For a decision whose consequence is dropping a whole partition, that seems worth deciding inside Paimon rather than relying on an external optimizer to never pass a contradiction through. A future engine version, another engine binding, or a caller that builds the predicate directly would not have that protection.
Anything else?
The same class already returns false for everything it does not understand (
visitIsNull,visitIn, ranges, OR), so a conservative answer for a contradiction fits how it is written.Are you willing to submit a PR?