[Impala-ASF-CR] IMPALA-14716: Calcite Planner: Make condition estimates more similar to original planner

["Aman Sinha (Code Review)" <ge...@cloudera.org>]

Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/23930 )

Change subject: IMPALA-14716: Calcite Planner: Make condition estimates more similar to original planner
......................................................................


Patch Set 13:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23930/13/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java
File java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java:

http://gerrit.cloudera.org:8080/#/c/23930/13/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java@255
PS13, Line 255:         return Expr.DEFAULT_SELECTIVITY;
I didn't notice this closely in the first pass but here it is returning the DEFAULT_SELECTIVITY for the whole expression even if a single operand (out of many) get a null from the estimatorSelectivityInternal().  This would negate all the other valid selectivities for other operands.  If I compare with how the conjuncts are handled, there the DEFAULT_SELECTIVITY is correctly applied only for the missing estimates.


http://gerrit.cloudera.org:8080/#/c/23930/13/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java@257
PS13, Line 257:       selectivity = selectivity + tmpSelectivity - selectivity * tmpSelectivity;
Taking a second look at this by going through an example:
WHERE qtr = 'Q1' OR qtr = 'Q2'.   Suppose we have 1 year's data, and assuming uniform distribution, each of these selectivity is  0.25.  So, ideally, I would expect 0.5 for this disjunctive predicate. 
However, here, it will produce 0.5 - 0.0025.  The formula does not seem right.  Is it brought over from Calcite ?



-- 
To view, visit http://gerrit.cloudera.org:8080/23930
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3b9a25259916504296dbd9a9cb9466be8fac8718
Gerrit-Change-Number: 23930
Gerrit-PatchSet: 13
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Thu, 02 Apr 2026 01:05:27 +0000
Gerrit-HasComments: Yes