You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Steve Carlin (Code Review)" <ge...@cloudera.org> on 2025/11/11 18:28:35 UTC
[Impala-ASF-CR] IMPALA-14095: Calcite Planner: Use pruned row counts in stats
Hello Aman Sinha, Fang-Yu Rao, Riza Suminto, Joe McDonnell, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23337
to look at the new patch set (#9).
Change subject: IMPALA-14095: Calcite Planner: Use pruned row counts in stats
......................................................................
IMPALA-14095: Calcite Planner: Use pruned row counts in stats
This commit uses the more accurate pruned row counts when collecting
filter cardinalities.
The pruned row counts are cached in the CalciteTable.
The ImpalaRelMdRowCount checks to see if the underlying RelNode
has a Table attached to it. The condition is broken up into
portions used by the HdfsPartitionPruner for pruning and those
conditions which are not used for pruning. The conditions not
used in the pruning still need to be calculated in the filter
selectivity estimates.
Change-Id: I442667a338d30d3daa66e223154bc9a1835b2880
---
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaHdfsScanRel.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/ExprConjunctsConverter.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/PrunedPartitionHelper.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaRelMdRowCount.java
M java/calcite-planner/src/test/java/org/apache/impala/planner/TestCalciteStats.java
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q01.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q02.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q03.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q04.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q05.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q06.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q07.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q08.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q09.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q10a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q11.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q12.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q13.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q14b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q15.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q16.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q17.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q18.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q19.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q20.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q21.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q22.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q23a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q23b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q24a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q24b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q25.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q26.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q27.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q29.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q30.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q31.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q32.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q33.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q34.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q35a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q36.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q37.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q38.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q39a.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q39b.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q40.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q41.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q42.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q43-verbose.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q43.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q44.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q45.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q46.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q47.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q48.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q49.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q50.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q51.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q52.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q53.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q54.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q55.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q56.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q57.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q58.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q59.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q60.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q61.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q62.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q63.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q64.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q65.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q66.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q67.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q68.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q69.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q70.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q71.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q72.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q73.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q74.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q75.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q76.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q77.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q78.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q79.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q80.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q81.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q82.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q83.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q84.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q85.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q86.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q87.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q88.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q89.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q90.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q91.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q92.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q93.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q94.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q95.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q96.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q97.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q98.test
M testdata/workloads/functional-planner/queries/PlannerTest/calcite_tpcds/tpcds-q99.test
110 files changed, 24,200 insertions(+), 22,845 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/23337/9
--
To view, visit http://gerrit.cloudera.org:8080/23337
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I442667a338d30d3daa66e223154bc9a1835b2880
Gerrit-Change-Number: 23337
Gerrit-PatchSet: 9
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>