You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2024/02/15 02:08:00 UTC
[jira] [Commented] (IMPALA-12790) ScanNode.getInputCardinality can overestimate if LIMIT is large.
[ https://issues.apache.org/jira/browse/IMPALA-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817570#comment-17817570 ]
ASF subversion and git services commented on IMPALA-12790:
----------------------------------------------------------
Commit 1dbde2c197a26971aabed7d6bb387aac7c64d83f in impala's branch refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1dbde2c19 ]
IMPALA-12790: Fix overestimation in ScanNode.getInputCardinality
ScanNode.getInputCardinality() can overestimate if LIMIT is large. If
table cardinality stats exist and it is less than the LIMIT, then the
table cardinality should be returned.
This patch adds those fixes and more test cases under
small-query-opt.test. This also moves tests added by IMPALA-5602 to
small-query-opt.test since only
PlannerTest.testSmallQueryOptimization runs with
EXEC_SINGLE_NODE_ROWS_THRESHOLD > 0.
Testing:
- Pass FE tests.
Change-Id: Icc5b39a7684fb8748185349d0b80baf8dcd6b126
Reviewed-on: http://gerrit.cloudera.org:8080/21005
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> ScanNode.getInputCardinality can overestimate if LIMIT is large.
> ----------------------------------------------------------------
>
> Key: IMPALA-12790
> URL: https://issues.apache.org/jira/browse/IMPALA-12790
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Minor
>
> The bug is first found in [https://gerrit.cloudera.org/c/20993/1/fe/src/main/java/org/apache/impala/planner/ScanNode.java#338]
> Simple scan query can have ScanNode.getInputCardinality() return larger number than it should be if the query has LIMIT larger than table cardinality. This bug is visible in following test query with low EXEC_SINGLE_NODE_ROWS_THRESHOLD option set:
> {code:java}
> Section DISTRIBUTEDPLAN of query at line 451:
> select * from functional_kudu.tinytable limit 1000;
> Actual does not match expected result:
> PLAN-ROOT SINK
> |
> 01:EXCHANGE [UNPARTITIONED]
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
> | limit: 1000
> |
> 00:SCAN KUDU [functional_kudu.tinytable]
> limit: 1000
> row-size=43B cardinality=3
> Expected:
> PLAN-ROOT SINK
> |
> 00:SCAN KUDU [functional_kudu.tinytable]
> limit: 1000
> row-size=43B cardinality=3 {code}
> The distributed plan should not have EXCHANGE added since it is a small query (cardinality=3) and can run in coordinator only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org