You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/07/02 21:19:00 UTC
[jira] [Commented] (IMPALA-9911) IS [NOT] NULL predicate
selectivity estimate is wrong if #nulls is 0
[ https://issues.apache.org/jira/browse/IMPALA-9911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150589#comment-17150589 ]
ASF subversion and git services commented on IMPALA-9911:
---------------------------------------------------------
Commit 08a7569d4c372d6fb5003609c31c87a55db4e71b in impala's branch refs/heads/master from Aman Sinha
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=08a7569 ]
IMPALA-9911: Fix IS [NOT] NULL predicate selectivity
When null count is 0, the IsNullPredicate's selectivity
was not being computed since it did not distinguish
between a -1 (no stats) vs a 0 null count. This caused
a default selectivity estimate being applied. This
patch fixes it by explicitly checking whether nulls
count stat is present and if so, use it regardless of
whether it is 0 or more.
Testing:
- Added cardinality tests for IS NULL and IS NOT NULL.
- Ran PlannerTest and updated baseline plans.
- Updated expected selectivity for null predicate tests
in ExprCardinalityTest.
- Ran precommit tests through gerrit-verify-dryrun
Change-Id: I46c084be780b8f5aead9e2b9656fbab6cc8c8874
Reviewed-on: http://gerrit.cloudera.org:8080/16131
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> IS [NOT] NULL predicate selectivity estimate is wrong if #nulls is 0
> --------------------------------------------------------------------
>
> Key: IMPALA-9911
> URL: https://issues.apache.org/jira/browse/IMPALA-9911
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Aman Sinha
> Assignee: Aman Sinha
> Priority: Major
>
> Consider the tpcds customer table .. its c_current_addr_sk column has #Nulls = 0 as shown below.
> {noformat}
> tpcds> show column stats customer;
> +------------------------+--------+------------------+--------+----------+-------------------+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +------------------------+--------+------------------+--------+----------+-------------------+
> ....
> | c_current_cdemo_sk | INT | 91558 | 3438 | 4 | 4 |
> | c_current_hdemo_sk | INT | 7376 | 3431 | 4 | 4 |
> | c_current_addr_sk | INT | 42003 | 0 | 4 | 4 |
> ....
> {noformat}
> The cardinality estimate for the following predicates shows a default selectivity of 10% being applied which is not correct:
> {noformat}
> explain select c_current_addr_sk from customer where c_current_addr_sk is not null;
> | 00:SCAN HDFS [tpcds.customer] |
> | HDFS partitions=1/1 files=1 size=12.60MB |
> | predicates: c_current_addr_sk IS NOT NULL |
> | row-size=4B cardinality=10.00K |
> +------------------------------------------------------------+
> explain select c_current_addr_sk from customer where c_current_addr_sk is null;
> | 00:SCAN HDFS [tpcds.customer] |
> | HDFS partitions=1/1 files=1 size=12.60MB |
> | predicates: c_current_addr_sk IS NULL |
> | row-size=4B cardinality=10.00K |
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org