[jira] [Created] (IMPALA-10136) Cardinality estimates for aggregation operations don't consider conjuncts on grouping expressions correctly

["Shant Hovsepian (Jira)" <ji...@apache.org>]

Shant Hovsepian created IMPALA-10136:
----------------------------------------

             Summary: Cardinality estimates for aggregation operations don't consider conjuncts on grouping expressions correctly
                 Key: IMPALA-10136
                 URL: https://issues.apache.org/jira/browse/IMPALA-10136
             Project: IMPALA
          Issue Type: Bug
          Components: Frontend
    Affects Versions: Impala 3.4.0
            Reporter: Shant Hovsepian
            Assignee: Shant Hovsepian


ComputeStats() in the PlanNode calls estimateNumGroups() for the AggregationNode to calculate the cardinality of a grouping expression. Then in a later step applyConjunctsSelectivity() is called to adjust the cardinality based on the available conjuncts. However with aggregation operations certain conjuncts i.e. those from the HAVING clause or conjuncts on the grouping expressions affect the number of groups produced.

ndv(day) = 11 

count(alltypesagg) = 10280
{code:java}
Query: explain select day, count(*) from alltypesagg where day=2 group by 1
+------------------------------------------------------------+
| Explain String                                             |
+------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=4.06MB Threads=4 |
| Per-Host Resource Estimates: Memory=52MB                   |
| Codegen disabled by planner                                |
|                                                            |
| PLAN-ROOT SINK                                             |
| |                                                          |
| 04:EXCHANGE [UNPARTITIONED]                                |
| |                                                          |
| 03:AGGREGATE [FINALIZE]                                    |
| |  output: count:merge(*)                                  |
| |  group by: `day`                                         |
| |  row-size=12B cardinality=11                             |
| |                                                          |
| 02:EXCHANGE [HASH(`day`)]                                  |
| |                                                          |
| 01:AGGREGATE [STREAMING]                                   |
| |  output: count(*)                                        |
| |  group by: `day`                                         |
| |  row-size=12B cardinality=11                             |
| |                                                          |
| 00:SCAN HDFS [functional.alltypesagg]                      |
|    partition predicates: `day` = 2                         |
|    HDFS partitions=1/11 files=1 size=74.48KB               |
|    row-size=4B cardinality=1.00K                           |
+------------------------------------------------------------+
Fetched 24 row(s) in 0.02s
{code}
 

Given the predicate day=1 applies to the grouping expression the cardinality of the aggregation node should b 1 as opposed to 11.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org