[GitHub] [spark] pengbo opened a new pull request #24286: [SPARK-27351][SQL] Wrong outputRows estimation after AggregateEstimation wit…

[GitBox <gi...@apache.org>]

pengbo opened a new pull request #24286: [SPARK-27351][SQL] Wrong outputRows estimation after AggregateEstimation wit…
URL: https://github.com/apache/spark/pull/24286
 
 
   ## What changes were proposed in this pull request?
   The upper bound of group-by columns row number is to multiply distinct counts of group-by columns. However, column with only null value will cause the output row number to be 0 which is incorrect.
   Ex:
   col1 (distinct: 2, rowCount 2)
   col2 (distinct: 0, rowCount 2)
   => group by col1, col2
   Actual: output rows: 0
   Expected: output rows: 2
   
   ## How was this patch tested?
   According unit test has been added, plus manual test has been done in our tpcds benchmark environement.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org