You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2022/04/29 15:27:00 UTC

[jira] [Commented] (SPARK-39073) Keep rowCount after hive table partition pruning if table only have hive statistics

    [ https://issues.apache.org/jira/browse/SPARK-39073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17530054#comment-17530054 ] 

Apache Spark commented on SPARK-39073:
--------------------------------------

User 'qiuliang988' has created a pull request for this issue:
https://github.com/apache/spark/pull/36412

> Keep rowCount after hive table partition pruning if table only have hive statistics
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-39073
>                 URL: https://issues.apache.org/jira/browse/SPARK-39073
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.2.1
>            Reporter: qiuliang
>            Priority: Major
>
> If the partitioned table only has hive generated statistics, and the statistics are stored in partition properties. HiveTableRelation cannot obtain rowCount because the table statistics do not exist. We can generate rowCount based on the statistics of these pruned partitions



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org