You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2022/05/16 05:30:00 UTC

[jira] [Assigned] (SPARK-39073) Keep rowCount after hive table partition pruning if table only have hive statistics

     [ https://issues.apache.org/jira/browse/SPARK-39073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan reassigned SPARK-39073:
-----------------------------------

    Assignee: qiuliang

> Keep rowCount after hive table partition pruning if table only have hive statistics
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-39073
>                 URL: https://issues.apache.org/jira/browse/SPARK-39073
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.2.1
>            Reporter: qiuliang
>            Assignee: qiuliang
>            Priority: Major
>             Fix For: 3.4.0
>
>
> If the partitioned table only has hive generated statistics, and the statistics are stored in partition properties. HiveTableRelation cannot obtain rowCount because the table statistics do not exist. We can generate rowCount based on the statistics of these pruned partitions



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org