You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Karuppayya (Jira)" <ji...@apache.org> on 2020/05/31 17:54:00 UTC

[jira] [Created] (SPARK-31877) Stats computed for HiveTable not used

Karuppayya created SPARK-31877:
----------------------------------

             Summary: Stats computed for HiveTable not used
                 Key: SPARK-31877
                 URL: https://issues.apache.org/jira/browse/SPARK-31877
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.0.0
            Reporter: Karuppayya


When fixing SPAK-31850, I made the following observation

As part of DetermineTableStats rule, we compute the stats for a HiveTableRelation.

In most case(the flag for converting Parquet and Orc table is enabled by default), RelationConversion rule converts the HiveTableRelation to LogicalRelation.

The stats computed as part of Hive Table relation does not get used.

The stats compute should be avoided if the Hive Tbale is going to be converted to data source table

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org