You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Aniket Namadeo Mokashi (Jira)" <ji...@apache.org> on 2020/05/27 02:45:00 UTC

[jira] [Created] (SPARK-31829) Check for partition existence for Insert overwrite if not exists queries on Hive Serde Table

Aniket Namadeo Mokashi created SPARK-31829:
----------------------------------------------

             Summary: Check for partition existence for Insert overwrite if not exists queries on Hive Serde Table
                 Key: SPARK-31829
                 URL: https://issues.apache.org/jira/browse/SPARK-31829
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.4.5, 2.3.4, 2.2.3, 2.1.3
            Reporter: Aniket Namadeo Mokashi


If T is a Hive table, Query: INSERT OVERWRITE table T partition(p='existing') IF NOT EXISTS select ... ; executes job/computation on Spark and then avoids loading partitions. It should avoid doing the wasteful computation and exit early.

For Datasource table, it does avoid the computation and exits early (due to work done in SPARK-20831).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org