You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2023/02/16 06:49:00 UTC

[jira] [Updated] (HUDI-5317) SparkSQL insert overwrite table doesn't work as expected for partitioned table

     [ https://issues.apache.org/jira/browse/HUDI-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo updated HUDI-5317:
----------------------------
    Fix Version/s: 0.13.0

> SparkSQL insert overwrite table doesn't work as expected for partitioned table
> ------------------------------------------------------------------------------
>
>                 Key: HUDI-5317
>                 URL: https://issues.apache.org/jira/browse/HUDI-5317
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: spark-sql
>            Reporter: Qijun Fu
>            Assignee: Qijun Fu
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 0.13.0
>
>
> For sql like insert overwrite table $table select xxx, we expect to drop all data in the table first and then insert the selected data into it.  But we found that the 'insert overwrite table' semantic works only for non-partitioned table. For partitioned table, current implementation will drop only partitions involved in the select sub-query, other partitions won't be dropped( which should be dropped as expected).  
> We will submit a pr to solve the problem that  insert overwrite table can drop all partitions at first then insert new data.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)