You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Danny Chen (Jira)" <ji...@apache.org> on 2023/10/25 01:38:00 UTC

[jira] [Closed] (HUDI-6959) Do not rollback current instant when bulk insert as row failed

     [ https://issues.apache.org/jira/browse/HUDI-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Danny Chen closed HUDI-6959.
----------------------------
    Resolution: Fixed

Fixed via master branch: 65dd645b487a61fbca7e4e4b849d1f2f1ec143f9

> Do not rollback current instant when bulk insert as row failed
> --------------------------------------------------------------
>
>                 Key: HUDI-6959
>                 URL: https://issues.apache.org/jira/browse/HUDI-6959
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: spark
>            Reporter: Qijun Fu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.0.0, 0.14.1
>
>
> When org.apache.hudi.spark3.internal.HoodieDataSourceInternalBatchWrite#abort is called, all the subtasks may not have already been canceled. So if we rollback current instant immediately, there may be new files been written after rollback scheduled, which will cause dirty data.
>  
> We should rollback the failed instant using common mechanism eager and lazy 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)