You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "lei w (Jira)" <ji...@apache.org> on 2023/01/17 10:40:00 UTC

[jira] [Updated] (HUDI-5565) Application restart may cause data lose when task parallelism is changed

     [ https://issues.apache.org/jira/browse/HUDI-5565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

lei w updated HUDI-5565:
------------------------
    Description: [HUDI-2084|https://github.com/apache/hudi/pull/3168] Resend the uncommitted write metadata when start up to avoid data lose. But when task parallelism is changed(increase  parallelism), data lose may happen too. Should we recommit instant when writeStatus is not empty in WriteMetadataEvent.  (was: [HUDI-2084|https://github.com/apache/hudi/pull/3168] Resend the uncommitted write metadata when start up avoid data lose. But when task parallelism is changed, data lose may happen too. Should we recommit instant when writeStatus is not empty in WriteMetadataEvent.)

> Application restart  may cause data lose when task parallelism is changed
> -------------------------------------------------------------------------
>
>                 Key: HUDI-5565
>                 URL: https://issues.apache.org/jira/browse/HUDI-5565
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: core
>            Reporter: lei w
>            Priority: Major
>
> [HUDI-2084|https://github.com/apache/hudi/pull/3168] Resend the uncommitted write metadata when start up to avoid data lose. But when task parallelism is changed(increase  parallelism), data lose may happen too. Should we recommit instant when writeStatus is not empty in WriteMetadataEvent.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)