You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yuzhaojing (Jira)" <ji...@apache.org> on 2021/05/25 07:16:00 UTC
[jira] [Updated] (HUDI-1923) Add state in StreamWriteFunction to
restore
[ https://issues.apache.org/jira/browse/HUDI-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
yuzhaojing updated HUDI-1923:
-----------------------------
Description:
In flink, notifyCheckpointComplete not in checkpoint life cycle. If a checkpoint is success and commit action in notifyCheckpointComplete is failed, when we restore from the latest checkpoint, the element belong this instant will be discard.
So, we should store commit state and restore it when flink restart.
was:if coordinator notifyCheckpointComplete funtion execute failed, when we restore from the latest checkpoint, the element belong this instant will be discard.
> Add state in StreamWriteFunction to restore
> -------------------------------------------
>
> Key: HUDI-1923
> URL: https://issues.apache.org/jira/browse/HUDI-1923
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Flink Integration
> Reporter: yuzhaojing
> Assignee: yuzhaojing
> Priority: Major
>
> In flink, notifyCheckpointComplete not in checkpoint life cycle. If a checkpoint is success and commit action in notifyCheckpointComplete is failed, when we restore from the latest checkpoint, the element belong this instant will be discard.
> So, we should store commit state and restore it when flink restart.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)