You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/05/08 10:09:00 UTC
[jira] [Updated] (FLINK-17568) Task may consume data after
checkpoint barrier before performing checkpoint for unaligned checkpoint
[ https://issues.apache.org/jira/browse/FLINK-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated FLINK-17568:
-----------------------------------
Labels: pull-request-available (was: )
> Task may consume data after checkpoint barrier before performing checkpoint for unaligned checkpoint
> ----------------------------------------------------------------------------------------------------
>
> Key: FLINK-17568
> URL: https://issues.apache.org/jira/browse/FLINK-17568
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
> Affects Versions: 1.11.0
> Reporter: Yingjie Cao
> Assignee: Yingjie Cao
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> For unaligned checkpoint, task may consume data after the checkpoint barrier before performing checkpoint which lead to consumption of duplicated data and corruption of data stream.
> More specifically, when the Netty thread notifies the checkpoint barrier for the first time and enqueue a checkpointing task in the mailbox, the task thread may still in data consumption loop and if it reads a new checkpoint barrier from another channel it will not return to the mailbox and instead it will continue to read data until a all data consumed or we have a full record, meanwhile, the data after checkpoint barrier may be read and consumed which lead to inconsistency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)