[jira] [Commented] (FLINK-19701) Unaligned Checkpoint might misuse the number of buffers to persist from the previous barrier

["Yun Gao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216439#comment-17216439 ] 

Yun Gao commented on FLINK-19701:
---------------------------------

[~AHeise] [~pnowojski] Could you have a look at whether this is truly a problem ? I test this case manually by blocking the barriers from the upstream tasks and it seems to be exist. 

> Unaligned Checkpoint might misuse the number of buffers to persist from the previous barrier
> --------------------------------------------------------------------------------------------
>
>                 Key: FLINK-19701
>                 URL: https://issues.apache.org/jira/browse/FLINK-19701
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Checkpointing
>    Affects Versions: 1.12.0
>            Reporter: Yun Gao
>            Priority: Major
>
> Current CheckpointUnaligner interacts with RemoteInputChannel to persisting the input buffers. However, based the current implementation it seems if we have the following case:
> {code:java}
> 1. There are 3 input channels.
> 2. Input channel 0 received barrier 1, and processed barrier 1 to start checkpoint 1.
> 3. Input channel 1 received barrier 1, and processed barrier 1. Now the state of input channel persister becomes BARRIER_RECEIVED and numBuffersOvertaken(channel 1) = n_1.
> 4. However, input 2 received nothing and the checkpoint expired, new checkpoint is trigger.
> 5. Input channel 0 received barrier 2, checkpoint 1 is deserted and checkpoint 2 is started. However, in this case the state of the input channels are not cleared. Thus now channel 1 is still BARRIER_RECEIVED and numBuffersOvertaken(channel 1) = n_1. Then channel 1 would only persist n_1 buffers in the channel for the new checkpoint 2. 
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)