You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Maximilian Michels (Jira)" <ji...@apache.org> on 2019/11/06 12:19:00 UTC

[jira] [Created] (BEAM-8566) Checkpoint buffer is flushed prematurely when another bundle is started during checkpointing

Maximilian Michels created BEAM-8566:
----------------------------------------

             Summary: Checkpoint buffer is flushed prematurely when another bundle is started during checkpointing
                 Key: BEAM-8566
                 URL: https://issues.apache.org/jira/browse/BEAM-8566
             Project: Beam
          Issue Type: Bug
          Components: runner-flink
            Reporter: Maximilian Michels
            Assignee: Maximilian Michels
             Fix For: 2.18.0


As part of a checkpoint, the current bundle is finalized. When the bundle is finalized, the watermark, currently held back, may also be progressed which can cause the start of another bundle. When a new bundle is started, any to-be-buffered items from the previous bundle for the pending checkpoint may be emitted. This should not happen.

This only effects portable pipelines where we have to hold back the watermark due to the asynchronous processing of elements.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)