You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Aljoscha Krettek (JIRA)" <ji...@apache.org> on 2018/02/22 13:07:00 UTC

[jira] [Created] (BEAM-3727) Never shutdown sources in Flink Streaming execution mode

Aljoscha Krettek created BEAM-3727:
--------------------------------------

             Summary: Never shutdown sources in Flink Streaming execution mode
                 Key: BEAM-3727
                 URL: https://issues.apache.org/jira/browse/BEAM-3727
             Project: Beam
          Issue Type: Bug
          Components: runner-flink
            Reporter: Aljoscha Krettek
            Assignee: Aljoscha Krettek


Because of FLINK-2491 checkpointing will not work for a streaming pipeline as soon as at least one source has finished. With the move to SDF IOs and the impulse primitive this is very problematic because essentially all jobs running in Flink Streaming mode would run without checkpointing.

We should, by default, never stop sources in streaming mode but add an option that allows doing so. Some people might want this and we also need it to make sure that the {{@ValidatesRunner}} tests for Flink in streaming mode finish. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)