[jira] [Commented] (BEAM-139) Print mode (batch/streaming) during translation

["Davor Bonaci (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206547#comment-15206547 ] 

Davor Bonaci commented on BEAM-139:
-----------------------------------

We are hoping to be able to remove the "--streaming" flag altogether in the near future. With the Runner API refactoring, a runner should be able to observe the pipeline graph in its entirety, and automatically choose parameters like the execution mode, without them being specified the user.

For example, if there are unbounded PCollections in the graph, a runner may choose the streaming execution mode. Note, however, that a runner may choose "micro-batch" too. Essentially, this is a runner-specific decision that with a better Runner API, each runner can do unilaterally, without the concept being exposed to the user.

The current "--streaming" flag is an unfortunate consequence of the existing runner API. In that sense, I'd close this issue as we think this will go away shortly, and doesn't need to be printed.

Sounds good?

> Print mode (batch/streaming) during translation 
> ------------------------------------------------
>
>                 Key: BEAM-139
>                 URL: https://issues.apache.org/jira/browse/BEAM-139
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-core, runner-flink
>            Reporter: Maximilian Michels
>            Assignee: Davor Bonaci
>
> Runners have different feature sets for batch and streaming. It may be a good idea to print/log the translation mode during parsing of the options in the {{PipelineOptionsFactory}}. That would help users to understand they are missing the streaming flag in the options and that the default mode is batch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)