You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Robert Bradshaw <ro...@google.com> on 2018/02/09 00:04:50 UTC

Re: [jira] [Commented] (BEAM-2591) Python shim for submitting to FlinkRunner

FYI, either way, this is probably just adding another case to
https://github.com/apache/beam/blob/c2a088339831c00ce021bd0d3197c3e84b739ede/sdks/python/apache_beam/runners/portability/universal_local_runner.py#L79

On Thu, Feb 8, 2018 at 3:54 PM, Ben Sidhom (JIRA) <ji...@apache.org> wrote:
>
>     [ https://issues.apache.org/jira/browse/BEAM-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357726#comment-16357726 ]
>
> Ben Sidhom commented on BEAM-2591:
> ----------------------------------
>
> [~kenn]
>
> Is this still the case? Do we want the Python SDK to spin up a Flink deployment and then submit jobs to it? If not, then we need the endpoint of an already-running Job API anyway. This is no different for Flink than any other runner.
>
> The main distinction here is if we want the FlinkRunner to run locally, in which case it might be reasonable to have Flink create an in-process "deployment" and execute against it.
>
>> Python shim for submitting to FlinkRunner
>> -----------------------------------------
>>
>>                 Key: BEAM-2591
>>                 URL: https://issues.apache.org/jira/browse/BEAM-2591
>>             Project: Beam
>>          Issue Type: Sub-task
>>          Components: runner-flink, sdk-py-core
>>            Reporter: Kenneth Knowles
>>            Priority: Major
>>              Labels: portability
>>
>> Whatever the result of https://s.apache.org/beam-job-api, Python users will need to be able to pass --runner=FlinkRunner and have it work.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)