You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Thomas Weise (JIRA)" <ji...@apache.org> on 2018/10/02 01:20:00 UTC

[jira] [Resolved] (BEAM-5520) Flink runner per operator SDK harness option

     [ https://issues.apache.org/jira/browse/BEAM-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Weise resolved BEAM-5520.
--------------------------------
       Resolution: Fixed
    Fix Version/s: 2.8.0

> Flink runner per operator SDK harness option
> --------------------------------------------
>
>                 Key: BEAM-5520
>                 URL: https://issues.apache.org/jira/browse/BEAM-5520
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-flink
>            Reporter: Thomas Weise
>            Assignee: Thomas Weise
>            Priority: Major
>              Labels: portability, portability-flink
>             Fix For: 2.8.0
>
>          Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> For streaming pipelines, the runner currently uses a single SDK harness process for all subtasks of a job that get deployed on the same task manager. In common deployments with 16 or more tasks slots, many executable stage operators all use the same SDK harness process. To scale, we need an option to run separate harness processes per subtask.
>    



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)