[jira] [Updated] (BEAM-3337) Create an Executable Graph representation in the Universal Local Runner

["Ismaël Mejía (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-3337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ismaël Mejía updated BEAM-3337:
-------------------------------
    Status: Open  (was: Triage Needed)

> Create an Executable Graph representation in the Universal Local Runner
> -----------------------------------------------------------------------
>
>                 Key: BEAM-3337
>                 URL: https://issues.apache.org/jira/browse/BEAM-3337
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-core
>            Reporter: Thomas Groh
>            Priority: Major
>              Labels: portability
>
> This represents the entirety of a Pipeline, divided into executable stages which are executed in either the runner or within a user container. This representation must include the ports over which the SDK harness communicates to the runner.
> The construction of this graph likely includes most of the nodes present within the Runner API graph (PTransform and PCollection), but injects additional nodes to represent a remote read or write between harnesses.
> Simple fusion (naive producer/consumer and sibling fusion) also should be performed here.
> This will also require rewriting some boundary coders (e.g. runner -> SDK Harness and vice versa) to be agnostic to the language of the runner harness, likely by converting into length-prefixed bytes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)