You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Henning Rohde (JIRA)" <ji...@apache.org> on 2017/12/08 01:51:00 UTC

[jira] [Created] (BEAM-3316) Consider: do not use graph.Graph for runtime bundle representation

Henning Rohde created BEAM-3316:
-----------------------------------

             Summary: Consider: do not use graph.Graph for runtime bundle representation
                 Key: BEAM-3316
                 URL: https://issues.apache.org/jira/browse/BEAM-3316
             Project: Beam
          Issue Type: Improvement
          Components: sdk-go
            Reporter: Henning Rohde
            Assignee: Henning Rohde
            Priority: Minor


The internal graph.Graph representation is used to unify user pipeline and the runtime subgraph. Perhaps we should instead translate the bundle descriptors directly to an execution plan (=
 []exec.Unit, but a encapsulating type seems useful). Aside: we would likely also need to have them be re-usable across bundles serially.

Such a change would keep the construction-time and runtime worlds more apart instead of trying to unify them and we could remove DataSource/DataSink from the graph. The direct runner would then only translate a graph.Graph to []Unit and not have an awkward injection point for a DataManager.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)