You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Robert Burke (Jira)" <ji...@apache.org> on 2022/03/18 17:25:00 UTC

[jira] [Created] (BEAM-14128) Quadratic behavior in go pipeline rewriting.

Robert Burke created BEAM-14128:
-----------------------------------

             Summary: Quadratic behavior in go pipeline rewriting.
                 Key: BEAM-14128
                 URL: https://issues.apache.org/jira/browse/BEAM-14128
             Project: Beam
          Issue Type: Bug
          Components: sdk-go
    Affects Versions: 2.37.0
            Reporter: Robert Burke


Work is repeated for every node outside of a given composite, when checking if anything is using any outputs of transforms internal to the composite.

This bug significantly increases the construction time large pipeline graphs, but can be resolved with bit of pre-computation and plumbing of which transforms are using which PCollections.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)