You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Robert Burke (Jira)" <ji...@apache.org> on 2022/03/18 17:26:00 UTC
[jira] [Commented] (BEAM-14128) Quadratic behavior in go pipeline rewriting.
[ https://issues.apache.org/jira/browse/BEAM-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508966#comment-17508966 ]
Robert Burke commented on BEAM-14128:
-------------------------------------
Assigning to me but the bug was found, and fixed by https://github.com/lukakalinovcic, but they didn't wish to go through the overhead of a full contributor set up.
PR: https://github.com/apache/beam/pull/17120
> Quadratic behavior in go pipeline rewriting.
> --------------------------------------------
>
> Key: BEAM-14128
> URL: https://issues.apache.org/jira/browse/BEAM-14128
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Affects Versions: 2.37.0
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: P2
>
> Work is repeated for every node outside of a given composite, when checking if anything is using any outputs of transforms internal to the composite.
> This bug significantly increases the construction time large pipeline graphs, but can be resolved with bit of pre-computation and plumbing of which transforms are using which PCollections.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)