You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Till Rohrmann (Jira)" <ji...@apache.org> on 2021/03/05 18:38:00 UTC
[jira] [Closed] (FLINK-21326) Optimize building topology when
initializing ExecutionGraph
[ https://issues.apache.org/jira/browse/FLINK-21326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Till Rohrmann closed FLINK-21326.
---------------------------------
Resolution: Fixed
Fixed via
7e5b388a23fc35353fe0bf174b31f727f594cfa5
e2379c1a09c3f92c32966cceb4987ec78001c5dd
91c2d95440c3c1432b7d2b856dd07923366d5842
> Optimize building topology when initializing ExecutionGraph
> -----------------------------------------------------------
>
> Key: FLINK-21326
> URL: https://issues.apache.org/jira/browse/FLINK-21326
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Zhilong Hong
> Assignee: Zhilong Hong
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> The main idea of optimizing the procedure of building topology is to put all the vertices that consumed the same result partitions into one group, and put all the result partitions that have the same consumer vertices into one consumer group. The corresponding data structure is {{ConsumedPartitionGroup}} and {{ConsumerVertexGroup}}. {{EdgeManager}} is used to store the relationship between the groups. The procedure of creating {{ExecutionEdge}} is replaced with building {{EdgeManager}}.
> With these improvements, the complexity of building topology in ExecutionGraph decreases from O(N^2) to O(N).
> Furthermore, {{ExecutionEdge}} and all its related calls are replaced with {{ConsumedPartitionGroup}} and {{ConsumerVertexGroup}}.
> The detailed design doc is located at: https://docs.google.com/document/d/1OjGAyJ9Z6KsxcMtBHr6vbbrwP9xye7CdCtrLvf8dFYw/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)