[GitHub] [flink] eaglewatcherwb edited a comment on issue #8446: [FLINK-12414] [runtime] Implement ExecutionGraph to SchedulingTopology

[GitBox <gi...@apache.org>]

eaglewatcherwb edited a comment on issue #8446: [FLINK-12414] [runtime] Implement ExecutionGraph to SchedulingTopology
URL: https://github.com/apache/flink/pull/8446#issuecomment-492557166
 
 
   Hi @tillrohrmann @GJL @zentol 
   
   I have submitted the firs version of the PR. When I implemented this, both memory overhead and performance are considered and two plans are optional. The first is to only store `ExecutionGraph` in the adapter, construct and return a temporary variable when `getVertex` or `getResultPartition`. The advantage is that it involves the least memory overhead. But I am wondering that the performance may be poor, even constructing a single `SchedulingExecutionVertex` or `SchedulingResultPartition`, the producers and consumers will be recursively constructed, which will contain all the ancestor and descendant nodes, may be all the nodes of the graph in the worst case. Since `getVertex` and `getResultPartition` are frequently called when partition/execution state change, the performance will be critical. What's more, too many temporary variables will also lead to GC regularly, which is another time-consuming process.
   
   
   Thus, in current PR, I adopt the second plan, which stores all the `SchedulingExecutionVertex` and `SchedulingResultPartition` in the adapter as member variables, which is of good performance but involves additional memory overhead to store the information.
   
   What do you think? Any comment is highly appreciated.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services