[jira] [Commented] (FLINK-14521) CoLocationGroup is not set into JobVertex if the stream node is chained with others

["Yun Gao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958823#comment-16958823 ] 

Yun Gao commented on FLINK-14521:
---------------------------------

The basic fix of this issue is to break the chain of nodes with different coLocation group directly. But I'm wondering if it is suitable to detect conflicts of coLocationGroup more fine-grained:

Suppose A->B->C->D satisfy other conditions for chaining, and A->B->C have get chained and we are considering C->D, and D has declared the coLocationGroup, Then
 # if none of A, B, C has declared the coLocationGroup, then D can be chained with A, B, C and the resulted vertex's coLocationGroup is same as D.
 # If some of A, B and C have declared the same coLocationGroup as D, then D can be chained with A, B, C and resulted vertex's coLocationGroup is same.
 # If If some of A, B and C have declared a different coLocationGroup from D , then D can not be chained with A, B, C.

With this method we should be able to achieve more chance to chain nodes.

> CoLocationGroup is not set into JobVertex if the stream node is chained with others
> -----------------------------------------------------------------------------------
>
>                 Key: FLINK-14521
>                 URL: https://issues.apache.org/jira/browse/FLINK-14521
>             Project: Flink
>          Issue Type: Bug
>          Components: API / DataStream
>    Affects Versions: 1.9.0, 1.9.1
>            Reporter: Yun Gao
>            Priority: Major
>
> StreamingJobGraphGenerator.isChainable dose not consider the coLocationGroup, if A -> B is chained, the coLocationGroup of the corresponding JobVertex will be set with that of the head node, namely A. Therefore, if B has declared coLocationGroup but A does not, then the coLocationGroup of B will be ignored.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)