[jira] [Commented] (TEZ-608) Fix 1-1 edge connection when parallelism is determined at runtime

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819817#comment-13819817 ] 

Bikas Saha commented on TEZ-608:
--------------------------------

Flow is as follows
When parallelism unknown then set to -1. When the source 1-1 vertex inits and splits then it will send a split event to the downstream 1-1 vertex with real number of tasks. This sets the parallelism of the vertex. Downstream vertex may get multiple split events legitimately (say in a rhombus shaped graph). Only the first one should cause a setParallelism. Others should be verified and ignored.
Changed vertex init to be sent in a topologically sorted manner so that tasks init in the correct order.
Added tests and real life example dag.

> Fix 1-1 edge connection when parallelism is determined at runtime
> -----------------------------------------------------------------
>
>                 Key: TEZ-608
>                 URL: https://issues.apache.org/jira/browse/TEZ-608
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-608.1.patch, TEZ-608.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.1#6144)