[jira] [Commented] (TEZ-800) One-one edge with parallelism -1 fails if source vertex parallelism is not -1 as well

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975958#comment-13975958 ] 

Bikas Saha commented on TEZ-800:
--------------------------------

This could be done at compile time. When the user call DAG.createDAG then the DAG compiler can assign the parallelism where its missing and error out if the parallelism is inconsistent. This way we can avoid having to solve this problem at runtime where the assumptions made by the current code make the code simpler.

> One-one edge with parallelism -1 fails if source vertex parallelism is not -1 as well
> -------------------------------------------------------------------------------------
>
>                 Key: TEZ-800
>                 URL: https://issues.apache.org/jira/browse/TEZ-800
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.3.0
>            Reporter: Rohini Palaniswamy
>            Assignee: Bikas Saha
>
> // no input initializers. At this moment, only other case is 1-1 edge
>           // with uninitialized sources
>           boolean hasOneToOneUninitedSource = false;
>           for (Map.Entry<Vertex, Edge> entry : vertex.sourceVertices.entrySet()) {
>             if (entry.getValue().getEdgeProperty().getDataMovementType() == 
>                 DataMovementType.ONE_TO_ONE) {
>               if (entry.getKey().getTotalTasks() == -1) {
>                 hasOneToOneUninitedSource = true;
>                 break;
>               }
>             }
>           }
> This checks for the source vertex which has the 1-1 edge to also have -1 parallelism.



--
This message was sent by Atlassian JIRA
(v6.2#6252)