[jira] [Updated] (TEZ-933) Race in getting source / destination numTasks on an Edge

["Siddharth Seth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siddharth Seth updated TEZ-933:
-------------------------------

    Attachment: TEZ-933.1.txt

Trivial patch - which makes sure getTotalTasks on a vertex returns what the user specified instead of 0. getNumTasks, depending on when it's invoked will now at least give back what the correct number is at the time (instead of 0 rightnow).

I haven't changed the edge API to include initialNumTasks / numTasks as separate values - that can be a separate jira - if required.

[~hitesh] - review please.

> Race in getting source / destination numTasks on an Edge
> --------------------------------------------------------
>
>                 Key: TEZ-933
>                 URL: https://issues.apache.org/jira/browse/TEZ-933
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>         Attachments: TEZ-933.1.txt
>
>
> Edges rely on getting properties (specifically numTasks in this case) from the source or destination vertex.
> This can end up with an incorrect value being used depending on the state of the vertex - whether the vertex has been initialized, whether the parallelism has been changed etc.
> As an example
> {code}
> edgeManager.getNumSourceTaskPhysicalOutputs(destinationVertex.getTotalTasks(), sourceTaskIndex))
> {code}
> destinationVertex.getTotalTasks() may be incorrect if the destinationVertex hasn't yet been initialized. Alternately, this value can change based on setParallelism calls.



--
This message was sent by Atlassian JIRA
(v6.2#6252)