[jira] [Updated] (TEZ-2646) Add scheduling casual dependency for attempts

["Bikas Saha (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bikas Saha updated TEZ-2646:
----------------------------
    Attachment: TEZ-2646.1.patch

Patch adds scheduling causal task attempt and scheduling time to every new attempt that starts. This allows analysers to know why and when an attempt was scheduled.
ATS File Parser updated to account for the new data. Tests added.
[~rajesh.balamohan] Please review.

> Add scheduling casual dependency for attempts
> ---------------------------------------------
>
>                 Key: TEZ-2646
>                 URL: https://issues.apache.org/jira/browse/TEZ-2646
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-2646.1.patch
>
>
> When a task gets scheduled then we dont know what caused it. Some possibilities are
> 1) initial scheduling by the vertex manager - causality determined by VM. E.g. dynamic partition pruning VM in Hive can point causality to the attempt that sent it the stats needed to complete the partition pruning logic.
> 2) re-scheduling due to own previous version failure - causality points to the previous version that just failed
> 3) re-scheduling because read error reported by consumer - causality points to the consumer attempt that reported the error and caused the scheduling.
> This causality relationship can be used to stitch together scheduling dependencies in the execution timeline of the DAG.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)