[jira] [Commented] (TEZ-2888) Make critical path calculation resilient to AM crash

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955935#comment-14955935 ] 

Bikas Saha commented on TEZ-2888:
---------------------------------

1) Fixes bad results when AM crashes and dag end time is not recorded. Now we rebase to the last attempt finish time for drawing the svg
2) Changes straggler identification to only look at the time spent after the last data event was received. this should prevent attempts from looking like stragglers because they were launched too early and spent most of their time waiting.
3) Calculates the number of running tasks with time along the job timeline. currently its used to get a guesstimate of the actual observed max concurrency in the job. In the absence of any definitive values from YARN, this would help provide some insight into what actually happened in terms of concurrency. Used max concurrency value to update some wave based heuristics for allocation overhead diagnostics. This concurrency with time series could be used in other things e.g. drawing it along with the critical path to aid in visual debugging.

[~rajesh.balamohan] please review. Thanks!


> Make critical path calculation resilient to AM crash
> ----------------------------------------------------
>
>                 Key: TEZ-2888
>                 URL: https://issues.apache.org/jira/browse/TEZ-2888
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: TEZ-2888.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)