You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Bikas Saha (JIRA)" <ji...@apache.org> on 2015/08/19 03:39:45 UTC
[jira] [Comment Edited] (TEZ-2690) Add critical path analyser
[ https://issues.apache.org/jira/browse/TEZ-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702282#comment-14702282 ]
Bikas Saha edited comment on TEZ-2690 at 8/19/15 1:39 AM:
----------------------------------------------------------
Adds a task attempt level critical path analyzer.
Uses the scheduling and data event dependencies to walk from the last attempt completion to first attempt creation to account for the time taken in the job.
The output of the analyzer is an svg rendering of the critical path. Attached sample. The svg code has been re-written to generate svg directly instead of using jaxb because of missing features in jaxb (e.g. setting the value of a text field).
Renames existing critical path analyzer to vertex level.
Adds an AnalyzerDriver to allow running analyzers from the command line using hadoop jar command. Only the latest CriticalPathAnalyzer has been added to the driver because I am not sure how the other analyzers would behave on the command line. They are written to output csv results. Perhaps we can create a base Csv analyzer that can take the csv results and output them on the console or write them to a file. Then they could be run on the command line.
The goal is to get this in and have motivated developers start running it and finding issues/improvements.
[~rajesh.balamohan] Please review.
!criticalPath.png|thumbnail!
was (Author: bikassaha):
Adds a task attempt level critical path analyzer.
Uses the scheduling and data event dependencies to walk from the last attempt completion to first attempt creation to account for the time taken in the job.
The output of the analyzer is an svg rendering of the critical path. Attached sample. The svg code has been re-written to generate svg directly instead of using jaxb because of missing features in jaxb (e.g. setting the value of a text field).
Renames existing critical path analyzer to vertex level.
Adds an AnalyzerDriver to allow running analyzers from the command line using hadoop jar command. Only the latest CriticalPathAnalyzer has been added to the driver because I am not sure how the other analyzers would behave on the command line. They are written to output csv results. Perhaps we can create a base Csv analyzer that can take the csv results and output them on the console or write them to a file. Then they could be run on the command line.
The goal is to get this in and have motivated developers start running it and finding issues/improvements.
[~rajesh.balamohan] Please review.
> Add critical path analyser
> --------------------------
>
> Key: TEZ-2690
> URL: https://issues.apache.org/jira/browse/TEZ-2690
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-2690.1.patch, criticalPath.png
>
>
> Use input and scheduling dependencies to create critical path for a DAG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)