[jira] [Commented] (TEZ-265) Provide aggregated information in the AM logs after / during job execution

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688965#comment-13688965 ] 

Siddharth Seth commented on TEZ-265:
------------------------------------

Statistics like average runtime for tasks in a stage, longest runtime etc will be useful. As long as the output is easy to use with awk etc - these stats should be easy to generate later as well. Locality information would be good, but I think that's a separate jira.
                
> Provide aggregated information in the AM logs after / during job execution
> --------------------------------------------------------------------------
>
>                 Key: TEZ-265
>                 URL: https://issues.apache.org/jira/browse/TEZ-265
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Hitesh Shah
>         Attachments: TEZ-265.1.patch
>
>
> The history events currently have specific information like task start time, finish time etc. It's rather difficult to analyze the job based on this.
> While there's no AM UI / history UI - it'd be useful to generate aggregated information from all the history events. e.g. Task Attempt runtime, Vertex runtime etc.
> This could be part of the AM, a separate script etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira