[jira] [Created] (TEZ-2370) Add stages information to RM UI for debugging / visibility on job progress

["Hari Sekhon (JIRA)" <ji...@apache.org>]

Hari Sekhon created TEZ-2370:
--------------------------------

             Summary: Add stages information to RM UI for debugging / visibility on job progress
                 Key: TEZ-2370
                 URL: https://issues.apache.org/jira/browse/TEZ-2370
             Project: Apache Tez
          Issue Type: Improvement
          Components: UI
    Affects Versions: 0.5.2
         Environment: HDP 2.2.0
            Reporter: Hari Sekhon
            Priority: Minor


Something that has been bugging me since last year is the difficulty of debugging Tez jobs compared to MapReduce jobs.

This is because Resource Manager / Application Master does not display the job stats and stages that we are used to seeing in MapReduce eg. Map and Reduce task counts and progress. I appreciate that Tez is a more flexible framework with a DAG but it would be nice if it could surface the information on the different stages, number of tasks running, completed, failed, killed, successful etc, similar to how Spark does, and the stage breakdown would be useful in understanding what the job is doing at different times, what stage is getting stuck/failing etc.

At the moment the only thing available is to trawl the logs or hope to have a console output where some of that information is available, both of which are non-ideal when debugging other's people's jobs after the fact.

Hari Sekhon
http://www.linkedin.com/in/harisekhon



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)