You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Robert Joseph Evans (JIRA)" <ji...@apache.org> on 2011/07/12 17:44:00 UTC
[jira] [Created] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
MR-279: JobHistory Server needs Analysis this job
-------------------------------------------------
Key: MAPREDUCE-2672
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.23.0
Reporter: Robert Joseph Evans
Assignee: Robert Joseph Evans
Fix For: 0.23.0
The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
This should include the following info
Hadoop Job ID
User :
JobName :
JobConf :
Submitted At :
Launched At : (including duration)
Finished At : (including duration)
Status :
Time taken by best performing Map task <TASK_LINK>:
Average time taken by Map tasks:
Worse performing map tasks: (including task links and duration)
The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
Time taken by best performing shuffle <TASK_LINK>:
Average time taken by shuffle:
Worse performing Shuffles: (including task links and duration)
The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
Time taken by best performing Reduce task <TASK_LINK>:
Average time taken by Reduce tasks:
Worse performing reduce tasks: (including task links and duration)
The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103291#comment-13103291 ]
Siddharth Seth commented on MAPREDUCE-2672:
-------------------------------------------
nice. 0.20 'analyze this job' info + more can be obtained via field sorts in the mrv2 UI. Required MR-2676 to apply. Meanwhile, couple of small issues
- Attempt elapsed time in case of reduce tasks is computing the diff between end time and shuffle finish time (instead of end and launch).
- sort finish time can also be added for ReduceTasks
- Should 'Attempt Shuffle Time' be renamed to 'Shuffle finish time'? Similarly 'attempt elapsed time shuffle' to 'shuffle elapsed time'
Otherwise looks good, the generic webapp tests are part of 2676.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106015#comment-13106015 ]
Hudson commented on MAPREDUCE-2672:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #8 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/8/])
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev) - Merging r1171297 from trunk
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171299
Files :
* /hadoop/common/branches/branch-0.23
* /hadoop/common/branches/branch-0.23/hadoop-common-project
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf/capacity-scheduler.xml.template
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/TaskAttempt.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/TaskAttemptReport.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskAttemptReportPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskReportPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksBlock.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/capacity-scheduler
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/dynamic-scheduler
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/fairscheduler
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106031#comment-13106031 ]
Hudson commented on MAPREDUCE-2672:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #802 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/802/])
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171297
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/TaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/TaskAttemptReport.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskAttemptReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105744#comment-13105744 ]
Hudson commented on MAPREDUCE-2672:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #911 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/911/])
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171297
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/TaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/TaskAttemptReport.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskAttemptReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2672:
-------------------------------------
Status: Open (was: Patch Available)
Bobby/Sid, can you please look at the test failure? Tx
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105111#comment-13105111 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12494562/MR-2672.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/744//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/744//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104025#comment-13104025 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12494330/MR-2672.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/726//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-2672:
--------------------------------------
Status: Open (was: Patch Available)
Updated patch looks good with elapsed times fixed and the additional fields added to the ReducesTasksPage. Cancelling it though to add sort/merge finish time.
They're not the same - a sort phase may run after shuffle completes.
Bobby, this should be a trivial change on top of your patch. Will post an update shortly, if you're ok with that.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104108#comment-13104108 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12494330/MR-2672.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/728//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/728//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105032#comment-13105032 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12494545/MR-2672.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/741//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/741//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2672:
-------------------------------------
Status: Open (was: Patch Available)
Cancelling patch while Sid's comments are addressed.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093185#comment-13093185 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12492138/MR-2672-trunk-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/556//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Patch Available (was: Open)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: MR-2672.patch
Added in more columns to task page and tasks page. Fixed some sorting and filtering issues. Fixed some issues with the task page where it could not get the proper task type.
All known issues should be fixed with this patch and all pages should have similar data.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2672:
-------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks Bobby and Sid!
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105647#comment-13105647 ]
Siddharth Seth commented on MAPREDUCE-2672:
-------------------------------------------
bq. In the job overview the average merge time was not divided by the number of reducers. I will upload a fix for that now.
Thanks Bobby. This should be good to go now.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Patch Available (was: Open)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105736#comment-13105736 ]
Hudson commented on MAPREDUCE-2672:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #973 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/973/])
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171297
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/TaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/TaskAttemptReport.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskAttemptReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099085#comment-13099085 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12493336/MR-2672-trunk-v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/631//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: MR-2672-trunk-v4.patch
An upmerge to the latest patch from MR-2676. Also some fixes for displaying the taskAttempts page.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: MR-2672.patch
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: MR-2672-trunk-v2.patch
Rebased patch on top of trunk.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Patch Available (was: Open)
This patch requires that MAPREDUCE-2676 be applied first.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106054#comment-13106054 ]
Hudson commented on MAPREDUCE-2672:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #8 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/8/])
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev) - Merging r1171297 from trunk
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171299
Files :
* /hadoop/common/branches/branch-0.23
* /hadoop/common/branches/branch-0.23/hadoop-common-project
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/docs
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/core
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/native
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/.gitignore
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/conf/capacity-scheduler.xml.template
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/TaskAttempt.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/TaskAttemptReport.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskAttemptReportPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskReportPBImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_protos.proto
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksBlock.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/c++
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/block_forensics
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build-contrib.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/build.xml
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/capacity-scheduler
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/data_join
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/dynamic-scheduler
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/eclipse-plugin
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/fairscheduler
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/index
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/streaming
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/contrib/vaidya
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/examples
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/hdfs
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/FileBench.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/io/TestSequenceFileMergeProgress.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/ipc
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/test/MapredTestDriver.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/webapps/job
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Patch Available (was: Open)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104105#comment-13104105 ]
Siddharth Seth commented on MAPREDUCE-2672:
-------------------------------------------
Bobby, could you also add SortFinishTime for the 'Reduce Tasks' page.
Not necessarily part of this Jira - but the TaskAttemptsPage can use the same set of columns for reduce tasks.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104751#comment-13104751 ]
Robert Joseph Evans commented on MAPREDUCE-2672:
------------------------------------------------
Yes I am fine with you adding it in. Happy to have all the help I can get.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-2672:
--------------------------------------
Status: Patch Available (was: Open)
Bobby, could you please take a look at the patch. Thx
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: Updated-Column-Layout.jpg
I think I have addressed Sid's comments. I will upload a patch shortly. Here is a screenshot of the new columns for a quick look at what it is doing.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106049#comment-13106049 ]
Hudson commented on MAPREDUCE-2672:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #832 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/832/])
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171297
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/TaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/TaskAttemptReport.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskAttemptReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Patch Available (was: Open)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: MR-2672-v1.patch
I did things a little bit differently for this patch then the JIRA explicitly asked for. I just made sure that all of the information that Analyze Job did could be found in existing pages. This was mostly adding in Shuffle time and adding in some extra functionality to a few data tables.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-2672:
--------------------------------------
Attachment: MR-2672.patch
Fixed the history tests and some AM webapp tests to construct the actual page instead of an error page.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-2672:
--------------------------------------
Status: Patch Available (was: Open)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Open (was: Patch Available)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100528#comment-13100528 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12493632/MR-2672-trunk-v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/653//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: MR-2672.patch
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105397#comment-13105397 ]
Hadoop QA commented on MAPREDUCE-2672:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12494626/MR-2672.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/751//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/751//console
This message is automatically generated.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105375#comment-13105375 ]
Robert Joseph Evans commented on MAPREDUCE-2672:
------------------------------------------------
At first glance it looks good to me. I will load it up and play around with it.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Open (was: Patch Available)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Attachment: MR-2672-trunk-v3.patch
Upmerged and added in some javadocs. This patch relies on MR-2676 patch already being applied.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104497#comment-13104497 ]
Robert Joseph Evans commented on MAPREDUCE-2672:
------------------------------------------------
I can add in the same columns to the attempts page.
I thought that shuffle finish time was the sort finish time. If not I don't see any API to get that information right now so if you need it we will have to add it in under a separate JIRA.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Fix Version/s: 0.24.0
Affects Version/s: 0.24.0
Status: Open (was: Patch Available)
Looks really good, except for one thing. In the job overview the average merge time was not divided by the number of reducers. I will upload a fix for that now.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105740#comment-13105740 ]
Hudson commented on MAPREDUCE-2672:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #896 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/896/])
MAPREDUCE-2672. MR-279: JobHistory Server needs Analysis this job. (Robert Evans via mahadev)
mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171297
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/TaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/TaskAttemptReport.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskAttemptReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/records/impl/pb/TaskReportPBImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/proto/mr_protos.proto
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryParser.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptUnsuccessfulCompletionEvent.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTaskAttempt.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTaskPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksBlock.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsTasksPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Times.java
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-2672:
--------------------------------------
Attachment: MR-2672.patch
Adds a sort/merge complete field to the task pages.
Also, unrelated to this jira, a JH UI fix for elapse time using finishTime as currentTime if it isn't set.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, MR-2672.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Patch Available (was: Open)
This patch still needs MAPREDUCE-2676 patch to be applied first.
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-trunk-v4.patch, MR-2672-v1.patch, MR-2672.patch, Updated-Column-Layout.jpg
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2672) MR-279: JobHistory Server needs
Analysis this job
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-2672:
-------------------------------------------
Status: Open (was: Patch Available)
> MR-279: JobHistory Server needs Analysis this job
> -------------------------------------------------
>
> Key: MAPREDUCE-2672
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2672
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 0.23.0
>
> Attachments: MR-2672-trunk-v2.patch, MR-2672-trunk-v3.patch, MR-2672-v1.patch
>
>
> The JobHistory Server needs to implement the Analysis this job functionality from the previous server.
> This should include the following info
> Hadoop Job ID
> User :
> JobName :
> JobConf :
> Submitted At :
> Launched At : (including duration)
> Finished At : (including duration)
> Status :
> Time taken by best performing Map task <TASK_LINK>:
> Average time taken by Map tasks:
> Worse performing map tasks: (including task links and duration)
> The last Map task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing shuffle <TASK_LINK>:
> Average time taken by shuffle:
> Worse performing Shuffles: (including task links and duration)
> The last Shuffle <TASK_LINK> finished at (relative to the Job launch time): (including duration)
> Time taken by best performing Reduce task <TASK_LINK>:
> Average time taken by Reduce tasks:
> Worse performing reduce tasks: (including task links and duration)
> The last Reduce task <TASK_LINK> finished at (relative to the Job launch time): (including duration)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira