You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (Created) (JIRA)" <ji...@apache.org> on 2011/10/29 00:16:32 UTC
[jira] [Created] (MAPREDUCE-3309) Report the AM of an application
in the UI
Report the AM of an application in the UI
-----------------------------------------
Key: MAPREDUCE-3309
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
Make provision to report the AM hostname of an application in the RM/JHS UI.
It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Attachment: MAPREDUCE-3309.patch
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149669#comment-13149669 ]
Jonathan Eagles commented on MAPREDUCE-3309:
--------------------------------------------
Canceling patch while looking into the test failure
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Attachment: MAPREDUCE-3309.patch
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149740#comment-13149740 ]
Jonathan Eagles commented on MAPREDUCE-3309:
--------------------------------------------
Fixed the test error for TestRMWebApp.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149751#comment-13149751 ]
jiraposter@reviews.apache.org commented on MAPREDUCE-3309:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2822/
-----------------------------------------------------------
Review request for Tom Graves, Robert Evans and Mark Holderbaugh.
Summary
-------
Make provision to report the AM hostname of an application in the RM/JHS UI.
It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
------------------
The big idea is to add the application master to the jobhistory ui so that can be sort and increase search capabilities. Currently, adding anything to the jobhistory apps page means to add to the job history file name since that is parsed to get job meta data. This is done for performance reasons to prevent reading all jobhistory files.
This addresses bug MAPREDUCE-3309.
http://issues.apache.org/jira/browse/MAPREDUCE-3309
Diffs
-----
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java c4ef938
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java a27ba15
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/DefaultSchedulerPage.java 8db4caf
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java 700fdb3
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmView.java 7c175d1
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java 1e9215f
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsList.java 57e695c
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java ca4ab18
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/PartialJob.java d412a63
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java 0a6b969
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsView.java ddb904d
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobIndexInfo.java 3c0f3bb
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java 3a35045
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java 0d0271c
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java ae886cf
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/Job.java 4abcd34
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/JobImpl.java dd19ed0
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockJobs.java 5a67576
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java f82c1d5
Diff: https://reviews.apache.org/r/2822/diff
Testing
-------
Manual test done:
1. Verify correct application master on job history apps page with jobhistory files using new format
2. Verify old job history files are still usable to retrieve meta data minus the application master
3. Verify all fields are searchable using field search as well and general search
Thanks,
Jonathan
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150572#comment-13150572 ]
Jonathan Eagles commented on MAPREDUCE-3309:
--------------------------------------------
No worries, Vinod. Thanks for pointing out MAPREDUCE-3350.
AMInfos information will be great for the cluster/app/application_* for the RM and the jobhistory/job/job_* for the JHS, so I'll save that for MAPREDUC-3350. It gets tricky to display the AMInfos the JHS jobhistory/app page, though. That page is displayed using data from the PartialJob that gets constructed from the meta data embedded in the job history file name. So unlike the CompletedJob, there the AMInfos is not populated. I had considered faking the AMInfos and reusing that API, but it was too messy. The other more drastic method is to stop using the PartialJob for the JHS and to only use the CompletedJob, but I worry about the performance and responsiveness of the JHS.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles reassigned MAPREDUCE-3309:
------------------------------------------
Assignee: Jonathan Eagles
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.0
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Status: Open (was: Patch Available)
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Attachment: MAPREDUCE-3309.patch
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149752#comment-13149752 ]
Hadoop QA commented on MAPREDUCE-3309:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12503637/MAPREDUCE-3309.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1303//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1303//console
This message is automatically generated.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150383#comment-13150383 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3309:
----------------------------------------------------
Oh and apologies for not noticing this earlier itself.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148804#comment-13148804 ]
Jonathan Eagles commented on MAPREDUCE-3309:
--------------------------------------------
Fixed job history searching with latest patch
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151279#comment-13151279 ]
Jonathan Eagles commented on MAPREDUCE-3309:
--------------------------------------------
On second thought, with much help from Vinod, AM on the JHS front can't be done correctly since an AM restart will cause two AMs. Let's leave this info to the per job page on the JHS and let's use MAPREDUCE-3350 to implement AM infos on the per job page for the RM.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Status: Patch Available (was: Open)
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148828#comment-13148828 ]
Hadoop QA commented on MAPREDUCE-3309:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12503435/MAPREDUCE-3309.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 9 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebApp
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1296//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1296//console
This message is automatically generated.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141717#comment-13141717 ]
Jonathan Eagles commented on MAPREDUCE-3309:
--------------------------------------------
Attaching a patch that has a working version of the RM portion of the fix.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3309:
-----------------------------------------------
Status: Open (was: Patch Available)
Jonathan,
The AM(possibly multiple per job) information is already being logged into JobHistory and displayed on JobHistoryServer. See AMInfo and HsJobBlock (+156).
Agreed that we need to show this on the RM and web-app UI too, you can use the already present {{List<AMInfo> getAMInfos()}} instead of a new API.
Once you do that, you can close MAPREDUCE-3350 as a duplicate.
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Attachment: MAPREDUCE-3309.patch
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Status: Patch Available (was: Open)
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles updated MAPREDUCE-3309:
---------------------------------------
Target Version/s: 0.23.1
Fix Version/s: (was: 0.23.1)
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3309) Report the AM of an application
in the UI
Posted by "Jonathan Eagles (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles resolved MAPREDUCE-3309.
----------------------------------------
Resolution: Won't Fix
> Report the AM of an application in the UI
> -----------------------------------------
>
> Key: MAPREDUCE-3309
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Jonathan Eagles
> Attachments: MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch, MAPREDUCE-3309.patch
>
>
> Make provision to report the AM hostname of an application in the RM/JHS UI.
> It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira