You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Bhallamudi Venkata Siva Kamesh (Created) (JIRA)" <ji...@apache.org> on 2011/11/05 10:28:51 UTC
[jira] [Created] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Provide information about lost nodes in the UI.
-----------------------------------------------
Key: MAPREDUCE-3360
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.23.0
Environment: NA
Reporter: Bhallamudi Venkata Siva Kamesh
Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181556#comment-13181556 ]
Hadoop QA commented on MAPREDUCE-3360:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509705/MAPREDUCE-3360-4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1557//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1557//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1557//console
This message is automatically generated.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Status: Open (was: Patch Available)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175120#comment-13175120 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
The reboot case is interesting since the code does not seem use {{RMNodeState.REBOOT}}. Therefore the status page for "Rebooted Nodes" will always be empty even with this patch.
This makes me wonder whether the metrics for the inactive states were intended to be current or historical. If the latter is the intent, then the node status pages for LOST, REBOOT, etc. would be for listing all nodes that went through those states since the RM started. For example, it might be useful to know that a particular node has been lost many times even though that node is currently active.
If we want the metrics to track the number of nodes currently in a given state, as the current patch does, then we should either remove the "Rebooted Nodes" column from the metrics since nodes will never be in the REBOOT state or add code to support the REBOOT state. For example, we'd have to update the {{StateMachineFactory}} in RMNodeImpl.java to transition to the REBOOT state instead of the LOST state when the REBOOTING event is delivered. And in the code you ran across, we'd probably not want to increment the reboot count if the node is in the inactive nodes map already. If it wasn't in the map, we'd increment the reboot counter and add it to the inactive node map.
I guess this brings up the question of whether we should allow the counts to ever get out of sync with what's stored in the inactive node map. Currently they're tracked separately and could become out of sync (like the code you ran across), but it would be easier to guarantee they're consistent if the counts were computed from the inactive node collection directly. (e.g.: by scanning the inactive map or having a map per inactive node state).
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jonathan Eagles (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles reassigned MAPREDUCE-3360:
------------------------------------------
Assignee: Jonathan Eagles
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Thomas Graves (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183497#comment-13183497 ]
Thomas Graves commented on MAPREDUCE-3360:
------------------------------------------
{quote}
If we change nodeHttpAddress to empty string in RMWebServices.java, then it conflicts with the way we have been displaying it in the UI, as we have been displaying it as "N/A" for inactive nodes. Or do we need to capture this behaviour some where in the documentation?
{quote}
its ok for the webui to display something slightly different then the web services. The UI should be "pretty and appealing to users" where as the web services should be programmatic. In my opinion, the empty string would be more programmatic and the N/A more user appealing. I will make sure to document what empty fields mean in MAPREDUCE-3549.
{quote}
After this change, if we provide true totalNodes info through web services, do we need to provide the same info on the UI also?.
{quote}
I don't have a strong opinion either way. I guess it would be nice if ops or someone wanted at quick glance to see total nodes to have it in the web ui. My only concern would be if it makes the page to wide.
In general, the web services should include everything the web ui does but I don't think it strictly necessarily to have everything from web services in the web ui though. The main reason I say to leave totalNodes in the webservices is because that field was already there and I would like to stay away from removing fields unless we change the version number.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182544#comment-13182544 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
+1, looks good to me.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Attachment: MAPREDUCE-3360-5.patch
Addressing above comments
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-3360:
----------------------------------
Attachment: MAPREDUCE-3360.patch
Upmerged patch to changes on trunk.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bh V S Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bh V S Kamesh updated MAPREDUCE-3360:
-------------------------------------
Attachment: LostNodes.png
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Attachments: LostNodes.png, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180760#comment-13180760 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
+1, looks good to me.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194253#comment-13194253 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #421 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/421/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
svn merge --ignore-ancestry -c 1236433 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236438
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bh V S Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bh V S Kamesh updated MAPREDUCE-3360:
-------------------------------------
Attachment: MAPREDUCE-3360-1.patch
Hi Jason,
Thanks for the review.
bq 1. Should the "Total Nodes" column header on the nodes pages be changed to something like "Active Nodes"? Currently it sounds like it should be a count of all the nodes (active or not) associated with the cluster, but it's only counting the active nodes. Related to this, should there be a page listing all nodes regardless of state (i.e.: a real Total Nodes page)?
Yes currently it shows only active nodes in the cluster. Even I *think*, changing it to Active Nodes would be more appropriate (as it does).
bq.3. Maintenance: ClusterMetrics.decr(RMNodeState) doesn't handle all the node states. May be better to just to remove this method and have the one place it's used do the switch.
I *think*, there are only three events, which causes the NM to lost, and all the 3 events have been handled.
RUNNING -> UNHEALTY, and UNHEALTY -> RUNNING transitions have been handled in their corresponding transition hooks.
bq. Nit: The "N/A" web address for inactive nodes shouldn't be a hyperlink, since it doesn't go anywhere useful.
For inactive nodes, hyperlink appears as "N/A" however clicking on that does not go anywhere.
Please once refer the attached screen shot.
2. There will be bookkeeping issues for the inactive nodes when a single host has been configured with multiple nodemanager instances. (A bit odd, but possible to setup.) Since the inactive nodes are tracked only by hostname, we will remove a node from the inactive list when a new nodemanager appears on a different port. Probably best to track that issue in a separate JIRA. There are other issues with that setup, e.g.: inability to detect redundant nodemanager launches, limit nodemanager instances, etc.
If there is only one NM running on a host, there won't be any problem. However if there are multiple NMs running on a single host, it will be a problem.
If the NMs running on a particular host configured to use ephemeral ports, there is no such mechanism to identify NMs comeback.
Filing a separate JIRA for this
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3360:
-----------------------------------------------
Priority: Critical (was: Major)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185268#comment-13185268 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3360:
----------------------------------------------------
Looking at the patch for final review/commit. Completely missed this ticket and all the conversation that happened, will take a little bit of time.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bh V S Kamesh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174947#comment-13174947 ]
Bh V S Kamesh commented on MAPREDUCE-3360:
------------------------------------------
Hi Jason,
Thanks for comments. Will incorporate your comments in my next patch. But before submitting patch, would like clarify this.
When the RM, does not receive node heartbeat from an NM for *node expiry* interval, RM removes the NM from its RM Nodes Map under node *EXPIRE* event.
Before removing the NM, corresponding Cluster metrics will be updated (In this case, incrementing *lost* node count)
If the same NM sends heartbeat after above operation, RM checks whether there is any node corresponding to this NodeId. If RM does not find any NM corresponding to the NodeId, RM simply returns *reboot* as its heartbeat response.
Before sending its heartbeat reponse, RM again updates the Cluster metrics (this time, incrementing *reboot* node count).
Is it necessary to update different metrics for the same node's unavailability?
IMO, it shows incorrect information. I *think* either we need to update *lost* node count or *reboot* node count but not both, in such circumstance.
any comments?
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193360#comment-13193360 ]
Mahadev konar commented on MAPREDUCE-3360:
------------------------------------------
+1 the patch looks good to me. Ill rerun through hudson to verify if the patch hasnt gone stale, will commit on ok from hudson.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194190#comment-13194190 ]
Mahadev konar commented on MAPREDUCE-3360:
------------------------------------------
Thanks Jason. I was also trying to update the patch :).
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193370#comment-13193370 ]
Hadoop QA commented on MAPREDUCE-3360:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12510160/MAPREDUCE-3360-6.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 27 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1676//console
This message is automatically generated.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Attachment: MAPREDUCE-3360-3.patch
Thanks Jason for review. Yes, Nodes in the Unhealthy state should not be part of inactive map. Updated my earlier patch removing that part. Also added some test cases.Please review
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170674#comment-13170674 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
This looks like a duplicate of MAPREDUCE-3271.
A few comments on the patch:
1. Should the "Total Nodes" column header on the nodes pages be changed to something like "Active Nodes"? Currently it sounds like it should be a count of all the nodes (active or not) associated with the cluster, but it's only counting the active nodes. Related to this, should there be a page listing all nodes regardless of state (i.e.: a real Total Nodes page)?
2. There will be bookkeeping issues for the inactive nodes when a single host has been configured with multiple nodemanager instances. (A bit odd, but possible to setup.) Since the inactive nodes are tracked only by hostname, we will remove a node from the inactive list when a new nodemanager appears on a different port. Probably best to track that issue in a separate JIRA. There are other issues with that setup, e.g.: inability to detect redundant nodemanager launches, limit nodemanager instances, etc.
3. Maintenance: ClusterMetrics.decr(RMNodeState) doesn't handle all the node states. May be better to just to remove this method and have the one place it's used do the switch.
3. Nit: The "N/A" web address for inactive nodes shouldn't be a hyperlink, since it doesn't go anywhere useful.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Attachments: MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194647#comment-13194647 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #938 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/938/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236433
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Thomas Graves (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13184463#comment-13184463 ]
Thomas Graves commented on MAPREDUCE-3360:
------------------------------------------
Thanks Bhallamudi
Changes look good to me. +1 (non-binding)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194240#comment-13194240 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3360:
----------------------------------------------------
Thanks for the update, Jason!
The latest patch looks good overall to me. Except one issue: the nodes which get rebooted will never move off that list as currently NMs don't automatically reboot themselves (MAPREDUCE-3034). But the rebooting itself happens very rarely - when NM lose RPCs (don't think it happens anymore now) or RM goes down and comes back which is a different issue altogether anyways.
So, we can get this in for now. +1. Pushing it in.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194262#comment-13194262 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #446 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/446/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
svn merge --ignore-ancestry -c 1236433 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236438
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Attachment: MAPREDUCE-3360-6.patch
Thanks Thomas. Addressing above comments.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Status: Patch Available (was: Open)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194258#comment-13194258 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1598 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1598/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236433
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3360:
-------------------------------------
Status: Open (was: Patch Available)
Can you please look at the findbugs' warning?
Also, can we ensure this information is available via the web-services api as well? Thanks!
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194657#comment-13194657 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #151 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/151/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
svn merge --ignore-ancestry -c 1236433 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236438
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194681#comment-13194681 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #971 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/971/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236433
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Attachment: MAPREDUCE-3360-4.patch
Hi Arun,
Thanks for looking into this issue.
findbugs' warning is not related this patch.
{noformat}org.apache.hadoop.mapreduce.v2.app.launcher.ContainerRemoteLaunchEvent defines equals but not hashCode{noformat}
Attaching a patch which provies the information through web-services api as well.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jonathan Eagles (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Eagles reassigned MAPREDUCE-3360:
------------------------------------------
Assignee: (was: Jonathan Eagles)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Priority: Blocker
> Attachments: MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175843#comment-13175843 ]
Bhallamudi Venkata Siva Kamesh commented on MAPREDUCE-3360:
-----------------------------------------------------------
bq. we should either remove the "Rebooted Nodes" column from the metrics since nodes will never be in the REBOOT state or add code to support the REBOOT state.For example, we'd have to update the StateMachineFactory in RMNodeImpl.java to transition to the REBOOT state instead of the LOST state when the REBOOTING event is delivered.
I *think* each metric denotes a specific reason for node's inactive state,so, updating the StateMachineFactory than removing the *Rebooted Nodes* column from the metrics seems appropriate.
bq. And in the code you ran across, we'd probably not want to increment the reboot count if the node is in the inactive nodes map already. If it wasn't in the map, we'd increment the reboot counter and add it to the inactive node map.
I *think*, updating the *REBOOT* metrics, as I mentioned in my earlier comment, seems to be unnecessary as updation of correct metrics has already been done while removing the node due to *LOST* event.
any comments?
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe reassigned MAPREDUCE-3360:
-------------------------------------
Assignee: Bhallamudi Venkata Siva Kamesh
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179027#comment-13179027 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
Thanks for the updates. A couple of things about the handling of UNHEALTHY nodes:
- They are not removed from the list of nodes being tracked in the context ( {{rmNode.context.getRMNodes()}} ), so I don't think we want to add them to the list of inactive nodes. Otherwise the node would be in the two node lists simultaneously, and that's probably not desireable. Specifically we'd want to remove this code insertion from the patch:
{code}
@@ -394,6 +411,8 @@ public class RMNodeImpl implements RMNode, EventHandler<RMNodeEvent> {
// Inform the scheduler
rmNode.context.getDispatcher().getEventHandler().handle(
new NodeRemovedSchedulerEvent(rmNode));
+ rmNode.context.getInactiveRMNodes()
+ .put(rmNode.nodeId.getHost(), rmNode);
ClusterMetrics.getMetrics().incrNumUnhealthyNMs();
return RMNodeState.UNHEALTHY;
}
{code}
- A node that's marked UNHEALTHY could still have a working nodemanager web page, so we don't want to remove the link to it on the status page. Since the UNHEALTHY nodes are tracked in the normal node list, it's simplest to remove the UNHEALTHY case from the switch statement in NodesPages.java.
At some point unit tests need to be added/updated for this change (e.g.: updating TestNodesPage.java to verify nodes that transition into the LOST state appear on the LOST page, etc.)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180725#comment-13180725 ]
Hadoop QA commented on MAPREDUCE-3360:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509577/MAPREDUCE-3360-3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1545//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1545//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1545//console
This message is automatically generated.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3360:
-----------------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.1
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Release Note: Added information about lost/rebooted/decommissioned nodes on the webapps.
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I just committed this to trunk and 0.23. Thanks Kamesh! And Jason too for all the reviews and the last update!
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194252#comment-13194252 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1670 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1670/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236433
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173536#comment-13173536 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
{quote}
I think, there are only three events, which causes the NM to lost, and all the 3 events have been handled.
RUNNING -> UNHEALTY, and UNHEALTY -> RUNNING transitions have been handled in their corresponding transition hooks.
{quote}
For this comment I'm not worried about the way the decr() method is called with the patch but rather how it could be mishandled in the future. Someone could look through the interfaces, see the decr() method, and assume it will decrement the appropriate counter for the specified node state. That's not what this method does; it's only valid to call it with an inactive state, and if called inappropriately it silently does nothing. I'm thinking that's undesired behavior and also wondering if the method is necessary. There's only one place in the code where it's called, it seems unlikely to be called in other places legitimately, and the calling code can do this behavior directly. This isn't a showstopper issue for me, just a potential maintenance issue for the future.
bq. For inactive nodes, hyperlink appears as "N/A" however clicking on that does not go anywhere. Please once refer the attached screen shot.
I realized the "N/A" hyperlink doesn't fail when clicking it, but then why make it a hyperlink at all? It doesn't do anything useful. For example, we could do something like the following to only generate a hyperlink when the node is active:
{code:title=in NodesBlock.render() (NodesPage.java)}
TR<TBODY<TABLE<Hamlet>>> row = tbody.tr().
td(info.getRack()).
td(info.getState()).
td(info.getNodeId());
if (isInactive) {
row.td()._("N/A")._();
} else {
String httpAddress = info.getNodeHTTPAddress();
row.td().a("http://" + httpAddress, httpAddress)._();
}
row.td(info.getHealthStatus()).
td(Times.format(info.getLastHealthUpdate())).
td(info.getHealthReport()).
td(String.valueOf(info.getNumContainers())).
td().br().$title(String.valueOf(usedMemory))._().
_(StringUtils.byteDesc(usedMemory * BYTES_IN_MB))._().
td().br().$title(String.valueOf(usedMemory))._().
_(StringUtils.byteDesc(availableMemory * BYTES_IN_MB))._().
_();
{code}
again not a showstopper issue for me, just seemed like we could clean up some useless hyperlinks.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183906#comment-13183906 ]
Hadoop QA commented on MAPREDUCE-3360:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12510160/MAPREDUCE-3360-6.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 27 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1589//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1589//console
This message is automatically generated.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13183460#comment-13183460 ]
Bhallamudi Venkata Siva Kamesh commented on MAPREDUCE-3360:
-----------------------------------------------------------
Thanks Thomas for comments.
bq. minor nit: I would prefer the web services to output empty string rather then "N/A" for the nodeHttpAddress. It seems more programmatic and I tried to make that change consistent throughout.
If we change nodeHttpAddress to empty string in RMWebServices.java, then it conflicts with the way we have been displaying it in the UI, as we have been displaying it as "N/A" for inactive nodes. Or do we need to capture this behaviour some where in the documentation?
bq. I think we need to change ClusterMetricsInfo.java to match if this field is Active instead of Total. It has a field totalNodes which appears to really be active nodes. So I think we should add a new activeNodes field and then make totalNodes the true total nodes of the cluster -> active + rebooted + lost + decommissioned
After this change, if we provide *true totalNodes* info through web services, do we need to provide the same info on the UI also?.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-3360:
----------------------------------
Target Version/s: 0.23.1, 0.24.0 (was: 0.23.1)
Status: Patch Available (was: Open)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Attachment: lostNodes.png
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Attachments: lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194185#comment-13194185 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
Test failures are unrelated to the patch. See MAPREDUCE-3729.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe reassigned MAPREDUCE-3360:
-------------------------------------
Assignee: Bhallamudi Venkata Siva Kamesh
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Status: Patch Available (was: Open)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Attachment: MAPREDUCE-3360-2.patch
Attached patch, incorporating above comments. Please review.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe reassigned MAPREDUCE-3360:
-------------------------------------
Assignee: (was: Bhallamudi Venkata Siva Kamesh)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bh V S Kamesh
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Status: Patch Available (was: Open)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194257#comment-13194257 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #430 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/430/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
svn merge --ignore-ancestry -c 1236433 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236438
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194692#comment-13194692 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #173 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/173/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
svn merge --ignore-ancestry -c 1236433 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236438
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Attachment: MAPREDUCE-3360.patch
Attaching a patch merely for solution review.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Jonathan Eagles
> Priority: Blocker
> Attachments: MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Bhallamudi Venkata Siva Kamesh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-3360:
------------------------------------------------------
Target Version/s: 0.23.1, 0.24.0 (was: 0.24.0, 0.23.1)
Status: Open (was: Patch Available)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182573#comment-13182573 ]
Hadoop QA commented on MAPREDUCE-3360:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12509882/MAPREDUCE-3360-5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1568//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1568//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1568//console
This message is automatically generated.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Thomas Graves (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182656#comment-13182656 ]
Thomas Graves commented on MAPREDUCE-3360:
------------------------------------------
A couple of comments:
RMWebServices.java
- minor nit: I would prefer the web services to output empty string rather then "N/A" for the nodeHttpAddress. It seems more programmatic and I tried to make that change consistent throughout.
- I think you can use the nid.getHost() instead of doing the split on ":" to get the hostName.
MetricsOverviewTable.java
- I think we need to change ClusterMetricsInfo.java to match if this field is Active instead of Total. It has a field totalNodes which appears to really be active nodes. So I think we should add a new activeNodes field and then make totalNodes the true total nodes of the cluster -> active + rebooted + lost + decommissioned.
ClusterMetrics.java
- do we want to change the heading output to match active nodes?
@Metric("# of NMs") MutableGaugeInt numNMs;
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3360:
-------------------------------------
Priority: Major (was: Blocker)
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Attachments: MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181729#comment-13181729 ]
Jason Lowe commented on MAPREDUCE-3360:
---------------------------------------
Thanks for adding the web services support. A few issues I noticed:
1) Copy-n-paste error on line 234 of TestRMWebServicesNodes.java in testNodesQueryStateLost():
{code}
WebServicesTestUtils.checkStringMatch("nodeHTTPAddress", rmNode
.getState().toString(), info.getString("state"));
{code}
{{"nodeHTTPAddress"}} should be {{"state"}}. This also occurs on line 264.
2) In testNodesQueryStateLost() the test is assuming that the order of nodes returned is deterministic, but I'm not sure that's guaranteed by the API. In addition, it simply checks if the node ID is what it's looking for, and if it isn't, it silently allows the test to pass. Since both nodes should be lost at this point in the test, we could just loop over the returned nodes and verify we see the fields we expect for each node entry.
3) In testSingleNodeQueryStateLost() the code similarly checks for the expected node ID. If it doesn't match it should fail.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194261#comment-13194261 ]
Hudson commented on MAPREDUCE-3360:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1614 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1614/])
MAPREDUCE-3360. Added information about lost/rebooted/decommissioned nodes on the webapps. Contributed by Bhallamudi Venkata Siva Kamesh and Jason Lowe.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236433
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/NodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ClusterMetricsInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/NodeInfo.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestNodesPage.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3360) Provide information about lost
nodes in the UI.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194177#comment-13194177 ]
Hadoop QA commented on MAPREDUCE-3360:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12512033/MAPREDUCE-3360.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 27 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.mapred.TestJobClientGetJob
org.apache.hadoop.mapred.TestMRWithDistributedCache
org.apache.hadoop.mapred.TestLocalModeWithNewApis
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1690//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1690//console
This message is automatically generated.
> Provide information about lost nodes in the UI.
> -----------------------------------------------
>
> Key: MAPREDUCE-3360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Environment: NA
> Reporter: Bhallamudi Venkata Siva Kamesh
> Assignee: Bhallamudi Venkata Siva Kamesh
> Priority: Critical
> Attachments: LostNodes.png, MAPREDUCE-3360-1.patch, MAPREDUCE-3360-2.patch, MAPREDUCE-3360-3.patch, MAPREDUCE-3360-4.patch, MAPREDUCE-3360-5.patch, MAPREDUCE-3360-6.patch, MAPREDUCE-3360.patch, MAPREDUCE-3360.patch, lostNodes.png
>
>
> Currently there is no information provided about *lost nodes*. Provide information in the UI.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira