You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by ta...@apache.org on 2021/04/28 09:53:46 UTC
[hadoop] branch branch-3.3 updated (187b4a6 -> 0e9042f)
This is an automated email from the ASF dual-hosted git repository.
tasanuma pushed a change to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from 187b4a6 HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina.
new 3c7433f HDFS-15247. RBF: Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
new f809b86 HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.
new 842e05b HDFS-15698. Fix the typo of dfshealth.html after HDFS-15358 (#2495)
new 0e9042f HDFS-15991. Add location into datanode info for NameNodeMXBean (#2933)
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../src/main/webapps/router/federationhealth.html | 74 +++++++++++--
.../src/main/webapps/router/federationhealth.js | 120 +++++++++++++++++++--
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 4 +
.../src/main/webapps/hdfs/dfshealth.html | 12 ++-
.../hdfs/server/namenode/TestNameNodeMXBean.java | 25 ++++-
5 files changed, 215 insertions(+), 20 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/04: HDFS-15247. RBF: Provide Non DFS Used per DataNode
in DataNode UI. Contributed by Lisheng Sun.
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tasanuma pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 3c7433f2a10cfb308314befd2d3ee4c37a0ff7a3
Author: Ayush Saxena <ay...@apache.org>
AuthorDate: Sun Apr 12 12:05:50 2020 +0530
HDFS-15247. RBF: Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
(cherry picked from commit 8d49229c3764a205f21750225005a2c9a8124ab9)
---
.../hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html | 4 ++++
.../hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js | 2 ++
2 files changed, 6 insertions(+)
diff --git a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
index 66c0309..534dd95 100644
--- a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
+++ b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
@@ -326,6 +326,8 @@
<tr>
<th>Node</th>
<th>Last contact</th>
+ <th>Used</th>
+ <th>Non DFS Used</th>
<th style="width:180px; text-align:center">Capacity</th>
<!--th>Blocks</th-->
<th>Block pool used</th>
@@ -336,6 +338,8 @@
<tr>
<td ng-value="{state}-{name}" class="dfshealth-node-icon dfshealth-node-{state}">{location}/{name} ({xferaddr})</td>
<td ng-value="{lastContact}">{#helper_relative_time value="{lastContact}"/}</td>
+ <td ng-value="{used}">{used|fmt_bytes}</td>
+ <td ng-value="{nonDfsUsedSpace}">{nonDfsUsedSpace|fmt_bytes}</td>
<td ng-value="{usedPercentage}" style="width:210px">
<div>
<div style="display:inline-block; float: left; padding-right: 10px; width:60px;">{capacity|fmt_bytes}</div>
diff --git a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js
index 94ba67c..b7b2ba3 100644
--- a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js
+++ b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js
@@ -319,6 +319,8 @@
{ 'orderDataType': 'ng-value', 'searchable': true },
{ 'orderDataType': 'ng-value', 'type': 'numeric' },
{ 'orderDataType': 'ng-value', 'type': 'numeric' },
+ { 'orderDataType': 'ng-value', 'type': 'numeric' },
+ { 'orderDataType': 'ng-value', 'type': 'numeric' },
{ 'orderDataType': 'ng-value', 'type': 'numeric'}
]});
$('#ui-tabs a[href="#tab-datanode"]').tab('show');
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 03/04: HDFS-15698. Fix the typo of dfshealth.html after
HDFS-15358 (#2495)
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tasanuma pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 842e05ba53465de0e5ca3d04c77e30f0effe996b
Author: Hui Fei <fe...@apache.org>
AuthorDate: Sat Nov 28 12:19:57 2020 +0800
HDFS-15698. Fix the typo of dfshealth.html after HDFS-15358 (#2495)
(cherry picked from commit 68442b48c8a044b1ead89e22003b6db4d300bfad)
---
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html b/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
index bc062a5d..6eeac85 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
@@ -373,7 +373,7 @@
<td></td>
<td></td>
<td></td>
- <td></td>>
+ <td></td>
</tr>
{/DeadNodes}
</table>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 04/04: HDFS-15991. Add location into datanode info for
NameNodeMXBean (#2933)
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tasanuma pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 0e9042fed587bb327a069d643986d5ac7d9c0c8b
Author: litao <to...@gmail.com>
AuthorDate: Mon Apr 26 17:38:43 2021 +0800
HDFS-15991. Add location into datanode info for NameNodeMXBean (#2933)
Signed-off-by: Takanobu Asanuma <ta...@apache.org>
(cherry picked from commit b968fa0957ddd0f59c9adfd068507722db7207c6)
---
.../src/main/webapps/router/federationhealth.html | 2 +-
.../hadoop/hdfs/server/namenode/FSNamesystem.java | 4 ++++
.../src/main/webapps/hdfs/dfshealth.html | 10 ++++-----
.../hdfs/server/namenode/TestNameNodeMXBean.java | 25 ++++++++++++++++++++--
4 files changed, 33 insertions(+), 8 deletions(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
index a9afe40..389491c 100644
--- a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
+++ b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
@@ -406,7 +406,7 @@
</thead>
{#EnteringMaintenanceNodes}
<tr>
- <td>{name} ({xferaddr})</td>
+ <td>{location}/{name} ({xferaddr})</td>
<td>{underReplicatedBlocks}</td>
<td>{maintenanceOnlyReplicas}</td>
<td>{underReplicateInOpenFiles}</td>
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
index ba52441..a7c4312 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
@@ -6408,6 +6408,7 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
.put("infoAddr", node.getInfoAddr())
.put("infoSecureAddr", node.getInfoSecureAddr())
.put("xferaddr", node.getXferAddr())
+ .put("location", node.getNetworkLocation())
.put("lastContact", getLastContact(node))
.put("usedSpace", getDfsUsed(node))
.put("adminState", node.getAdminState().toString())
@@ -6457,6 +6458,7 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
.put("decommissioned", node.isDecommissioned())
.put("adminState", node.getAdminState().toString())
.put("xferaddr", node.getXferAddr())
+ .put("location", node.getNetworkLocation())
.build();
info.put(node.getHostName() + ":" + node.getXferPort(), innerinfo);
}
@@ -6478,6 +6480,7 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
Map<String, Object> innerinfo = ImmutableMap
.<String, Object> builder()
.put("xferaddr", node.getXferAddr())
+ .put("location", node.getNetworkLocation())
.put("underReplicatedBlocks",
node.getLeavingServiceStatus().getUnderReplicatedBlocks())
.put("decommissionOnlyReplicas",
@@ -6505,6 +6508,7 @@ public class FSNamesystem implements Namesystem, FSNamesystemMBean,
Map<String, Object> attrMap = ImmutableMap
.<String, Object> builder()
.put("xferaddr", node.getXferAddr())
+ .put("location", node.getNetworkLocation())
.put("underReplicatedBlocks",
node.getLeavingServiceStatus().getUnderReplicatedBlocks())
.put("maintenanceOnlyReplicas",
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html b/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
index 6eeac85..0751785 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
@@ -341,7 +341,7 @@
{#LiveNodes}
<tr>
<td ng-value="{state}">{state}</td>
- <td ng-value="{state}-{name}" class="dfshealth-node-icon dfshealth-node-{state}">{name} ({xferaddr})</td>
+ <td ng-value="{state}-{name}" class="dfshealth-node-icon dfshealth-node-{state}">{location}/{name} ({xferaddr})</td>
<td ng-value="{state}-{name}"><a href='{dnWebAddress}'>{dnWebAddress}</a></td>
<td ng-value="{lastContact}">{lastContact}s</td>
<td ng-value="{lastBlockReport}">{lastBlockReport}m</td>
@@ -364,7 +364,7 @@
{#DeadNodes}
<tr class="danger">
<td ng-value="{state}">{state}</td>
- <td ng-value="{state}-{name}" class="dfshealth-node-icon dfshealth-node-{state}">{name} ({xferaddr})</td>
+ <td ng-value="{state}-{name}" class="dfshealth-node-icon dfshealth-node-{state}">{location}/{name} ({xferaddr})</td>
<td></td>
<td>{#helper_relative_time value="{lastContact}"/}</td>
<td></td>
@@ -393,7 +393,7 @@
</thead>
{#EnteringMaintenanceNodes}
<tr>
- <td>{name} ({xferaddr})</td>
+ <td>{location}/{name} ({xferaddr})</td>
<td>{underReplicatedBlocks}</td>
<td>{maintenanceOnlyReplicas}</td>
<td>{underReplicateInOpenFiles}</td>
@@ -419,7 +419,7 @@
</thead>
{#DecomNodes}
<tr>
- <td>{name} ({xferaddr})</td>
+ <td>{location}/{name} ({xferaddr})</td>
<td>{underReplicatedBlocks}</td>
<td>{decommissionOnlyReplicas}</td>
<td>{underReplicateInOpenFiles}</td>
@@ -448,7 +448,7 @@ No nodes are decommissioning.
</thead>
{#LiveNodes}
<tr>
- <td>{name} ({xferaddr})</td>
+ <td>{location}/{name} ({xferaddr})</td>
<td>{#helper_date_tostring value="{lastVolumeFailureDate}"/}</td>
<td>{volfails}</td>
<td>{estimatedCapacityLostTotal|fmt_bytes}</td>
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
index a309e32..81c9cb8 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
@@ -112,7 +112,7 @@ public class TestNameNodeMXBean {
MiniDFSCluster cluster = null;
try {
- cluster = new MiniDFSCluster.Builder(conf).numDataNodes(2).build();
+ cluster = new MiniDFSCluster.Builder(conf).numDataNodes(4).build();
cluster.waitActive();
// Set upgrade domain on the first DN.
@@ -171,7 +171,7 @@ public class TestNameNodeMXBean {
"LiveNodes"));
Map<String, Map<String, Object>> liveNodes =
(Map<String, Map<String, Object>>) JSON.parse(alivenodeinfo);
- assertTrue(liveNodes.size() == 2);
+ assertTrue(liveNodes.size() == 4);
for (Map<String, Object> liveNode : liveNodes.values()) {
assertTrue(liveNode.containsKey("nonDfsUsedSpace"));
assertTrue(((Long)liveNode.get("nonDfsUsedSpace")) >= 0);
@@ -195,6 +195,27 @@ public class TestNameNodeMXBean {
assertFalse(xferAddr.equals(dnXferAddrInMaintenance) ^ inMaintenance);
}
assertEquals(fsn.getLiveNodes(), alivenodeinfo);
+
+ // Put the third DN to decommissioning state.
+ DatanodeDescriptor decommissioningNode = dm.getDatanode(
+ cluster.getDataNodes().get(2).getDatanodeId());
+ decommissioningNode.startDecommission();
+
+ // Put the fourth DN to decommissioned state.
+ DatanodeDescriptor decommissionedNode = dm.getDatanode(
+ cluster.getDataNodes().get(3).getDatanodeId());
+ decommissionedNode.setDecommissioned();
+
+ // Assert the location field is included in the mxbeanName
+ // under different states
+ String alivenodeinfo1 = (String) (mbs.getAttribute(mxbeanName,
+ "LiveNodes"));
+ Map<String, Map<String, Object>> liveNodes1 =
+ (Map<String, Map<String, Object>>) JSON.parse(alivenodeinfo1);
+ for (Map<String, Object> liveNode : liveNodes1.values()) {
+ assertTrue(liveNode.containsKey("location"));
+ }
+
// get attributes DeadNodes
String deadNodeInfo = (String) (mbs.getAttribute(mxbeanName,
"DeadNodes"));
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/04: HDFS-15358. RBF: Unify router datanode UI with
namenode datanode UI. Contributed by Ayush Saxena.
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tasanuma pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit f809b863fefb16e731fe97c49c3356be0e0bd5c4
Author: Ayush Saxena <ay...@apache.org>
AuthorDate: Sun May 17 03:06:27 2020 +0530
HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.
(cherry picked from commit 6e416a83d1e674ecd018d1db74a2d88e738deb40)
---
.../src/main/webapps/router/federationhealth.html | 70 ++++++++++--
.../src/main/webapps/router/federationhealth.js | 122 +++++++++++++++++++--
.../src/main/webapps/hdfs/dfshealth.html | 2 +
3 files changed, 179 insertions(+), 15 deletions(-)
diff --git a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
index 534dd95..a9afe40 100644
--- a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
+++ b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html
@@ -23,6 +23,11 @@
<link rel="stylesheet" type="text/css" href="static/dataTables.bootstrap.css" />
<link rel="stylesheet" type="text/css" href="static/rbf.css" />
<title>Router Information</title>
+<style>
+ .col-sm-6 {
+ width: 30%;
+ }
+</style>
</head>
<body>
@@ -315,29 +320,44 @@
<ul class="dfshealth-node-legend">
<li class="dfshealth-node-icon dfshealth-node-alive">In service</li>
<li class="dfshealth-node-icon dfshealth-node-down">Down</li>
+ <li class="dfshealth-node-icon dfshealth-node-decommissioning">Decommissioning</li>
<li class="dfshealth-node-icon dfshealth-node-decommissioned">Decommissioned</li>
<li class="dfshealth-node-icon dfshealth-node-down-decommissioned">Decommissioned & dead</li>
</ul>
+ <ul class="dfshealth-node-legend">
+ <li class="dfshealth-node-icon dfshealth-node-entering-maintenance">Entering Maintenance</li>
+ <li class="dfshealth-node-icon dfshealth-node-in-maintenance">In Maintenance</li>
+ <li class="dfshealth-node-icon dfshealth-node-down-maintenance">In Maintenance & dead</li>
+ </ul>
</div>
+<div class="page-header"><h1><small>Datanode usage histogram</small></h1></div>
+<small><div id="datanode-usage-histogram"></div></small>
<div class="page-header"><h1><small>In operation</small></h1></div>
<small>
+<p id="datanodefilter" class="col-sm-6">DataNode State </p>
<table class="table" id="table-datanodes">
<thead>
<tr>
+ <th>State</th>
<th>Node</th>
+ <th>Http Address</th>
<th>Last contact</th>
+ <th>Last Block Report</th>
<th>Used</th>
<th>Non DFS Used</th>
<th style="width:180px; text-align:center">Capacity</th>
- <!--th>Blocks</th-->
+ <th>Blocks</th-->
<th>Block pool used</th>
- <!--th>Version</th-->
+ <th>Version</th>
</tr>
</thead>
{#LiveNodes}
<tr>
+ <td ng-value="{state}">{state}</td>
<td ng-value="{state}-{name}" class="dfshealth-node-icon dfshealth-node-{state}">{location}/{name} ({xferaddr})</td>
- <td ng-value="{lastContact}">{#helper_relative_time value="{lastContact}"/}</td>
+ <td ng-value="{state}-{name}"><a href='{dnWebAddress}'>{dnWebAddress}</a></td>
+ <td ng-value="{lastContact}">{lastContact}s</td>
+ <td ng-value="{lastBlockReport}">{lastBlockReport}m</td>
<td ng-value="{used}">{used|fmt_bytes}</td>
<td ng-value="{nonDfsUsedSpace}">{nonDfsUsedSpace|fmt_bytes}</td>
<td ng-value="{usedPercentage}" style="width:210px">
@@ -349,26 +369,58 @@
</div>
</div>
</td>
- <!--td>{numBlocks}</td-->
+ <td>{numBlocks}</td>
<td ng-value="{blockPoolUsedPercent}">{blockPoolUsed|fmt_bytes} ({blockPoolUsedPercent|fmt_percentage})</td>
- <!--td>{version}</td-->
+ <td>{version}</td>
</tr>
{/LiveNodes}
{#DeadNodes}
<tr class="danger">
+ <td ng-value="{state}">{state}</td>
<td ng-value="{state}-{name}" class="dfshealth-node-icon dfshealth-node-{state}">{location}/{name} ({xferaddr})</td>
+ <td></td>
<td>{#helper_relative_time value="{lastContact}"/}</td>
<td></td>
- <!--td></td-->
<td></td>
- <!--td></td-->
+ <td></td>
+ <td></td>
+ <td></td>
+ <td></td>
+ <td></td>
</tr>
{/DeadNodes}
</table>
</small>
+<div class="page-header"><h1><small>Entering Maintenance</small></h1></div>
+<small>
+ {?EnteringMaintenanceNodes}
+ <table class="table">
+ <thead>
+ <tr>
+ <th>Node</th>
+ <th>Under replicated blocks</th>
+ <th>Blocks with no live replicas</th>
+ <th>Under Replicated Blocks <br/>In files under construction</th>
+ </tr>
+ </thead>
+ {#EnteringMaintenanceNodes}
+ <tr>
+ <td>{name} ({xferaddr})</td>
+ <td>{underReplicatedBlocks}</td>
+ <td>{maintenanceOnlyReplicas}</td>
+ <td>{underReplicateInOpenFiles}</td>
+ </tr>
+ {/EnteringMaintenanceNodes}
+ </table>
+ {:else}
+ No nodes are entering maintenance.
+ {/EnteringMaintenanceNodes}
+</small>
+
<div class="page-header"><h1><small>Decommissioning</small></h1></div>
<small>
+{?DecomNodes}
<table class="table">
<thead>
<tr>
@@ -387,6 +439,9 @@
</tr>
{/DecomNodes}
</table>
+{:else}
+No nodes are decommissioning.
+{/DecomNodes}
</small>
</script>
@@ -449,6 +504,7 @@
<script type="text/javascript" src="static/dust-full-2.0.0.min.js"></script>
<script type="text/javascript" src="static/dust-helpers-1.1.1.min.js"></script>
<script type="text/javascript" src="static/dfs-dust.js"></script>
+<script type="text/javascript" src="/static/d3-v4.1.1.min.js"></script>
<script type="text/javascript" src="federationhealth.js"></script>
</body>
</html>
diff --git a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js
index b7b2ba3..86eda24 100644
--- a/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js
+++ b/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js
@@ -278,20 +278,34 @@
for (var i = 0, e = nodes.length; i < e; ++i) {
var n = nodes[i];
n.usedPercentage = Math.round((n.used + n.nonDfsUsedSpace) * 1.0 / n.capacity * 100);
+ var port = n.infoAddr.split(":")[1];
+ var securePort = n.infoSecureAddr.split(":")[1];
+ var dnHost = n.name.split(":")[0];
+ n.dnWebAddress = "http://" + dnHost + ":" + port;
+ if (securePort != 0) {
+ n.dnWebAddress = "https://" + dnHost + ":" + securePort;
+ }
+
if (n.adminState === "In Service") {
n.state = "alive";
} else if (nodes[i].adminState === "Decommission In Progress") {
n.state = "decommissioning";
} else if (nodes[i].adminState === "Decommissioned") {
n.state = "decommissioned";
+ } else if (nodes[i].adminState === "Entering Maintenance") {
+ n.state = "entering-maintenance";
+ } else if (nodes[i].adminState === "In Maintenance") {
+ n.state = "in-maintenance";
}
}
}
function augment_dead_nodes(nodes) {
for (var i = 0, e = nodes.length; i < e; ++i) {
- if (nodes[i].decommissioned) {
+ if (nodes[i].adminState === "Decommissioned") {
nodes[i].state = "down-decommissioned";
+ } else if (nodes[i].adminState === "In Maintenance") {
+ nodes[i].state = "down-maintenance";
} else {
nodes[i].state = "down";
}
@@ -303,9 +317,77 @@
r.DeadNodes = node_map_to_array(JSON.parse(r.DeadNodes));
augment_dead_nodes(r.DeadNodes);
r.DecomNodes = node_map_to_array(JSON.parse(r.DecomNodes));
+ r.EnteringMaintenanceNodes = node_map_to_array(JSON.parse(r.EnteringMaintenanceNodes));
return r;
}
+ function renderHistogram(dnData) {
+ var data = dnData.LiveNodes.map(function(dn) {
+ return (dn.usedSpace / dn.capacity) * 100.0;
+ });
+
+ var formatCount = d3.format(",.0f");
+
+ var widthCap = $("div.container").width();
+ var heightCap = 150;
+
+ var margin = {top: 10, right: 60, bottom: 30, left: 30},
+ width = widthCap * 0.9,
+ height = heightCap - margin.top - margin.bottom;
+
+ var x = d3.scaleLinear()
+ .domain([0.0, 100.0])
+ .range([0, width]);
+
+ var bins = d3.histogram()
+ .domain(x.domain())
+ .thresholds(x.ticks(20))
+ (data);
+
+ var y = d3.scaleLinear()
+ .domain([0, d3.max(bins, function(d) { return d.length; })])
+ .range([height, 0]);
+
+ var svg = d3.select("#datanode-usage-histogram").append("svg")
+ .attr("width", width + 50.0)
+ .attr("height", height + margin.top + margin.bottom)
+ .append("g")
+ .attr("transform", "translate(" + margin.left + "," + margin.top + ")");
+
+ svg.append("text")
+ .attr("x", (width / 2))
+ .attr("y", heightCap - 6 - (margin.top / 2))
+ .attr("text-anchor", "middle")
+ .style("font-size", "15px")
+ .text("Disk usage of each DataNode (%)");
+
+ var bar = svg.selectAll(".bar")
+ .data(bins)
+ .enter().append("g")
+ .attr("class", "bar")
+ .attr("transform", function(d) { return "translate(" + x(d.x0) + "," + y(d.length) + ")"; });
+
+ window.liveNodes = dnData.LiveNodes;
+
+ bar.append("rect")
+ .attr("x", 1)
+ .attr("width", x(bins[0].x1) - x(bins[0].x0) - 1)
+ .attr("height", function(d) { return height - y(d.length); })
+ .attr("onclick", function (d) { return "open_hostip_list(" + d.x0 + "," + d.x1 + ")"; });
+
+ bar.append("text")
+ .attr("dy", ".75em")
+ .attr("y", 6)
+ .attr("x", (x(bins[0].x1) - x(bins[0].x0)) / 2)
+ .attr("text-anchor", "middle")
+ .text(function(d) { return formatCount(d.length); });
+
+ svg.append("g")
+ .attr("class", "axis axis--x")
+ .attr("transform", "translate(0," + height + ")")
+ .call(d3.axisBottom(x));
+ }
+
$.get(
'jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo',
guard_with_startup_progress(function (resp) {
@@ -315,14 +397,38 @@
$('#tab-datanode').html(out);
$('#table-datanodes').dataTable( {
'lengthMenu': [ [25, 50, 100, -1], [25, 50, 100, "All"] ],
+ 'columnDefs': [
+ { 'targets': [ 0 ], 'visible': false, 'searchable': false }
+ ],
'columns': [
- { 'orderDataType': 'ng-value', 'searchable': true },
- { 'orderDataType': 'ng-value', 'type': 'numeric' },
- { 'orderDataType': 'ng-value', 'type': 'numeric' },
- { 'orderDataType': 'ng-value', 'type': 'numeric' },
- { 'orderDataType': 'ng-value', 'type': 'numeric' },
- { 'orderDataType': 'ng-value', 'type': 'numeric'}
- ]});
+ { 'orderDataType': 'ng-value', 'searchable': true , "defaultContent": "" },
+ { 'orderDataType': 'ng-value', 'searchable': true , "defaultContent": "" },
+ { 'orderDataType': 'ng-value', 'searchable': true , "defaultContent": ""},
+ { 'orderDataType': 'ng-value', 'type': 'num' , "defaultContent": 0},
+ { 'orderDataType': 'ng-value', 'type': 'num' , "defaultContent": 0},
+ { 'orderDataType': 'ng-value', 'type': 'num' , "defaultContent": 0},
+ { 'orderDataType': 'ng-value', 'type': 'num' , "defaultContent": 0},
+ { 'orderDataType': 'ng-value', 'type': 'num' , "defaultContent": 0},
+ { 'type': 'num' , "defaultContent": 0},
+ { 'orderDataType': 'ng-value', 'type': 'num' , "defaultContent": 0},
+ { 'type': 'string' , "defaultContent": ""}
+ ],
+ initComplete: function () {
+ var column = this.api().column([0]);
+ var select = $('<select class="datanodestatus form-control input-sm"><option value="">All</option></select>')
+ .appendTo('#datanodefilter')
+ .on('change', function () {
+ var val = $.fn.dataTable.util.escapeRegex(
+ $(this).val());
+ column.search(val ? '^' + val + '$' : '', true, false).draw();
+ });
+ console.log(select);
+ column.data().unique().sort().each(function (d, j) {
+ select.append('<option value="' + d + '">' + d + '</option>');
+ });
+ }
+ });
+ renderHistogram(data);
$('#ui-tabs a[href="#tab-datanode"]').tab('show');
});
})).fail(ajax_error_handler);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html b/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
index ec4cb5a..bc062a5d 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
@@ -372,6 +372,8 @@
<td></td>
<td></td>
<td></td>
+ <td></td>
+ <td></td>>
</tr>
{/DeadNodes}
</table>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org