You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/09/22 13:01:45 UTC
[GitHub] [hadoop] slfan1989 opened a new pull request, #4924: YARN-11310. [Yarn Federation] Refactoring Router's Federation Web Page.
slfan1989 opened a new pull request, #4924:
URL: https://github.com/apache/hadoop/pull/4924
JIRA: YARN-11310. [Yarn Federation] Refactoring Router's Federation Web Page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r977908197
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
Review Comment:
Where did all this go?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
Review Comment:
This was fine as it was.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
+ Hamlet.DIV<Hamlet> div = html.div("#div_id");
+ div.p().$style("color:red").__("Federation is not Enabled.").__()
+ .p().__()
+ .p().__("We can refer to the following documents to configure Yarn Federation. ").__()
+ .p().$style("color:blue").__()
+ .a("https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/Federation.html",
+ "Hadoop: YARN Federation").
+ __();
}
}
private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ private static void initFederationBlockTableJs(Block html, List<Map<String, String>> jsonMap) {
+ Gson gson =new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
+ " var appsTableData = " + gson.toJson(jsonMap) + "; " +
+ " var table = $('#rms').DataTable(); " +
+ " $('#rms tbody').on('click', 'td.details-control', function () { " +
+ " var tr = $(this).closest('tr'); " +
+ " var row = table.row(tr); " +
+ " if (row.child.isShown()) { " +
+ " row.child.hide(); " +
+ " tr.removeClass('shown'); " +
+ " } else { " +
+ " console.log(row.id());\n " +
+ " var capabilityArr = appsTableData.filter(item=>(item.subcluster === row.id())); " +
+ " var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics; " +
+ " row.child(" +
+ " '<table>" +
+ " <tr>" +
+ " <td>" +
+ " <h3>Application Metrics</h3>" +
+ " appsSubmitted : '+capabilityObj.appsSubmitted+' </p>" +
+ " appsCompleted : '+capabilityObj.appsCompleted+' </p>" +
+ " appsPending : '+capabilityObj.appsPending+' </p>" +
+ " appsRunning : '+capabilityObj.appsRunning+' </p>" +
+ " appsFailed : '+capabilityObj.appsFailed+' </p> " +
+ " appsKilled : '+capabilityObj.appsKilled+' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Resource Metrics</h3>" +
+ " <h4>Memory</h4>" +
+ " totalMB : '+capabilityObj.totalMB+' </p>" +
Review Comment:
Spacing
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
Review Comment:
initFederationBlockTableJs?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationPage.java:
##########
@@ -47,10 +48,10 @@ protected Class<? extends SubView> content() {
private String rmsTableInit() {
StringBuilder b = tableInit().append(", aoColumnDefs: [");
- b.append("{'bSearchable': false, 'aTargets': [ 7 ]}")
+ b.append("{'bSearchable': false, 'aTargets': [ 2 ]}")
Review Comment:
As we are at it, can we make this numbers variables with names?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
+ Hamlet.DIV<Hamlet> div = html.div("#div_id");
+ div.p().$style("color:red").__("Federation is not Enabled.").__()
+ .p().__()
+ .p().__("We can refer to the following documents to configure Yarn Federation. ").__()
+ .p().$style("color:blue").__()
+ .a("https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/Federation.html",
+ "Hadoop: YARN Federation").
+ __();
}
}
private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ private static void initFederationBlockTableJs(Block html, List<Map<String, String>> jsonMap) {
Review Comment:
Add javadoc explaining what table this sets.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
Review Comment:
It might be cleaner to have the `if (!isEnabled)` first and return after this.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
+ Hamlet.DIV<Hamlet> div = html.div("#div_id");
+ div.p().$style("color:red").__("Federation is not Enabled.").__()
+ .p().__()
+ .p().__("We can refer to the following documents to configure Yarn Federation. ").__()
+ .p().$style("color:blue").__()
+ .a("https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/Federation.html",
+ "Hadoop: YARN Federation").
+ __();
}
}
private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ private static void initFederationBlockTableJs(Block html, List<Map<String, String>> jsonMap) {
+ Gson gson =new Gson();
Review Comment:
Space
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1258874360
If the user does not configure Yarn Federation, we will give the user some help information.
The page is as follows:
<img width="1675" alt="image" src="https://user-images.githubusercontent.com/55643692/192415502-9301306a-ebb3-46fd-a68c-89405f1b253b.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1260032481
@goiri Thank you very much for your help reviewing the code!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r978208379
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
+ Hamlet.DIV<Hamlet> div = html.div("#div_id");
+ div.p().$style("color:red").__("Federation is not Enabled.").__()
+ .p().__()
+ .p().__("We can refer to the following documents to configure Yarn Federation. ").__()
+ .p().$style("color:blue").__()
+ .a("https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/Federation.html",
+ "Hadoop: YARN Federation").
+ __();
}
}
private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ private static void initFederationBlockTableJs(Block html, List<Map<String, String>> jsonMap) {
+ Gson gson =new Gson();
Review Comment:
I'll fix it.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
+ Hamlet.DIV<Hamlet> div = html.div("#div_id");
+ div.p().$style("color:red").__("Federation is not Enabled.").__()
+ .p().__()
+ .p().__("We can refer to the following documents to configure Yarn Federation. ").__()
+ .p().$style("color:blue").__()
+ .a("https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/Federation.html",
+ "Hadoop: YARN Federation").
+ __();
}
}
private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ private static void initFederationBlockTableJs(Block html, List<Map<String, String>> jsonMap) {
+ Gson gson =new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
+ " var appsTableData = " + gson.toJson(jsonMap) + "; " +
+ " var table = $('#rms').DataTable(); " +
+ " $('#rms tbody').on('click', 'td.details-control', function () { " +
+ " var tr = $(this).closest('tr'); " +
+ " var row = table.row(tr); " +
+ " if (row.child.isShown()) { " +
+ " row.child.hide(); " +
+ " tr.removeClass('shown'); " +
+ " } else { " +
+ " console.log(row.id());\n " +
+ " var capabilityArr = appsTableData.filter(item=>(item.subcluster === row.id())); " +
+ " var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics; " +
+ " row.child(" +
+ " '<table>" +
+ " <tr>" +
+ " <td>" +
+ " <h3>Application Metrics</h3>" +
+ " appsSubmitted : '+capabilityObj.appsSubmitted+' </p>" +
+ " appsCompleted : '+capabilityObj.appsCompleted+' </p>" +
+ " appsPending : '+capabilityObj.appsPending+' </p>" +
+ " appsRunning : '+capabilityObj.appsRunning+' </p>" +
+ " appsFailed : '+capabilityObj.appsFailed+' </p> " +
+ " appsKilled : '+capabilityObj.appsKilled+' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Resource Metrics</h3>" +
+ " <h4>Memory</h4>" +
+ " totalMB : '+capabilityObj.totalMB+' </p>" +
Review Comment:
I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r979134048
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
+ initHtmlPageFederationNotEnabled(html);
}
}
- private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
+ /**
+ * Parse the capability and obtain the metric information of the cluster.
+ *
+ * @param capability metric json obtained from RM.
+ * @return ClusterMetricsInfo Object
+ */
+ private ClusterMetricsInfo getClusterMetricsInfo(String capability) {
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ /**
+ * Initialize the subCluster details JavaScript of the Federation page.
+ *
+ * This part of the js script will control to display or hide the detailed information
+ * of the subCluster when the user clicks on the subClusterId.
+ *
+ * We will obtain the specific information of a SubCluster,
+ * including the information of Applications, Resources, and Nodes.
+ *
+ * @param html html object
+ * @param subClusterDetailMap subCluster Detail Map
+ */
+ private void initFederationSubClusterDetailTableJs(Block html,
+ List<Map<String, String>> subClusterDetailMap) {
+ Gson gson = new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
+ " var scTableData = " + gson.toJson(subClusterDetailMap) + "; " +
+ " var table = $('#rms').DataTable(); " +
+ " $('#rms tbody').on('click', 'td.details-control', function () { " +
+ " var tr = $(this).closest('tr'); " +
+ " var row = table.row(tr); " +
+ " if (row.child.isShown()) { " +
+ " row.child.hide(); " +
+ " tr.removeClass('shown'); " +
+ " } else { " +
+ " var capabilityArr = scTableData.filter(item => (item.subcluster === row.id())); " +
+ " var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics; " +
+ " row.child(" +
+ " '<table>" +
+ " <tr>" +
+ " <td>" +
+ " <h3>Application Metrics</h3>" +
+ " ApplicationSubmitted* : '+ capabilityObj.appsSubmitted +' </p>" +
+ " ApplicationCompleted* : '+ capabilityObj.appsCompleted +' </p>" +
+ " ApplicationPending* : '+ capabilityObj.appsPending +' </p>" +
+ " ApplicationRunning* : '+ capabilityObj.appsRunning +' </p>" +
+ " ApplicationFailed* : '+ capabilityObj.appsFailed +' </p> " +
+ " ApplicationKilled* : '+ capabilityObj.appsKilled +' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Resource Metrics</h3>" +
+ " <h4>Memory</h4>" +
+ " TotalMB : '+ capabilityObj.totalMB +' </p>" +
+ " ReservedMB : '+ capabilityObj.reservedMB +' </p>" +
+ " AvailableMB : '+ capabilityObj.availableMB +' </p>" +
+ " AllocatedMB : '+ capabilityObj.allocatedMB +' </p>" +
+ " PendingMB : '+ capabilityObj.pendingMB +' </p>" +
+ " <h4>VirtualCores</h4>" +
+ " TotalVirtualCores : '+capabilityObj.totalVirtualCores+' </p>" +
+ " ReservedVirtualCores : '+capabilityObj.reservedVirtualCores+' </p>" +
+ " AvailableVirtualCore : '+capabilityObj.availableVirtualCores+' </p>" +
+ " AllocatedVirtualCores : '+capabilityObj.allocatedVirtualCores+' </p>" +
+ " PendingVirtualCores : '+capabilityObj.pendingVirtualCores+' </p>" +
+ " <h4>Containers</h4>" +
+ " ContainersAllocated : '+capabilityObj.containersAllocated+' </p>" +
+ " ContainersReserved : '+capabilityObj.containersReserved+' </p>" +
+ " ContainersPending : '+capabilityObj.containersPending+' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Node Metrics</h3>" +
+ " TotalNodes : '+capabilityObj.totalNodes+' </p>" +
+ " LostNodes : '+capabilityObj.lostNodes+' </p>" +
+ " UnhealthyNodes : '+capabilityObj.unhealthyNodes+' </p>" +
+ " DecommissioningNodes : '+capabilityObj.decommissioningNodes+' </p>" +
+ " DecommissionedNodes : '+capabilityObj.decommissionedNodes+' </p>" +
+ " RebootedNodes : '+capabilityObj.rebootedNodes+' </p>" +
+ " ActiveNodes : '+capabilityObj.activeNodes+' </p>" +
+ " ShutdownNodes : '+capabilityObj.shutdownNodes+' " +
+ " </td>" +
+ " </tr>" +
+ " </table>').show(); "+
+ " tr.addClass('shown'); " +
+ " } " +
+ " }); });").__();
+ }
+
+ /**
+ * Initialize the Html page when Federation is enabled.
+ *
+ * @param html html object
+ */
+ private void initHtmlPageFederationEnabled(Block html) {
+ List<Map<String, String>> lists = new ArrayList<>();
+
+ // Table header
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("cell-border").$style("width:100%").thead().tr()
+ .th(".id", "SubCluster")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resources", "Resources")
+ .th(".nodes", "Nodes")
+ .__().__().tbody();
+
+ try {
+ // Binding to the FederationStateStore
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
+ Map<SubClusterId, SubClusterInfo> subClustersInfo =
+ facade.getSubClusters(true);
+
+ // Sort the SubClusters
+ List<SubClusterInfo> subclusters = new ArrayList<>();
+ subclusters.addAll(subClustersInfo.values());
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
+ Collections.sort(subclusters, cmp);
+
+ for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
+ SubClusterId subClusterId = subcluster.getSubClusterId();
+ String subClusterIdText = subClusterId.getId();
+
+ // Prepare WebAppAddress
+ String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
+ String capability = subcluster.getCapability();
+ ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
+
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1256387907
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 13s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 27s | | trunk passed |
| +1 :green_heart: | compile | 12m 7s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 10m 2s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 53s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 50s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 11m 0s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 11m 0s | | the patch passed |
| +1 :green_heart: | compile | 10m 49s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 10m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 2m 5s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 13s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 10s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 1m 24s | [/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/3/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 27m 48s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 32s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 19s | | hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 25s | | The patch does not generate ASF License warnings. |
| | | 186m 47s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| | Possible null pointer dereference of subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.initHtmlPageFederationEnabled(HtmlBlock$Block) Dereferenced at FederationBlock.java:subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.initHtmlPageFederationEnabled(HtmlBlock$Block) Dereferenced at FederationBlock.java:[line 251] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c337503d7289 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c8bc0c38a2533afb15fb89537b2c8fa669691c27 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/3/testReport/ |
| Max. process+thread count | 700 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r978076679
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
Review Comment:
This information is preserved and will be set in `initFederationBlockTableJs`.
For the Federation page, as a user, I first want to know the running status of the SubCluster, and then I care about the capacity information of the cluster. I have made some changes to the information in the displayed list, firstly showing
```
1.subClusterId
2.subClusterName
3.lastStartTime
4.lastHeartBeat
5.resources info
6.nodes info
```
When we click on the cluster id field, the metric information of the cluster will be displayed, which are Application Metric, Resource Metric, and Node Metric. When we click on the cluster id again, these information can be hidden, and only the most basic SubCluster status is exposed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r978078329
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationPage.java:
##########
@@ -47,10 +48,10 @@ protected Class<? extends SubView> content() {
private String rmsTableInit() {
StringBuilder b = tableInit().append(", aoColumnDefs: [");
- b.append("{'bSearchable': false, 'aTargets': [ 7 ]}")
+ b.append("{'bSearchable': false, 'aTargets': [ 2 ]}")
Review Comment:
I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1256827782
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 8s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 27s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 7s | | trunk passed |
| -1 :x: | compile | 7m 48s | [/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt) | hadoop-yarn in trunk failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 6m 24s | [/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-yarn in trunk failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| +1 :green_heart: | checkstyle | 2m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 29s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 38s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 53s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| -1 :x: | compile | 7m 1s | [/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt) | hadoop-yarn in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | javac | 7m 1s | [/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt) | hadoop-yarn in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04. |
| -1 :x: | compile | 6m 15s | [/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-yarn in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| -1 :x: | javac | 6m 15s | [/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-yarn in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 1m 57s | [/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt) | hadoop-yarn-project/hadoop-yarn: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) |
| +1 :green_heart: | mvnsite | 2m 10s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 2m 10s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 51s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 51s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 14s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 16s | | hadoop-yarn-server-router in the patch passed. |
| -1 :x: | asflicense | 1m 21s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/results-asflicense.txt) | The patch generated 1 ASF License warnings. |
| | | 156m 30s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets shellcheck shelldocs |
| uname | Linux daca58dafcb7 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d2c7ccaee8b8c362bb7d3b61357956af7fa77326 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/testReport/ |
| Max. process+thread count | 720 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1257179605
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +0 :ok: | jsonlint | 0m 1s | | jsonlint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 4 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 54s | | trunk passed |
| +1 :green_heart: | compile | 9m 55s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 48s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 34s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 24s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 32s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 9m 12s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 12s | | the patch passed |
| +1 :green_heart: | compile | 8m 38s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 47s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 26s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 55s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 23s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 17s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 18s | | hadoop-yarn-server-router in the patch passed. |
| -1 :x: | asflicense | 1m 8s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/7/artifact/out/results-asflicense.txt) | The patch generated 1 ASF License warnings. |
| | | 162m 36s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets shellcheck shelldocs jsonlint |
| uname | Linux bd92c50b2afc 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 481a99579326d5644f875359751bcca2501de33c |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/7/testReport/ |
| Max. process+thread count | 801 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1256377690
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 0s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 55s | | trunk passed |
| +1 :green_heart: | compile | 9m 53s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 46s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 13s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 28s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 9m 7s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 7s | | the patch passed |
| +1 :green_heart: | compile | 8m 31s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 1m 46s | [/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt) | hadoop-yarn-project/hadoop-yarn: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) |
| +1 :green_heart: | mvnsite | 2m 4s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 58s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 1s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 1m 20s | [/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/4/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 21m 30s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 14s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 17s | | hadoop-yarn-server-router in the patch passed. |
| -1 :x: | asflicense | 1m 20s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/4/artifact/out/results-asflicense.txt) | The patch generated 1 ASF License warnings. |
| | | 161m 20s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| | Possible null pointer dereference of subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.initHtmlPageFederationEnabled(HtmlBlock$Block) Dereferenced at FederationBlock.java:subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.initHtmlPageFederationEnabled(HtmlBlock$Block) Dereferenced at FederationBlock.java:[line 260] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets shellcheck shelldocs |
| uname | Linux 92489eecec78 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b58186ad307d57e84c6fd272d57b5e484f3dceaf |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/4/testReport/ |
| Max. process+thread count | 775 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1257581454
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | jshint | 0m 0s | | jshint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 22s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 54s | | trunk passed |
| +1 :green_heart: | compile | 9m 53s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 51s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 19s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 8s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 32s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 57s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 9m 11s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 11s | | the patch passed |
| +1 :green_heart: | compile | 8m 33s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 48s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 10s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 2m 5s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 1s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 33s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 27s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 19s | | hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not generate ASF License warnings. |
| | | 162m 30s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jshint shellcheck shelldocs |
| uname | Linux c7d001d04bed 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dd8aabcfeb13c0176c22ecb1dbc5d9882e47bc48 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/10/testReport/ |
| Max. process+thread count | 776 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1255808206
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 26s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 54s | | trunk passed |
| +1 :green_heart: | compile | 9m 50s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 50s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 16s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 8s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 31s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 43s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 9s | | the patch passed |
| +1 :green_heart: | compile | 9m 7s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 7s | | the patch passed |
| +1 :green_heart: | compile | 8m 35s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 51s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 1m 30s | [/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/2/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 21m 26s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 16s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 4s | | hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 21s | | The patch does not generate ASF License warnings. |
| | | 163m 49s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| | Possible null pointer dereference of subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.initHtmlPageFederationEnabled(HtmlBlock$Block) Dereferenced at FederationBlock.java:subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.initHtmlPageFederationEnabled(HtmlBlock$Block) Dereferenced at FederationBlock.java:[line 251] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 0e481057c4ad 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 27795b064d77bdabc0d57451e8235140f5b0bc45 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/2/testReport/ |
| Max. process+thread count | 745 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1256194380
Modify web page style
1. Information contained in Metirc, capitalized
2. The table adopts the cell-border style to allow the table to display part of the border.
3. Add some Junit Test, test yarn.federation.enabled=true/false, the web page can conform to the original page test specification
The new page is displayed as follows:
<img width="1665" alt="image" src="https://user-images.githubusercontent.com/55643692/191967220-363c2572-20c2-486f-8ab9-eda2d9909cce.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1255219512
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 13s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 43s | | trunk passed |
| +1 :green_heart: | compile | 9m 49s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 56s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 19s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 23s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 15s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 9m 38s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 38s | | the patch passed |
| +1 :green_heart: | compile | 9m 12s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 9m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 50s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 0s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 1m 26s | [/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/1/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.html) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 22m 24s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 27s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 1s | | hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not generate ASF License warnings. |
| | | 164m 4s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| | Possible null pointer dereference of subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.render(HtmlBlock$Block) Dereferenced at FederationBlock.java:subClusterId in org.apache.hadoop.yarn.server.router.webapp.FederationBlock.render(HtmlBlock$Block) Dereferenced at FederationBlock.java:[line 135] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 33e329c52432 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2ad4f09e128c3c6355aece5d0aed26bb9d6659fb |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/1/testReport/ |
| Max. process+thread count | 700 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r979134026
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationPage.java:
##########
@@ -33,8 +33,9 @@ class FederationPage extends RouterView {
@Override
protected void preHead(Page.HTML<__> html) {
commonPreHead(html);
- setTitle("Federation");
+ setTitle("About The Federation");
Review Comment:
I will modify the code.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/RouterController.java:
##########
@@ -34,7 +34,7 @@ public class RouterController extends Controller {
@Override
public void index() {
- setTitle("Router");
+ setTitle("About the Router");
Review Comment:
I will modify the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1258996912
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | jshint | 0m 0s | | jshint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 9s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 1s | | trunk passed |
| +1 :green_heart: | compile | 9m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 47s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 20s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 29s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 15s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 39s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 5s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 9m 7s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 7s | | the patch passed |
| +1 :green_heart: | compile | 8m 40s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 49s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 58s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 14s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 41s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 30s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 22s | | hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not generate ASF License warnings. |
| | | 163m 10s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/12/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jshint shellcheck shelldocs |
| uname | Linux 05ac04915865 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f9a7497815eb9becfcfcf4b3681630235d491a1f |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/12/testReport/ |
| Max. process+thread count | 769 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/12/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
goiri merged PR #4924:
URL: https://github.com/apache/hadoop/pull/4924
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1257457257
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 9s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | jshint | 0m 0s | | jshint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 32s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 25s | | trunk passed |
| +1 :green_heart: | compile | 10m 51s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 50s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 15s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 0s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 56s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 22s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 9m 8s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 8s | | the patch passed |
| +1 :green_heart: | compile | 8m 49s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 47s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 58s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 52s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 28s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 23s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 18s | | hadoop-yarn-server-router in the patch passed. |
| -1 :x: | asflicense | 1m 14s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/9/artifact/out/results-asflicense.txt) | The patch generated 1 ASF License warnings. |
| | | 166m 43s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jshint shellcheck shelldocs |
| uname | Linux 2737eaad4734 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2d4fa43519dce519997bed13ba49103d32eba9d9 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/9/testReport/ |
| Max. process+thread count | 738 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1257419464
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 29s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 54s | | trunk passed |
| +1 :green_heart: | compile | 9m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 1s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 29s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 52s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 19s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 9m 28s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 28s | | the patch passed |
| +1 :green_heart: | compile | 9m 24s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 9m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 2s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 56s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 53s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 30s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 19s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 12s | | hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not generate ASF License warnings. |
| | | 166m 14s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets shellcheck shelldocs |
| uname | Linux 6f5b355205c3 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / af57e7f89b9dfd6a734bffa9f4b722bbf0e06c30 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/8/testReport/ |
| Max. process+thread count | 771 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r980576264
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/federation/federation.js:
##########
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+$(document).ready(function() {
+ var table = $('#rms').DataTable();
+ $('#rms tbody').on('click', 'td.details-control', function () {
+ var tr = $(this).closest('tr');
+ var row = table.row(tr);
+ if (row.child.isShown()) {
+ row.child.hide();
+ tr.removeClass('shown');
+ } else {
+ var capabilityArr = scTableData.filter(item => (item.subcluster === row.id()));
+ var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics;
+ row.child(
+ '<table>' +
+ ' <tr>' +
+ ' <td> ' +
+ ' <h3>Application Metrics</h3> ' +
+ ' ApplicationSubmitted* : '+ capabilityObj.appsSubmitted +' </p>' +
+ ' ApplicationCompleted* : '+ capabilityObj.appsCompleted +' </p>' +
+ ' ApplicationPending* : '+ capabilityObj.appsPending +' </p>' +
+ ' ApplicationRunning* : '+ capabilityObj.appsRunning +' </p>'+
+ ' ApplicationFailed* : '+ capabilityObj.appsFailed +' </p>'+
+ ' ApplicationKilled* : '+ capabilityObj.appsKilled +' </p>'+
+ ' </td>' +
+ ' <td>' +
+ ' <h3>Resource Metrics</h3>'+
+ ' <h4>Memory</h4>'+
+ ' TotalMB : '+ capabilityObj.totalMB +' </p>' +
+ ' ReservedMB : '+ capabilityObj.reservedMB +' </p>'+
+ ' AvailableMB : '+ capabilityObj.availableMB +' </p> '+
+ ' AllocatedMB : '+ capabilityObj.allocatedMB +' </p> '+
+ ' PendingMB : '+ capabilityObj.pendingMB +' </p>' +
+ ' <h4>VirtualCores</h4>'+
+ ' TotalVirtualCores : '+capabilityObj.totalVirtualCores+' </p>'+
+ ' ReservedVirtualCores : '+capabilityObj.reservedVirtualCores+' </p>'+
+ ' AvailableVirtualCore : '+capabilityObj.availableVirtualCores+' </p>'+
+ ' AllocatedVirtualCores : '+capabilityObj.allocatedVirtualCores+' </p>'+
+ ' PendingVirtualCores : '+capabilityObj.pendingVirtualCores+' </p>'+
+ ' <h4>Containers</h4>'+
Review Comment:
Spaces before and after the plus sign
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1258977084
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | jshint | 0m 1s | | jshint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 7s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 52s | | trunk passed |
| +1 :green_heart: | compile | 9m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 44s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 13s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 16s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 40s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 8s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 9m 17s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 17s | | the patch passed |
| +1 :green_heart: | compile | 8m 35s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 51s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 1s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 58s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 57s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 23s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 15s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 31s | | hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not generate ASF License warnings. |
| | | 161m 47s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/11/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jshint shellcheck shelldocs |
| uname | Linux a30ee6c63474 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e9d2ae23147e3cfa66f976f4796187a55b4ad6cb |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/11/testReport/ |
| Max. process+thread count | 730 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r978071429
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
Review Comment:
Thanks a lot for your help reviewing the code, I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r979133698
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
+ initHtmlPageFederationNotEnabled(html);
}
}
- private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
+ /**
+ * Parse the capability and obtain the metric information of the cluster.
+ *
+ * @param capability metric json obtained from RM.
+ * @return ClusterMetricsInfo Object
+ */
+ private ClusterMetricsInfo getClusterMetricsInfo(String capability) {
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ /**
+ * Initialize the subCluster details JavaScript of the Federation page.
+ *
+ * This part of the js script will control to display or hide the detailed information
+ * of the subCluster when the user clicks on the subClusterId.
+ *
+ * We will obtain the specific information of a SubCluster,
+ * including the information of Applications, Resources, and Nodes.
+ *
+ * @param html html object
+ * @param subClusterDetailMap subCluster Detail Map
+ */
+ private void initFederationSubClusterDetailTableJs(Block html,
+ List<Map<String, String>> subClusterDetailMap) {
+ Gson gson = new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
+ " var scTableData = " + gson.toJson(subClusterDetailMap) + "; " +
+ " var table = $('#rms').DataTable(); " +
+ " $('#rms tbody').on('click', 'td.details-control', function () { " +
+ " var tr = $(this).closest('tr'); " +
+ " var row = table.row(tr); " +
+ " if (row.child.isShown()) { " +
+ " row.child.hide(); " +
+ " tr.removeClass('shown'); " +
+ " } else { " +
+ " var capabilityArr = scTableData.filter(item => (item.subcluster === row.id())); " +
+ " var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics; " +
+ " row.child(" +
+ " '<table>" +
+ " <tr>" +
+ " <td>" +
+ " <h3>Application Metrics</h3>" +
+ " ApplicationSubmitted* : '+ capabilityObj.appsSubmitted +' </p>" +
+ " ApplicationCompleted* : '+ capabilityObj.appsCompleted +' </p>" +
+ " ApplicationPending* : '+ capabilityObj.appsPending +' </p>" +
+ " ApplicationRunning* : '+ capabilityObj.appsRunning +' </p>" +
+ " ApplicationFailed* : '+ capabilityObj.appsFailed +' </p> " +
+ " ApplicationKilled* : '+ capabilityObj.appsKilled +' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Resource Metrics</h3>" +
+ " <h4>Memory</h4>" +
+ " TotalMB : '+ capabilityObj.totalMB +' </p>" +
+ " ReservedMB : '+ capabilityObj.reservedMB +' </p>" +
+ " AvailableMB : '+ capabilityObj.availableMB +' </p>" +
+ " AllocatedMB : '+ capabilityObj.allocatedMB +' </p>" +
+ " PendingMB : '+ capabilityObj.pendingMB +' </p>" +
+ " <h4>VirtualCores</h4>" +
+ " TotalVirtualCores : '+capabilityObj.totalVirtualCores+' </p>" +
+ " ReservedVirtualCores : '+capabilityObj.reservedVirtualCores+' </p>" +
+ " AvailableVirtualCore : '+capabilityObj.availableVirtualCores+' </p>" +
+ " AllocatedVirtualCores : '+capabilityObj.allocatedVirtualCores+' </p>" +
+ " PendingVirtualCores : '+capabilityObj.pendingVirtualCores+' </p>" +
+ " <h4>Containers</h4>" +
+ " ContainersAllocated : '+capabilityObj.containersAllocated+' </p>" +
+ " ContainersReserved : '+capabilityObj.containersReserved+' </p>" +
+ " ContainersPending : '+capabilityObj.containersPending+' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Node Metrics</h3>" +
+ " TotalNodes : '+capabilityObj.totalNodes+' </p>" +
+ " LostNodes : '+capabilityObj.lostNodes+' </p>" +
+ " UnhealthyNodes : '+capabilityObj.unhealthyNodes+' </p>" +
+ " DecommissioningNodes : '+capabilityObj.decommissioningNodes+' </p>" +
+ " DecommissionedNodes : '+capabilityObj.decommissionedNodes+' </p>" +
+ " RebootedNodes : '+capabilityObj.rebootedNodes+' </p>" +
+ " ActiveNodes : '+capabilityObj.activeNodes+' </p>" +
+ " ShutdownNodes : '+capabilityObj.shutdownNodes+' " +
+ " </td>" +
+ " </tr>" +
+ " </table>').show(); "+
+ " tr.addClass('shown'); " +
+ " } " +
+ " }); });").__();
+ }
+
+ /**
+ * Initialize the Html page when Federation is enabled.
+ *
+ * @param html html object
+ */
+ private void initHtmlPageFederationEnabled(Block html) {
+ List<Map<String, String>> lists = new ArrayList<>();
+
+ // Table header
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("cell-border").$style("width:100%").thead().tr()
+ .th(".id", "SubCluster")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resources", "Resources")
+ .th(".nodes", "Nodes")
+ .__().__().tbody();
+
+ try {
+ // Binding to the FederationStateStore
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
+ Map<SubClusterId, SubClusterInfo> subClustersInfo =
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r979133645
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
+ initHtmlPageFederationNotEnabled(html);
}
}
- private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
+ /**
+ * Parse the capability and obtain the metric information of the cluster.
+ *
+ * @param capability metric json obtained from RM.
+ * @return ClusterMetricsInfo Object
+ */
+ private ClusterMetricsInfo getClusterMetricsInfo(String capability) {
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ /**
+ * Initialize the subCluster details JavaScript of the Federation page.
+ *
+ * This part of the js script will control to display or hide the detailed information
+ * of the subCluster when the user clicks on the subClusterId.
+ *
+ * We will obtain the specific information of a SubCluster,
+ * including the information of Applications, Resources, and Nodes.
+ *
+ * @param html html object
+ * @param subClusterDetailMap subCluster Detail Map
+ */
+ private void initFederationSubClusterDetailTableJs(Block html,
+ List<Map<String, String>> subClusterDetailMap) {
+ Gson gson = new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
Review Comment:
Thanks for your help in reviewing the code, I agree with you, I will add js file.
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r978078002
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
+ Hamlet.DIV<Hamlet> div = html.div("#div_id");
+ div.p().$style("color:red").__("Federation is not Enabled.").__()
+ .p().__()
+ .p().__("We can refer to the following documents to configure Yarn Federation. ").__()
+ .p().$style("color:blue").__()
+ .a("https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/Federation.html",
+ "Hadoop: YARN Federation").
+ __();
}
}
private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ private static void initFederationBlockTableJs(Block html, List<Map<String, String>> jsonMap) {
Review Comment:
I will add the java doc to explain the setting of this function for basic information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r978078613
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resource", "Resource")
+ .th(".nodes", "Nodes")
.__().__().tbody();
try {
// Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
Map<SubClusterId, SubClusterInfo> subClustersInfo =
facade.getSubClusters(true);
// Sort the SubClusters
List<SubClusterInfo> subclusters = new ArrayList<>();
subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
Collections.sort(subclusters, cmp);
for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
SubClusterId subClusterId = subcluster.getSubClusterId();
+ String anchorText = "";
+ if (subClusterId != null) {
+ anchorText = subClusterId.getId();
+ }
+
+ // Prepare WebAppAddress
String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
String capability = subcluster.getCapability();
ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
+ totalNodes, activeNodes);
+
+ // Prepare HTML Table
+ tbody.tr().$id(subClusterId.toString())
+ .td().$class("details-control").a(herfWebAppAddress, anchorText).__()
+ .td(subcluster.getState().name())
+ .td(lastStartTime)
+ .td(lastHeartBeat)
+ .td(resources)
+ .td(nodes)
+ .__();
+
+ subclusterMap.put("subcluster", subClusterId.getId());
+ subclusterMap.put("capability", capability);
+ lists.add(subclusterMap);
}
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
+ } catch (Exception e) {
+ LOG.error("Cannot render Router Federation.", e);
}
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
+ // Init FederationBlockTableJs
+ initFederationBlockTableJs(html, lists);
+
+ // Tips
+ tbody.__().__().div().p().$style("color:red")
+ .__("*The application counts are local per subcluster").__().__();
} else {
- setTitle("Federation is not Enabled!");
+ // When Federation is not enabled, user information needs to be prompted
Review Comment:
Thanks for your suggestion, I will modify the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r978076679
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,195 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
- boolean isEnabled = conf.getBoolean(
- YarnConfiguration.FEDERATION_ENABLED,
+ boolean isEnabled = conf.getBoolean(YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
if (isEnabled) {
- setTitle("Federation");
+
+ List<Map<String, String>> lists = new ArrayList<>();
// Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("display").$style("width:100%").thead().tr()
.th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
Review Comment:
This information is preserved and will be set in `initFederationBlockTableJs`.
For the Federation page, as a user, I first want to know the running status of the SubCluster, and then I care about the capacity information of the cluster. I have made some changes to the information in the displayed list, firstly showing
```
1.subClusterId
2.subClusterName
3.lastStartTime
4.lastHeartBeat
5.resources info
6.nodes info
```
When we click on the cluster id field, the metric information of the cluster will be displayed, which are divided into Application Metric, Resource Metric, and Node Metric.
When we click on the cluster id field, the metric information of the cluster will be displayed, which are Application Metric, Resource Metric, and Node Metric. When we click on the cluster id again, these information can be hidden, and only the most basic SubCluster status is exposed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
slfan1989 commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r980626019
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/federation/federation.js:
##########
@@ -0,0 +1,77 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+$(document).ready(function() {
+ var table = $('#rms').DataTable();
+ $('#rms tbody').on('click', 'td.details-control', function () {
+ var tr = $(this).closest('tr');
+ var row = table.row(tr);
+ if (row.child.isShown()) {
+ row.child.hide();
+ tr.removeClass('shown');
+ } else {
+ var capabilityArr = scTableData.filter(item => (item.subcluster === row.id()));
+ var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics;
+ row.child(
+ '<table>' +
+ ' <tr>' +
+ ' <td> ' +
+ ' <h3>Application Metrics</h3> ' +
+ ' ApplicationSubmitted* : '+ capabilityObj.appsSubmitted +' </p>' +
+ ' ApplicationCompleted* : '+ capabilityObj.appsCompleted +' </p>' +
+ ' ApplicationPending* : '+ capabilityObj.appsPending +' </p>' +
+ ' ApplicationRunning* : '+ capabilityObj.appsRunning +' </p>'+
+ ' ApplicationFailed* : '+ capabilityObj.appsFailed +' </p>'+
+ ' ApplicationKilled* : '+ capabilityObj.appsKilled +' </p>'+
+ ' </td>' +
+ ' <td>' +
+ ' <h3>Resource Metrics</h3>'+
+ ' <h4>Memory</h4>'+
+ ' TotalMB : '+ capabilityObj.totalMB +' </p>' +
+ ' ReservedMB : '+ capabilityObj.reservedMB +' </p>'+
+ ' AvailableMB : '+ capabilityObj.availableMB +' </p> '+
+ ' AllocatedMB : '+ capabilityObj.allocatedMB +' </p> '+
+ ' PendingMB : '+ capabilityObj.pendingMB +' </p>' +
+ ' <h4>VirtualCores</h4>'+
+ ' TotalVirtualCores : '+capabilityObj.totalVirtualCores+' </p>'+
+ ' ReservedVirtualCores : '+capabilityObj.reservedVirtualCores+' </p>'+
+ ' AvailableVirtualCore : '+capabilityObj.availableVirtualCores+' </p>'+
+ ' AllocatedVirtualCores : '+capabilityObj.allocatedVirtualCores+' </p>'+
+ ' PendingVirtualCores : '+capabilityObj.pendingVirtualCores+' </p>'+
+ ' <h4>Containers</h4>'+
Review Comment:
Thank you very much for helping to review the code, I will modify the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#discussion_r979129686
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
Review Comment:
else?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
+ initHtmlPageFederationNotEnabled(html);
}
}
- private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
+ /**
+ * Parse the capability and obtain the metric information of the cluster.
+ *
+ * @param capability metric json obtained from RM.
+ * @return ClusterMetricsInfo Object
+ */
+ private ClusterMetricsInfo getClusterMetricsInfo(String capability) {
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ /**
+ * Initialize the subCluster details JavaScript of the Federation page.
+ *
+ * This part of the js script will control to display or hide the detailed information
+ * of the subCluster when the user clicks on the subClusterId.
+ *
+ * We will obtain the specific information of a SubCluster,
+ * including the information of Applications, Resources, and Nodes.
+ *
+ * @param html html object
+ * @param subClusterDetailMap subCluster Detail Map
+ */
+ private void initFederationSubClusterDetailTableJs(Block html,
+ List<Map<String, String>> subClusterDetailMap) {
+ Gson gson = new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
+ " var scTableData = " + gson.toJson(subClusterDetailMap) + "; " +
+ " var table = $('#rms').DataTable(); " +
+ " $('#rms tbody').on('click', 'td.details-control', function () { " +
+ " var tr = $(this).closest('tr'); " +
+ " var row = table.row(tr); " +
+ " if (row.child.isShown()) { " +
+ " row.child.hide(); " +
+ " tr.removeClass('shown'); " +
+ " } else { " +
+ " var capabilityArr = scTableData.filter(item => (item.subcluster === row.id())); " +
+ " var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics; " +
+ " row.child(" +
+ " '<table>" +
+ " <tr>" +
+ " <td>" +
+ " <h3>Application Metrics</h3>" +
+ " ApplicationSubmitted* : '+ capabilityObj.appsSubmitted +' </p>" +
+ " ApplicationCompleted* : '+ capabilityObj.appsCompleted +' </p>" +
+ " ApplicationPending* : '+ capabilityObj.appsPending +' </p>" +
+ " ApplicationRunning* : '+ capabilityObj.appsRunning +' </p>" +
+ " ApplicationFailed* : '+ capabilityObj.appsFailed +' </p> " +
+ " ApplicationKilled* : '+ capabilityObj.appsKilled +' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Resource Metrics</h3>" +
+ " <h4>Memory</h4>" +
+ " TotalMB : '+ capabilityObj.totalMB +' </p>" +
+ " ReservedMB : '+ capabilityObj.reservedMB +' </p>" +
+ " AvailableMB : '+ capabilityObj.availableMB +' </p>" +
+ " AllocatedMB : '+ capabilityObj.allocatedMB +' </p>" +
+ " PendingMB : '+ capabilityObj.pendingMB +' </p>" +
+ " <h4>VirtualCores</h4>" +
+ " TotalVirtualCores : '+capabilityObj.totalVirtualCores+' </p>" +
+ " ReservedVirtualCores : '+capabilityObj.reservedVirtualCores+' </p>" +
+ " AvailableVirtualCore : '+capabilityObj.availableVirtualCores+' </p>" +
+ " AllocatedVirtualCores : '+capabilityObj.allocatedVirtualCores+' </p>" +
+ " PendingVirtualCores : '+capabilityObj.pendingVirtualCores+' </p>" +
+ " <h4>Containers</h4>" +
+ " ContainersAllocated : '+capabilityObj.containersAllocated+' </p>" +
+ " ContainersReserved : '+capabilityObj.containersReserved+' </p>" +
+ " ContainersPending : '+capabilityObj.containersPending+' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Node Metrics</h3>" +
+ " TotalNodes : '+capabilityObj.totalNodes+' </p>" +
+ " LostNodes : '+capabilityObj.lostNodes+' </p>" +
+ " UnhealthyNodes : '+capabilityObj.unhealthyNodes+' </p>" +
+ " DecommissioningNodes : '+capabilityObj.decommissioningNodes+' </p>" +
+ " DecommissionedNodes : '+capabilityObj.decommissionedNodes+' </p>" +
+ " RebootedNodes : '+capabilityObj.rebootedNodes+' </p>" +
+ " ActiveNodes : '+capabilityObj.activeNodes+' </p>" +
+ " ShutdownNodes : '+capabilityObj.shutdownNodes+' " +
+ " </td>" +
+ " </tr>" +
+ " </table>').show(); "+
+ " tr.addClass('shown'); " +
+ " } " +
+ " }); });").__();
+ }
+
+ /**
+ * Initialize the Html page when Federation is enabled.
+ *
+ * @param html html object
+ */
+ private void initHtmlPageFederationEnabled(Block html) {
+ List<Map<String, String>> lists = new ArrayList<>();
+
+ // Table header
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("cell-border").$style("width:100%").thead().tr()
+ .th(".id", "SubCluster")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resources", "Resources")
+ .th(".nodes", "Nodes")
+ .__().__().tbody();
+
+ try {
+ // Binding to the FederationStateStore
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
+ Map<SubClusterId, SubClusterInfo> subClustersInfo =
+ facade.getSubClusters(true);
+
+ // Sort the SubClusters
+ List<SubClusterInfo> subclusters = new ArrayList<>();
+ subclusters.addAll(subClustersInfo.values());
+ Comparator<? super SubClusterInfo> cmp = Comparator.comparing(o -> o.getSubClusterId());
+ Collections.sort(subclusters, cmp);
+
+ for (SubClusterInfo subcluster : subclusters) {
+
+ Map<String, String> subclusterMap = new HashMap<>();
+
+ // Prepare subCluster
+ SubClusterId subClusterId = subcluster.getSubClusterId();
+ String subClusterIdText = subClusterId.getId();
+
+ // Prepare WebAppAddress
+ String webAppAddress = subcluster.getRMWebServiceAddress();
+ String herfWebAppAddress = "";
+ if (webAppAddress != null && !webAppAddress.isEmpty()) {
+ herfWebAppAddress = "//" + webAppAddress;
+ }
+
+ // Prepare Capability
+ String capability = subcluster.getCapability();
+ ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
+
+ // Prepare LastStartTime & LastHeartBeat
+ String lastStartTime =
+ DateFormatUtils.format(subcluster.getLastStartTime(), DATE_PATTERN);
+ String lastHeartBeat =
+ DateFormatUtils.format(subcluster.getLastHeartBeat(), DATE_PATTERN);
+
+ // Prepare Resource
+ long totalMB = subClusterInfo.getTotalMB();
+ String totalMBDesc = StringUtils.byteDesc(totalMB * BYTES_IN_MB);
+ long totalVirtualCores = subClusterInfo.getTotalVirtualCores();
+ String resources = String.format("<Memory:%s, VCore:%s>", totalMBDesc, totalVirtualCores);
+
+ // Prepare Node
+ long totalNodes = subClusterInfo.getTotalNodes();
+ long activeNodes = subClusterInfo.getActiveNodes();
+ String nodes = String.format("<Total Nodes:%s, Active Nodes:%s>",
Review Comment:
Single line?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/RouterController.java:
##########
@@ -34,7 +34,7 @@ public class RouterController extends Controller {
@Override
public void index() {
- setTitle("Router");
+ setTitle("About the Router");
Review Comment:
`About the YARN Router`?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
+ initHtmlPageFederationNotEnabled(html);
}
}
- private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
+ /**
+ * Parse the capability and obtain the metric information of the cluster.
+ *
+ * @param capability metric json obtained from RM.
+ * @return ClusterMetricsInfo Object
+ */
+ private ClusterMetricsInfo getClusterMetricsInfo(String capability) {
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ /**
+ * Initialize the subCluster details JavaScript of the Federation page.
+ *
+ * This part of the js script will control to display or hide the detailed information
+ * of the subCluster when the user clicks on the subClusterId.
+ *
+ * We will obtain the specific information of a SubCluster,
+ * including the information of Applications, Resources, and Nodes.
+ *
+ * @param html html object
+ * @param subClusterDetailMap subCluster Detail Map
+ */
+ private void initFederationSubClusterDetailTableJs(Block html,
+ List<Map<String, String>> subClusterDetailMap) {
+ Gson gson = new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
Review Comment:
This massive string with double embedding is very dangerous.
Isn't there a better way to have some javascript file and some html template?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationPage.java:
##########
@@ -33,8 +33,9 @@ class FederationPage extends RouterView {
@Override
protected void preHead(Page.HTML<__> html) {
commonPreHead(html);
- setTitle("Federation");
+ setTitle("About The Federation");
Review Comment:
I think it makes more sense to have `About YARN federation`
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationBlock.java:
##########
@@ -58,119 +58,232 @@ class FederationBlock extends HtmlBlock {
@Override
public void render(Block html) {
+
Configuration conf = this.router.getConfig();
boolean isEnabled = conf.getBoolean(
YarnConfiguration.FEDERATION_ENABLED,
YarnConfiguration.DEFAULT_FEDERATION_ENABLED);
+
+ // If Yarn Federation is enabled.
if (isEnabled) {
- setTitle("Federation");
-
- // Table header
- TBODY<TABLE<Hamlet>> tbody = html.table("#rms").thead().tr()
- .th(".id", "SubCluster")
- .th(".submittedA", "Applications Submitted*")
- .th(".pendingA", "Applications Pending*")
- .th(".runningA", "Applications Running*")
- .th(".failedA", "Applications Failed*")
- .th(".killedA", "Applications Killed*")
- .th(".completedA", "Applications Completed*")
- .th(".contAllocated", "Containers Allocated")
- .th(".contReserved", "Containers Reserved")
- .th(".contPending", "Containers Pending")
- .th(".availableM", "Available Memory")
- .th(".allocatedM", "Allocated Memory")
- .th(".reservedM", "Reserved Memory")
- .th(".totalM", "Total Memory")
- .th(".availableVC", "Available VirtualCores")
- .th(".allocatedVC", "Allocated VirtualCores")
- .th(".reservedVC", "Reserved VirtualCores")
- .th(".totalVC", "Total VirtualCores")
- .th(".activeN", "Active Nodes")
- .th(".lostN", "Lost Nodes")
- .th(".availableN", "Available Nodes")
- .th(".unhealtyN", "Unhealthy Nodes")
- .th(".rebootedN", "Rebooted Nodes")
- .th(".totalN", "Total Nodes")
- .__().__().tbody();
-
- try {
- // Binding to the FederationStateStore
- FederationStateStoreFacade facade =
- FederationStateStoreFacade.getInstance();
- Map<SubClusterId, SubClusterInfo> subClustersInfo =
- facade.getSubClusters(true);
-
- // Sort the SubClusters
- List<SubClusterInfo> subclusters = new ArrayList<>();
- subclusters.addAll(subClustersInfo.values());
- Comparator<? super SubClusterInfo> cmp =
- new Comparator<SubClusterInfo>() {
- @Override
- public int compare(SubClusterInfo o1, SubClusterInfo o2) {
- return o1.getSubClusterId().compareTo(o2.getSubClusterId());
- }
- };
- Collections.sort(subclusters, cmp);
-
- for (SubClusterInfo subcluster : subclusters) {
- SubClusterId subClusterId = subcluster.getSubClusterId();
- String webAppAddress = subcluster.getRMWebServiceAddress();
- String capability = subcluster.getCapability();
- ClusterMetricsInfo subClusterInfo = getClusterMetricsInfo(capability);
-
- // Building row per SubCluster
- tbody.tr().td().a("//" + webAppAddress, subClusterId.toString()).__()
- .td(Integer.toString(subClusterInfo.getAppsSubmitted()))
- .td(Integer.toString(subClusterInfo.getAppsPending()))
- .td(Integer.toString(subClusterInfo.getAppsRunning()))
- .td(Integer.toString(subClusterInfo.getAppsFailed()))
- .td(Integer.toString(subClusterInfo.getAppsKilled()))
- .td(Integer.toString(subClusterInfo.getAppsCompleted()))
- .td(Integer.toString(subClusterInfo.getContainersAllocated()))
- .td(Integer.toString(subClusterInfo.getReservedContainers()))
- .td(Integer.toString(subClusterInfo.getPendingContainers()))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAvailableMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getAllocatedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getReservedMB() * BYTES_IN_MB))
- .td(StringUtils.byteDesc(
- subClusterInfo.getTotalMB() * BYTES_IN_MB))
- .td(Long.toString(subClusterInfo.getAvailableVirtualCores()))
- .td(Long.toString(subClusterInfo.getAllocatedVirtualCores()))
- .td(Long.toString(subClusterInfo.getReservedVirtualCores()))
- .td(Long.toString(subClusterInfo.getTotalVirtualCores()))
- .td(Integer.toString(subClusterInfo.getActiveNodes()))
- .td(Integer.toString(subClusterInfo.getLostNodes()))
- .td(Integer.toString(subClusterInfo.getDecommissionedNodes()))
- .td(Integer.toString(subClusterInfo.getUnhealthyNodes()))
- .td(Integer.toString(subClusterInfo.getRebootedNodes()))
- .td(Integer.toString(subClusterInfo.getTotalNodes())).__();
- }
- } catch (YarnException e) {
- LOG.error("Cannot render ResourceManager", e);
- }
+ initHtmlPageFederationEnabled(html);
+ }
- tbody.__().__().div()
- .p().__("*The application counts are local per subcluster").__().__();
- } else {
- setTitle("Federation is not Enabled!");
+ // If Yarn Federation is not enabled.
+ if(!isEnabled) {
+ initHtmlPageFederationNotEnabled(html);
}
}
- private static ClusterMetricsInfo getClusterMetricsInfo(String capability) {
- ClusterMetricsInfo clusterMetrics = null;
+ /**
+ * Parse the capability and obtain the metric information of the cluster.
+ *
+ * @param capability metric json obtained from RM.
+ * @return ClusterMetricsInfo Object
+ */
+ private ClusterMetricsInfo getClusterMetricsInfo(String capability) {
try {
- JSONJAXBContext jc = new JSONJAXBContext(
- JSONConfiguration.mapped().rootUnwrapping(false).build(),
- ClusterMetricsInfo.class);
- JSONUnmarshaller unmarshaller = jc.createJSONUnmarshaller();
- clusterMetrics = unmarshaller.unmarshalFromJSON(
- new StringReader(capability), ClusterMetricsInfo.class);
+ if (capability != null && !capability.isEmpty()) {
+ JSONJAXBContext jc = new JSONJAXBContext(
+ JSONConfiguration.mapped().rootUnwrapping(false).build(), ClusterMetricsInfo.class);
+ JSONUnmarshaller unmarShaller = jc.createJSONUnmarshaller();
+ StringReader stringReader = new StringReader(capability);
+ ClusterMetricsInfo clusterMetrics =
+ unmarShaller.unmarshalFromJSON(stringReader, ClusterMetricsInfo.class);
+ return clusterMetrics;
+ }
} catch (Exception e) {
LOG.error("Cannot parse SubCluster info", e);
}
- return clusterMetrics;
+ return null;
+ }
+
+ /**
+ * Initialize the subCluster details JavaScript of the Federation page.
+ *
+ * This part of the js script will control to display or hide the detailed information
+ * of the subCluster when the user clicks on the subClusterId.
+ *
+ * We will obtain the specific information of a SubCluster,
+ * including the information of Applications, Resources, and Nodes.
+ *
+ * @param html html object
+ * @param subClusterDetailMap subCluster Detail Map
+ */
+ private void initFederationSubClusterDetailTableJs(Block html,
+ List<Map<String, String>> subClusterDetailMap) {
+ Gson gson = new Gson();
+ html.script().$type("text/javascript").
+ __("$(document).ready(function() { " +
+ " var scTableData = " + gson.toJson(subClusterDetailMap) + "; " +
+ " var table = $('#rms').DataTable(); " +
+ " $('#rms tbody').on('click', 'td.details-control', function () { " +
+ " var tr = $(this).closest('tr'); " +
+ " var row = table.row(tr); " +
+ " if (row.child.isShown()) { " +
+ " row.child.hide(); " +
+ " tr.removeClass('shown'); " +
+ " } else { " +
+ " var capabilityArr = scTableData.filter(item => (item.subcluster === row.id())); " +
+ " var capabilityObj = JSON.parse(capabilityArr[0].capability).clusterMetrics; " +
+ " row.child(" +
+ " '<table>" +
+ " <tr>" +
+ " <td>" +
+ " <h3>Application Metrics</h3>" +
+ " ApplicationSubmitted* : '+ capabilityObj.appsSubmitted +' </p>" +
+ " ApplicationCompleted* : '+ capabilityObj.appsCompleted +' </p>" +
+ " ApplicationPending* : '+ capabilityObj.appsPending +' </p>" +
+ " ApplicationRunning* : '+ capabilityObj.appsRunning +' </p>" +
+ " ApplicationFailed* : '+ capabilityObj.appsFailed +' </p> " +
+ " ApplicationKilled* : '+ capabilityObj.appsKilled +' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Resource Metrics</h3>" +
+ " <h4>Memory</h4>" +
+ " TotalMB : '+ capabilityObj.totalMB +' </p>" +
+ " ReservedMB : '+ capabilityObj.reservedMB +' </p>" +
+ " AvailableMB : '+ capabilityObj.availableMB +' </p>" +
+ " AllocatedMB : '+ capabilityObj.allocatedMB +' </p>" +
+ " PendingMB : '+ capabilityObj.pendingMB +' </p>" +
+ " <h4>VirtualCores</h4>" +
+ " TotalVirtualCores : '+capabilityObj.totalVirtualCores+' </p>" +
+ " ReservedVirtualCores : '+capabilityObj.reservedVirtualCores+' </p>" +
+ " AvailableVirtualCore : '+capabilityObj.availableVirtualCores+' </p>" +
+ " AllocatedVirtualCores : '+capabilityObj.allocatedVirtualCores+' </p>" +
+ " PendingVirtualCores : '+capabilityObj.pendingVirtualCores+' </p>" +
+ " <h4>Containers</h4>" +
+ " ContainersAllocated : '+capabilityObj.containersAllocated+' </p>" +
+ " ContainersReserved : '+capabilityObj.containersReserved+' </p>" +
+ " ContainersPending : '+capabilityObj.containersPending+' </p>" +
+ " </td>" +
+ " <td>" +
+ " <h3>Node Metrics</h3>" +
+ " TotalNodes : '+capabilityObj.totalNodes+' </p>" +
+ " LostNodes : '+capabilityObj.lostNodes+' </p>" +
+ " UnhealthyNodes : '+capabilityObj.unhealthyNodes+' </p>" +
+ " DecommissioningNodes : '+capabilityObj.decommissioningNodes+' </p>" +
+ " DecommissionedNodes : '+capabilityObj.decommissionedNodes+' </p>" +
+ " RebootedNodes : '+capabilityObj.rebootedNodes+' </p>" +
+ " ActiveNodes : '+capabilityObj.activeNodes+' </p>" +
+ " ShutdownNodes : '+capabilityObj.shutdownNodes+' " +
+ " </td>" +
+ " </tr>" +
+ " </table>').show(); "+
+ " tr.addClass('shown'); " +
+ " } " +
+ " }); });").__();
+ }
+
+ /**
+ * Initialize the Html page when Federation is enabled.
+ *
+ * @param html html object
+ */
+ private void initHtmlPageFederationEnabled(Block html) {
+ List<Map<String, String>> lists = new ArrayList<>();
+
+ // Table header
+ TBODY<TABLE<Hamlet>> tbody =
+ html.table("#rms").$class("cell-border").$style("width:100%").thead().tr()
+ .th(".id", "SubCluster")
+ .th(".state", "State")
+ .th(".lastStartTime", "LastStartTime")
+ .th(".lastHeartBeat", "LastHeartBeat")
+ .th(".resources", "Resources")
+ .th(".nodes", "Nodes")
+ .__().__().tbody();
+
+ try {
+ // Binding to the FederationStateStore
+ FederationStateStoreFacade facade = FederationStateStoreFacade.getInstance();
+
+ Map<SubClusterId, SubClusterInfo> subClustersInfo =
Review Comment:
Single line?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4924: YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4924:
URL: https://github.com/apache/hadoop/pull/4924#issuecomment-1257142488
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +0 :ok: | jsonlint | 0m 1s | | jsonlint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 4 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 0s | | trunk passed |
| +1 :green_heart: | compile | 11m 9s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 35s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 26s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 54s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 51s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 9m 46s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 46s | | the patch passed |
| +1 :green_heart: | compile | 9m 7s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 9m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 1m 50s | [/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/6/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt) | hadoop-yarn-project/hadoop-yarn: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) |
| +1 :green_heart: | mvnsite | 2m 9s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 58s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 11s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 3s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 5m 28s | | hadoop-yarn-common in the patch passed. |
| +1 :green_heart: | unit | 4m 26s | | hadoop-yarn-server-router in the patch passed. |
| -1 :x: | asflicense | 1m 7s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/6/artifact/out/results-asflicense.txt) | The patch generated 1 ASF License warnings. |
| | | 169m 28s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4924 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets shellcheck shelldocs jsonlint |
| uname | Linux 99c2fa9326ab 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a31411239f80b94ca1a7200774aa30681d8a416b |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/6/testReport/ |
| Max. process+thread count | 726 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4924/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org