You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "priyeshkaratha (via GitHub)" <gi...@apache.org> on 2026/02/12 05:03:54 UTC
[PR] HDDS-14621. [Recon] Populate Missing Fields in /clusterState StorageReport [ozone]
priyeshkaratha opened a new pull request, #9754:
URL: https://github.com/apache/ozone/pull/9754
## What changes were proposed in this pull request?
`DatanodeStorageReport` has recently been updated to include several additional fields. Some of these fields, such as `hostName` and `datanodeUuid`, are specific to individual datanodes and are not applicable to the `clusterState` storage report.
To address this, in this PR I have separated the models:
* `DatanodeStorageReport` is now used specifically for datanode-level reporting.
* A new `ClusterStorageReport` class has been introduced for the cluster state storage report, containing only the relevant cluster-level fields.
Additionally, the Recon OpenAPI specification was not fully aligned with the recent changes. This PR also includes updates to the OpenAPI spec to ensure it is consistent with the current API models.
Also added few clean up in ClusterStateEndpoint
## What is the link to the Apache JIRA
HDDS-14621
## How was this patch tested?
Tested using unit testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14621. [Recon] Populate Missing Fields in /clusterState StorageReport [ozone]
Posted by "ArafatKhan2198 (via GitHub)" <gi...@apache.org>.
ArafatKhan2198 merged PR #9754:
URL: https://github.com/apache/ozone/pull/9754
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14621. [Recon] Populate Missing Fields in /clusterState StorageReport [ozone]
Posted by "priyeshkaratha (via GitHub)" <gi...@apache.org>.
priyeshkaratha commented on code in PR #9754:
URL: https://github.com/apache/ozone/pull/9754#discussion_r2799157115
##########
hadoop-hdds/docs/themes/ozonedoc/static/swagger-resources/recon-api.yaml:
##########
@@ -1767,6 +1767,12 @@ components:
StorageReport:
Review Comment:
Yes, It is better to rename to DataNodeStorageReport.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14621. [Recon] Populate Missing Fields in /clusterState StorageReport [ozone]
Posted by "priyeshkaratha (via GitHub)" <gi...@apache.org>.
priyeshkaratha commented on code in PR #9754:
URL: https://github.com/apache/ozone/pull/9754#discussion_r2799158382
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -119,18 +117,34 @@ public Response getClusterState() {
this.containerManager.getContainerStateCount(
HddsProtos.LifeCycleState.DELETED));
- int healthyDatanodes =
+ int healthyDataNodes =
nodeManager.getNodeCount(NodeStatus.inServiceHealthy()) +
nodeManager.getNodeCount(NodeStatus.inServiceHealthyReadOnly());
SCMNodeStat stats = nodeManager.getStats();
+ long fsCapacity = 0;
+ long fsUsed = 0;
+ long fsAvailable = 0;
+
+ for (DatanodeInfo datanode : nodeManager.getAllNodes()) {
+ SpaceUsageSource.Fixed fsUsage = nodeManager.getTotalFilesystemUsage(datanode);
+ if (fsUsage != null) {
+ fsCapacity += fsUsage.getCapacity();
+ fsAvailable += fsUsage.getAvailable();
+ fsUsed += fsUsage.getUsedSpace();
+ }
+ }
Review Comment:
Added the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14621. [Recon] Populate Missing Fields in /clusterState StorageReport [ozone]
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #9754:
URL: https://github.com/apache/ozone/pull/9754#discussion_r2799084097
##########
hadoop-hdds/docs/themes/ozonedoc/static/swagger-resources/recon-api.yaml:
##########
@@ -1767,6 +1767,12 @@ components:
StorageReport:
Review Comment:
Not sure why this is not renamed. This creates naming confusion if it is clusterstatestoragereport or datanodestoragereport
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -119,18 +117,34 @@ public Response getClusterState() {
this.containerManager.getContainerStateCount(
HddsProtos.LifeCycleState.DELETED));
- int healthyDatanodes =
+ int healthyDataNodes =
nodeManager.getNodeCount(NodeStatus.inServiceHealthy()) +
nodeManager.getNodeCount(NodeStatus.inServiceHealthyReadOnly());
SCMNodeStat stats = nodeManager.getStats();
+ long fsCapacity = 0;
+ long fsUsed = 0;
+ long fsAvailable = 0;
+
+ for (DatanodeInfo datanode : nodeManager.getAllNodes()) {
+ SpaceUsageSource.Fixed fsUsage = nodeManager.getTotalFilesystemUsage(datanode);
+ if (fsUsage != null) {
+ fsCapacity += fsUsage.getCapacity();
+ fsAvailable += fsUsage.getAvailable();
+ fsUsed += fsUsage.getUsedSpace();
+ }
+ }
Review Comment:
```suggestion
}List<DatanodeInfo> datanodes = nodeManager.getAllNodes();
if (datanodes == null || datanodes.isEmpty()) {
LOG.warn("No datanodes available for filesystem usage calculation");
// Set to 0 or -1 to indicate unavailable
}
int reportedNodes = 0;
int totalNodes = datanodes.size();
for (DatanodeInfo datanode : datanodes) {
SpaceUsageSource.Fixed fsUsage = nodeManager.getTotalFilesystemUsage(datanode);
if (fsUsage != null) {
fsCapacity += fsUsage.getCapacity();
fsAvailable += fsUsage.getAvailable();
fsUsed += fsUsage.getUsedSpace();
reportedNodes++;
} else {
LOG.debug("Datanode {} has not reported filesystem usage",
datanode.getUuid());
}
}
if (reportedNodes < totalNodes) {
LOG.warn("Filesystem usage incomplete: {}/{} datanodes reported",
reportedNodes, totalNodes);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org