You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "slfan1989 (via GitHub)" <gi...@apache.org> on 2026/05/14 10:08:43 UTC
[PR] HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly. [ozone]
slfan1989 opened a new pull request, #10267:
URL: https://github.com/apache/ozone/pull/10267
## What changes were proposed in this pull request?
This PR makes DiskBalancerVolumeCalculation#getIdealUsage handle empty volume lists and zero total capacity explicitly.
Previously, getIdealUsage directly divided totalEffectiveUsed by totalCapacity. When the volume list was empty, totalCapacity was 0, which caused the method to return NaN. When all volumes had zero capacity, the method could also produce invalid utilization values. These invalid values could propagate to DiskBalancer reporting and balancing calculations.
## What is the link to the Apache JIRA
JIRA: HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly.
## How was this patch tested?
Add Junit Test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly. [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #10267:
URL: https://github.com/apache/ozone/pull/10267#issuecomment-4456990560
Last patch LGTM, +1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly. [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo merged PR #10267:
URL: https://github.com/apache/ozone/pull/10267
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly. [ozone]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #10267:
URL: https://github.com/apache/ozone/pull/10267#issuecomment-4457341626
@szetszwo @Gargi-jais11 Thanks for the review and for merging the patch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly. [ozone]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on code in PR #10267:
URL: https://github.com/apache/ozone/pull/10267#discussion_r3245537661
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerVolumeCalculation.java:
##########
@@ -70,13 +70,22 @@ public static List<VolumeFixedUsage> getVolumeUsages(MutableVolumeSet volumeSet,
* @throws IllegalArgumentException if total capacity is zero
*/
public static double getIdealUsage(List<VolumeFixedUsage> volumes) {
+ if (volumes == null || volumes.isEmpty()) {
+ return 0.0;
+ }
+
long totalCapacity = 0L, totalEffectiveUsed = 0L;
-
+
for (VolumeFixedUsage volumeUsage : volumes) {
totalCapacity += volumeUsage.getUsage().getCapacity();
totalEffectiveUsed += volumeUsage.getEffectiveUsed();
}
-
+
+ if (totalCapacity <= 0) {
Review Comment:
Updated. `totalCapacity == 0` now returns `0.0` for consistency with the empty volume case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly. [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #10267:
URL: https://github.com/apache/ozone/pull/10267#discussion_r3243469251
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerVolumeCalculation.java:
##########
@@ -70,13 +70,22 @@ public static List<VolumeFixedUsage> getVolumeUsages(MutableVolumeSet volumeSet,
* @throws IllegalArgumentException if total capacity is zero
*/
public static double getIdealUsage(List<VolumeFixedUsage> volumes) {
+ if (volumes == null || volumes.isEmpty()) {
+ return 0.0;
+ }
+
long totalCapacity = 0L, totalEffectiveUsed = 0L;
-
+
for (VolumeFixedUsage volumeUsage : volumes) {
totalCapacity += volumeUsage.getUsage().getCapacity();
totalEffectiveUsed += volumeUsage.getEffectiveUsed();
Review Comment:
Let's check each volume and check more cases:
```java
for (VolumeFixedUsage volumeUsage : volumes) {
final long capacity = volumeUsage.getUsage().getCapacity();
if (capacity < 0) {
throw new IllegalArgumentException("Negative capacity = " + capacity
+ ": " + volumeUsage.getVolume());
}
final long effectiveUsed = volumeUsage.getEffectiveUsed();
if (effectiveUsed < 0) {
throw new IllegalArgumentException("Negative effective used = " + effectiveUsed
+ ": " + volumeUsage.getVolume());
}
if (effectiveUsed > capacity) {
throw new IllegalArgumentException("Effective used = " + effectiveUsed + " > capacity = " + capacity
+ ": " + volumeUsage.getVolume());
}
totalCapacity += capacity;
totalEffectiveUsed += effectiveUsed;
}
```
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerVolumeCalculation.java:
##########
@@ -70,13 +70,22 @@ public static List<VolumeFixedUsage> getVolumeUsages(MutableVolumeSet volumeSet,
* @throws IllegalArgumentException if total capacity is zero
*/
public static double getIdealUsage(List<VolumeFixedUsage> volumes) {
+ if (volumes == null || volumes.isEmpty()) {
+ return 0.0;
+ }
+
long totalCapacity = 0L, totalEffectiveUsed = 0L;
-
+
for (VolumeFixedUsage volumeUsage : volumes) {
totalCapacity += volumeUsage.getUsage().getCapacity();
totalEffectiveUsed += volumeUsage.getEffectiveUsed();
}
-
+
+ if (totalCapacity <= 0) {
Review Comment:
To be consistent, if (totalCapacity == 0), let's return 0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15275. DiskBalancer getIdealUsage handles zero capacity incorrectly. [ozone]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on code in PR #10267:
URL: https://github.com/apache/ozone/pull/10267#discussion_r3245512923
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerVolumeCalculation.java:
##########
@@ -70,13 +70,22 @@ public static List<VolumeFixedUsage> getVolumeUsages(MutableVolumeSet volumeSet,
* @throws IllegalArgumentException if total capacity is zero
*/
public static double getIdealUsage(List<VolumeFixedUsage> volumes) {
+ if (volumes == null || volumes.isEmpty()) {
+ return 0.0;
+ }
+
long totalCapacity = 0L, totalEffectiveUsed = 0L;
-
+
for (VolumeFixedUsage volumeUsage : volumes) {
totalCapacity += volumeUsage.getUsage().getCapacity();
totalEffectiveUsed += volumeUsage.getEffectiveUsed();
Review Comment:
@szetszwo Thanks for the suggestion! Addressed by validating each volume’s capacity and effective used before calculating ideal usage, and added tests for the new edge cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org