You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/09/23 07:01:06 UTC
[GitHub] [ozone] JacksonYao287 opened a new pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
JacksonYao287 opened a new pull request #2676:
URL: https://github.com/apache/ozone/pull/2676
## What changes were proposed in this pull request?
when i test container balancer in k8s cluster, i use the command line :
./ozone admin containerbalancer start -i -1 -t 0.000001 -d 1 -s 500, and set the contaienr size to 1G.
i found that balancer thread can not stop when the cluster is close to balance.
in my cluster , i have three datanodes d1,d2,d3(disk usage are 67G, 67G, 68G), and the fourth d4 datanode's disk usage is 1G. when i start balancer, it begin to balance and work well, many containers have been moved from d1,d2,d3 to d4. but when the cluster is close to balance(the disk usages of the four datanodes are 50G,52G,50G,51G), the balancer is still running , it will move container among those datanodes again and again.
let us see a general example. if we have two datanode d1 and d2, the disk usage are 3G and 7G respectively, then the average usage is 5G. if we set the threshold to 0.00001(close to 0), the lowerLimit and upperLimit is close to 5G. if we set the container size to 4G, in the first iteraion, we can recognize the over-utilized datanode d2 and the under-utilized datanode d1, then we schedule a move option from d2 to d1 with a 4G container. after move option is finished , d1 is 7G and d2 is 3G. so the balancer thread will go on for ever.
so we should let the balancer thread exit when the cluster can not be more balanced
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5757
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 edited a comment on pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 edited a comment on pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#issuecomment-927785016
> I feel we can add extra check in `canSizeEnterTarget` function in ContainerBalancer. That might be sufficient for what we need.
@lokeshj1703 make sense , thanks for the review! i will do this in new commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r718176413
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -187,7 +190,13 @@ private void balance() {
stop();
return;
}
- doIteration();
+
+ //if no new move option is generated, it means the cluster can
+ //not be balanced any more , so just stop
+ if (doIteration() == IterationResult.CAN_NOT_BALANCE_ANY_MORE) {
Review comment:
sure, will do this in a new commit
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -652,8 +670,24 @@ private double ratioToBytes(Long nodeCapacity, double utilizationRatio) {
*/
boolean canSizeEnterTarget(DatanodeDetails target, long size) {
if (sizeEnteringNode.containsKey(target)) {
- return sizeEnteringNode.get(target) + size <=
- config.getMaxSizeEnteringTarget();
+ long sizeEnteringAfterMove = sizeEnteringNode.get(target) + size;
+ SCMNodeMetric scmNM = nodeManager.getNodeStat(target);
+ Preconditions.checkNotNull(scmNM);
+ SCMNodeStat scmNodeStat = scmNM.get();
+ double capacity = scmNodeStat.getCapacity().get();
+ Preconditions.checkArgument(capacity > 0);
+
+ //size can be moved into target datanode only when the following
+ //two condition are met.
+ //1 sizeEnteringAfterMove does not succeed the configured
+ // MaxSizeEnteringTarget
+ //2 current usage of target datanode plus sizeEnteringAfterMove
+ // is smaller than or equal to upperLimit
+ double usedAfterMove =
+ capacity - scmNodeStat.getRemaining().get() + sizeEnteringAfterMove;
+
Review comment:
yea , i have updated this patch , PTAL!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#issuecomment-925555203
@lokeshj1703 @siddhantsangwan PTAL, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#issuecomment-932282210
@siddhantsangwan @lokeshj1703 thank you for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r718176413
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -187,7 +190,13 @@ private void balance() {
stop();
return;
}
- doIteration();
+
+ //if no new move option is generated, it means the cluster can
+ //not be balanced any more , so just stop
+ if (doIteration() == IterationResult.CAN_NOT_BALANCE_ANY_MORE) {
Review comment:
sure, will do this in a new commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r718201671
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -652,8 +670,24 @@ private double ratioToBytes(Long nodeCapacity, double utilizationRatio) {
*/
boolean canSizeEnterTarget(DatanodeDetails target, long size) {
if (sizeEnteringNode.containsKey(target)) {
- return sizeEnteringNode.get(target) + size <=
- config.getMaxSizeEnteringTarget();
+ long sizeEnteringAfterMove = sizeEnteringNode.get(target) + size;
+ SCMNodeMetric scmNM = nodeManager.getNodeStat(target);
+ Preconditions.checkNotNull(scmNM);
+ SCMNodeStat scmNodeStat = scmNM.get();
+ double capacity = scmNodeStat.getCapacity().get();
+ Preconditions.checkArgument(capacity > 0);
+
+ //size can be moved into target datanode only when the following
+ //two condition are met.
+ //1 sizeEnteringAfterMove does not succeed the configured
+ // MaxSizeEnteringTarget
+ //2 current usage of target datanode plus sizeEnteringAfterMove
+ // is smaller than or equal to upperLimit
+ double usedAfterMove =
+ capacity - scmNodeStat.getRemaining().get() + sizeEnteringAfterMove;
+
Review comment:
yea , i have updated this patch , PTAL!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r718157577
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -652,8 +670,24 @@ private double ratioToBytes(Long nodeCapacity, double utilizationRatio) {
*/
boolean canSizeEnterTarget(DatanodeDetails target, long size) {
if (sizeEnteringNode.containsKey(target)) {
- return sizeEnteringNode.get(target) + size <=
- config.getMaxSizeEnteringTarget();
+ long sizeEnteringAfterMove = sizeEnteringNode.get(target) + size;
+ SCMNodeMetric scmNM = nodeManager.getNodeStat(target);
+ Preconditions.checkNotNull(scmNM);
+ SCMNodeStat scmNodeStat = scmNM.get();
+ double capacity = scmNodeStat.getCapacity().get();
+ Preconditions.checkArgument(capacity > 0);
+
+ //size can be moved into target datanode only when the following
+ //two condition are met.
+ //1 sizeEnteringAfterMove does not succeed the configured
+ // MaxSizeEnteringTarget
+ //2 current usage of target datanode plus sizeEnteringAfterMove
+ // is smaller than or equal to upperLimit
+ double usedAfterMove =
+ capacity - scmNodeStat.getRemaining().get() + sizeEnteringAfterMove;
+
Review comment:
`DatanodeUsageInfo.calculateUtilization`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] siddhantsangwan commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r718157577
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -652,8 +670,24 @@ private double ratioToBytes(Long nodeCapacity, double utilizationRatio) {
*/
boolean canSizeEnterTarget(DatanodeDetails target, long size) {
if (sizeEnteringNode.containsKey(target)) {
- return sizeEnteringNode.get(target) + size <=
- config.getMaxSizeEnteringTarget();
+ long sizeEnteringAfterMove = sizeEnteringNode.get(target) + size;
+ SCMNodeMetric scmNM = nodeManager.getNodeStat(target);
+ Preconditions.checkNotNull(scmNM);
+ SCMNodeStat scmNodeStat = scmNM.get();
+ double capacity = scmNodeStat.getCapacity().get();
+ Preconditions.checkArgument(capacity > 0);
+
+ //size can be moved into target datanode only when the following
+ //two condition are met.
+ //1 sizeEnteringAfterMove does not succeed the configured
+ // MaxSizeEnteringTarget
+ //2 current usage of target datanode plus sizeEnteringAfterMove
+ // is smaller than or equal to upperLimit
+ double usedAfterMove =
+ capacity - scmNodeStat.getRemaining().get() + sizeEnteringAfterMove;
+
Review comment:
`DatanodeUsageInfo.calculateUtilization`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#issuecomment-927785016
> I feel we can add extra check in `canSizeEnterTarget` function in ContainerBalancer. That might be sufficient for what we need.
@lokeshj1703 make sense , thanks for the review! i will do this in new commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#issuecomment-928995928
thanks @siddhantsangwan for the review , i have add more documentation in the new commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 merged pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
lokeshj1703 merged pull request #2676:
URL: https://github.com/apache/ozone/pull/2676
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#issuecomment-928995928
thanks @siddhantsangwan for the review , i have add more documentation in the new commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r717485960
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -652,8 +670,24 @@ private double ratioToBytes(Long nodeCapacity, double utilizationRatio) {
*/
boolean canSizeEnterTarget(DatanodeDetails target, long size) {
if (sizeEnteringNode.containsKey(target)) {
- return sizeEnteringNode.get(target) + size <=
- config.getMaxSizeEnteringTarget();
+ long sizeEnteringAfterMove = sizeEnteringNode.get(target) + size;
+ SCMNodeMetric scmNM = nodeManager.getNodeStat(target);
+ Preconditions.checkNotNull(scmNM);
+ SCMNodeStat scmNodeStat = scmNM.get();
+ double capacity = scmNodeStat.getCapacity().get();
+ Preconditions.checkArgument(capacity > 0);
+
+ //size can be moved into target datanode only when the following
+ //two condition are met.
+ //1 sizeEnteringAfterMove does not succeed the configured
+ // MaxSizeEnteringTarget
+ //2 current usage of target datanode plus sizeEnteringAfterMove
+ // is smaller than or equal to upperLimit
+ double usedAfterMove =
+ capacity - scmNodeStat.getRemaining().get() + sizeEnteringAfterMove;
+
Review comment:
Can we move this calculation in a separate Utils class or sth like that? @siddhantsangwan had added a function in DatanodeUsageInfo I think?
I feel we can reuse the same function. We will only need a parameter for used size to add.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -187,7 +190,13 @@ private void balance() {
stop();
return;
}
- doIteration();
+
+ //if no new move option is generated, it means the cluster can
+ //not be balanced any more , so just stop
+ if (doIteration() == IterationResult.CAN_NOT_BALANCE_ANY_MORE) {
Review comment:
Can we call the iteration function in a separate line for better visibility?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r717485960
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -652,8 +670,24 @@ private double ratioToBytes(Long nodeCapacity, double utilizationRatio) {
*/
boolean canSizeEnterTarget(DatanodeDetails target, long size) {
if (sizeEnteringNode.containsKey(target)) {
- return sizeEnteringNode.get(target) + size <=
- config.getMaxSizeEnteringTarget();
+ long sizeEnteringAfterMove = sizeEnteringNode.get(target) + size;
+ SCMNodeMetric scmNM = nodeManager.getNodeStat(target);
+ Preconditions.checkNotNull(scmNM);
+ SCMNodeStat scmNodeStat = scmNM.get();
+ double capacity = scmNodeStat.getCapacity().get();
+ Preconditions.checkArgument(capacity > 0);
+
+ //size can be moved into target datanode only when the following
+ //two condition are met.
+ //1 sizeEnteringAfterMove does not succeed the configured
+ // MaxSizeEnteringTarget
+ //2 current usage of target datanode plus sizeEnteringAfterMove
+ // is smaller than or equal to upperLimit
+ double usedAfterMove =
+ capacity - scmNodeStat.getRemaining().get() + sizeEnteringAfterMove;
+
Review comment:
Can we move this calculation in a separate Utils class or sth like that? @siddhantsangwan had added a function in DatanodeUsageInfo I think?
I feel we can reuse the same function. We will only need a parameter for used size to add.
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java
##########
@@ -187,7 +190,13 @@ private void balance() {
stop();
return;
}
- doIteration();
+
+ //if no new move option is generated, it means the cluster can
+ //not be balanced any more , so just stop
+ if (doIteration() == IterationResult.CAN_NOT_BALANCE_ANY_MORE) {
Review comment:
Can we call the iteration function in a separate line for better visibility?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
JacksonYao287 commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r719314416
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeUsageInfo.java
##########
@@ -60,25 +61,27 @@ private static int compareByUtilization(DatanodeUsageInfo first,
if (first.equals(second)) {
return 0;
}
- return Double.compare(first.calculateUtilization(),
- second.calculateUtilization());
+ return Double.compare(first.calculateUtilization(0),
+ second.calculateUtilization(0));
}
/**
- * Calculates utilization of a datanode. Utilization of a datanode is defined
- * as its used space divided by its capacity. Here, we prefer calculating
+ * Calculates utilization of a datanode after adding a specified size.
+ * Utilization of a datanode is defined as its used space divided
+ * by its capacity. Here, we prefer calculating
* used space as (capacity - remaining), instead of using
* {@link SCMNodeStat#getScmUsed()} (see HDDS-5728).
*
+ * @param plusSize the increased size
* @return (capacity - remaining) / capacity of this datanode
*/
- public double calculateUtilization() {
- double capacity = scmNodeStat.getCapacity().get();
+ public double calculateUtilization(long plusSize) {
Review comment:
sure, i will do this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] lokeshj1703 commented on a change in pull request #2676: HDDS-5757. balancer should stop when the cluster can not be balanced any more
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on a change in pull request #2676:
URL: https://github.com/apache/ozone/pull/2676#discussion_r719181094
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeUsageInfo.java
##########
@@ -60,25 +61,27 @@ private static int compareByUtilization(DatanodeUsageInfo first,
if (first.equals(second)) {
return 0;
}
- return Double.compare(first.calculateUtilization(),
- second.calculateUtilization());
+ return Double.compare(first.calculateUtilization(0),
+ second.calculateUtilization(0));
}
/**
- * Calculates utilization of a datanode. Utilization of a datanode is defined
- * as its used space divided by its capacity. Here, we prefer calculating
+ * Calculates utilization of a datanode after adding a specified size.
+ * Utilization of a datanode is defined as its used space divided
+ * by its capacity. Here, we prefer calculating
* used space as (capacity - remaining), instead of using
* {@link SCMNodeStat#getScmUsed()} (see HDDS-5728).
*
+ * @param plusSize the increased size
* @return (capacity - remaining) / capacity of this datanode
*/
- public double calculateUtilization() {
- double capacity = scmNodeStat.getCapacity().get();
+ public double calculateUtilization(long plusSize) {
Review comment:
NIT: Let's have another function `calculateUtilization()` which calls this function with 0 as parameter. Then we can keep using the older function in other places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org