You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/12/12 09:08:45 UTC

[GitHub] [ozone] siddhantsangwan opened a new pull request, #4067: HDDS-7605. Improve logging in Container Balancer

siddhantsangwan opened a new pull request, #4067:
URL: https://github.com/apache/ozone/pull/4067

   ## What changes were proposed in this pull request?
   
   Improve logging in `AbstractFindTargetGreedy` and `FindSourceGreedy`.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-7605
   
   ## How was this patch tested?
   
   Existing tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] kerneltime commented on a diff in pull request #4067: HDDS-7605. Improve logging in Container Balancer

Posted by GitBox <gi...@apache.org>.
kerneltime commented on code in PR #4067:
URL: https://github.com/apache/ozone/pull/4067#discussion_r1050498141


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/AbstractFindTargetGreedy.java:
##########
@@ -185,10 +192,26 @@ private boolean canSizeEnterTarget(DatanodeDetails target, long size) {
       // MaxSizeEnteringTarget
       //2 current usage of target datanode plus sizeEnteringAfterMove
       // is smaller than or equal to upperLimit
-      return sizeEnteringAfterMove <= config.getMaxSizeEnteringTarget() &&
-          Double.compare(nodeManager.getUsageInfo(target)
-              .calculateUtilization(sizeEnteringAfterMove), upperLimit) <= 0;
+      if (sizeEnteringAfterMove > config.getMaxSizeEnteringTarget()) {
+        logger.debug("{} bytes cannot enter datanode {} because 'size" +
+                ".entering.target.max' limit is {} and {} bytes have already " +
+                "entered.", size, target.getUuidString(),
+            config.getMaxSizeEnteringTarget(),
+            sizeEnteringNode.get(target));
+        return false;
+      }
+      if (Double.compare(nodeManager.getUsageInfo(target)
+          .calculateUtilization(sizeEnteringAfterMove), upperLimit) > 0) {
+        logger.debug("{} bytes cannot enter datanode {} because its " +
+                "utilization will exceed the upper limit of {}.", size,
+            target.getUuidString(), upperLimit);
+        return false;
+      }
+      return true;
     }
+
+    logger.warn("No record of how much size has entered datanode {}",

Review Comment:
   Is this `warn` worth? or should should it be a `debug` or `info`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] kerneltime commented on pull request #4067: HDDS-7605. Improve logging in Container Balancer

Posted by GitBox <gi...@apache.org>.
kerneltime commented on PR #4067:
URL: https://github.com/apache/ozone/pull/4067#issuecomment-1346901543

   @SaketaChalamchala @swamirishi @aswinshakil can you'll please take a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] siddhantsangwan commented on a diff in pull request #4067: HDDS-7605. Improve logging in Container Balancer

Posted by GitBox <gi...@apache.org>.
siddhantsangwan commented on code in PR #4067:
URL: https://github.com/apache/ozone/pull/4067#discussion_r1050515672


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/AbstractFindTargetGreedy.java:
##########
@@ -185,10 +192,26 @@ private boolean canSizeEnterTarget(DatanodeDetails target, long size) {
       // MaxSizeEnteringTarget
       //2 current usage of target datanode plus sizeEnteringAfterMove
       // is smaller than or equal to upperLimit
-      return sizeEnteringAfterMove <= config.getMaxSizeEnteringTarget() &&
-          Double.compare(nodeManager.getUsageInfo(target)
-              .calculateUtilization(sizeEnteringAfterMove), upperLimit) <= 0;
+      if (sizeEnteringAfterMove > config.getMaxSizeEnteringTarget()) {
+        logger.debug("{} bytes cannot enter datanode {} because 'size" +
+                ".entering.target.max' limit is {} and {} bytes have already " +
+                "entered.", size, target.getUuidString(),
+            config.getMaxSizeEnteringTarget(),
+            sizeEnteringNode.get(target));
+        return false;
+      }
+      if (Double.compare(nodeManager.getUsageInfo(target)
+          .calculateUtilization(sizeEnteringAfterMove), upperLimit) > 0) {
+        logger.debug("{} bytes cannot enter datanode {} because its " +
+                "utilization will exceed the upper limit of {}.", size,
+            target.getUuidString(), upperLimit);
+        return false;
+      }
+      return true;
     }
+
+    logger.warn("No record of how much size has entered datanode {}",

Review Comment:
   I don't have a strong opinion on this. When searching logs, if this warning shows up then I know something is wrong. It can be `info` too, but `debug` won't be much help.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] sodonnel merged pull request #4067: HDDS-7605. Improve logging in Container Balancer

Posted by GitBox <gi...@apache.org>.
sodonnel merged PR #4067:
URL: https://github.com/apache/ozone/pull/4067


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org