You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "sodonnel (via GitHub)" <gi...@apache.org> on 2023/08/14 11:05:49 UTC

[GitHub] [ozone] sodonnel opened a new pull request, #5184: HDDS-9167. Improve debug logging in SCMCommonPlacementPolicy when validating nodes

sodonnel opened a new pull request, #5184:
URL: https://github.com/apache/ozone/pull/5184

   ## What changes were proposed in this pull request?
   
   The current debug logs leave some ambiguity about why a node was not selected (data space, meta space, or the node is not writable). This Jira is to improve the debug logging to make it more clear why the node was not selected.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-9167
   
   ## How was this patch tested?
   
   Logging only change. Existing CI should check this.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] sodonnel merged pull request #5184: HDDS-9167. Improve debug logging in SCMCommonPlacementPolicy when validating nodes

Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel merged PR #5184:
URL: https://github.com/apache/ozone/pull/5184


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] siddhantsangwan commented on a diff in pull request #5184: HDDS-9167. Improve debug logging in SCMCommonPlacementPolicy when validating nodes

Posted by "siddhantsangwan (via GitHub)" <gi...@apache.org>.
siddhantsangwan commented on code in PR #5184:
URL: https://github.com/apache/ozone/pull/5184#discussion_r1293431568


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/SCMCommonPlacementPolicy.java:
##########
@@ -481,16 +486,20 @@ public boolean isValidNode(DatanodeDetails datanodeDetails,
     if (datanodeInfo == null) {
       LOG.error("Failed to find the DatanodeInfo for datanode {}",
           datanodeDetails);
-    } else {
-      if (datanodeInfo.getNodeStatus().isNodeWritable() &&
-          (hasEnoughSpace(datanodeInfo, metadataSizeRequired,
-              dataSizeRequired))) {
-        LOG.debug("Datanode {} is chosen. Required metadata size is {} and " +
-                "required data size is {}",
-            datanodeDetails, metadataSizeRequired, dataSizeRequired);
-        return true;
-      }
+      return false;
+    }
+    NodeStatus nodeStatus = datanodeInfo.getNodeStatus();
+    if (nodeStatus.isNodeWritable() &&
+        (hasEnoughSpace(datanodeInfo, metadataSizeRequired,
+            dataSizeRequired))) {
+      LOG.debug("Datanode {} is chosen. Required metadata size is {} and " +
+              "required data size is {} and NodeStatus is {}",
+          datanodeDetails, metadataSizeRequired, dataSizeRequired, nodeStatus);
+      return true;
     }
+    LOG.debug("Datanode {} is not chosen. Required metadata size is {} and " +

Review Comment:
   NIT: Do we need to wrap this one and line 495 above with a `LOG.isDebugEnabled()` check? `toString()` for `NodeStatus` does string concatenation using `+`. I suspect the compiler optimises this by using a `StringBuilder`, but not sure.
   
   ```
     public String toString() {
       return "OperationalState: " + operationalState + " Health: " + health +
           " OperationStateExpiry: " + opStateExpiryEpochSeconds;
     }
   ```
   It's only using existing variables so I don't think this is an expensive computation anyway.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] sodonnel commented on a diff in pull request #5184: HDDS-9167. Improve debug logging in SCMCommonPlacementPolicy when validating nodes

Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel commented on code in PR #5184:
URL: https://github.com/apache/ozone/pull/5184#discussion_r1293466204


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/SCMCommonPlacementPolicy.java:
##########
@@ -481,16 +486,20 @@ public boolean isValidNode(DatanodeDetails datanodeDetails,
     if (datanodeInfo == null) {
       LOG.error("Failed to find the DatanodeInfo for datanode {}",
           datanodeDetails);
-    } else {
-      if (datanodeInfo.getNodeStatus().isNodeWritable() &&
-          (hasEnoughSpace(datanodeInfo, metadataSizeRequired,
-              dataSizeRequired))) {
-        LOG.debug("Datanode {} is chosen. Required metadata size is {} and " +
-                "required data size is {}",
-            datanodeDetails, metadataSizeRequired, dataSizeRequired);
-        return true;
-      }
+      return false;
+    }
+    NodeStatus nodeStatus = datanodeInfo.getNodeStatus();
+    if (nodeStatus.isNodeWritable() &&
+        (hasEnoughSpace(datanodeInfo, metadataSizeRequired,
+            dataSizeRequired))) {
+      LOG.debug("Datanode {} is chosen. Required metadata size is {} and " +
+              "required data size is {} and NodeStatus is {}",
+          datanodeDetails, metadataSizeRequired, dataSizeRequired, nodeStatus);
+      return true;
     }
+    LOG.debug("Datanode {} is not chosen. Required metadata size is {} and " +

Review Comment:
   Inside the LOG.debug method, it will only call the toString() method if the log is going to be produced. At parameter passing time, just the reference to the object is passed, and it should be passed as type Object, so no toString conversion gets done:
   
   ```  public void debug(String format, Object arg1, Object arg2) {
       if (this.logger.isDebugEnabled()) {
         FormattingTuple ft = MessageFormatter.format(format, arg1, arg2);
         this.logger.log(FQCN, Level.DEBUG, ft.getMessage(), ft.getThrowable());
       }
   
     }
   ```
   
   The place you need to use `LOG.isDebugEnabled()` is when you need to build the string and pass that built string into the logger, eg:
   
   ```
   String s = some_concatenations_or_logic_to_construct
   LOG.debug("some log {}", s);
   ```
   
   This is better written as:
   
   ```
   if (LOG.isDebugEnabled()) {
     String s = some_concatenations_or_logic_to_construct
     LOG.debug("some log {}", s);
   }
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] siddhantsangwan commented on a diff in pull request #5184: HDDS-9167. Improve debug logging in SCMCommonPlacementPolicy when validating nodes

Posted by "siddhantsangwan (via GitHub)" <gi...@apache.org>.
siddhantsangwan commented on code in PR #5184:
URL: https://github.com/apache/ozone/pull/5184#discussion_r1293431568


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/SCMCommonPlacementPolicy.java:
##########
@@ -481,16 +486,20 @@ public boolean isValidNode(DatanodeDetails datanodeDetails,
     if (datanodeInfo == null) {
       LOG.error("Failed to find the DatanodeInfo for datanode {}",
           datanodeDetails);
-    } else {
-      if (datanodeInfo.getNodeStatus().isNodeWritable() &&
-          (hasEnoughSpace(datanodeInfo, metadataSizeRequired,
-              dataSizeRequired))) {
-        LOG.debug("Datanode {} is chosen. Required metadata size is {} and " +
-                "required data size is {}",
-            datanodeDetails, metadataSizeRequired, dataSizeRequired);
-        return true;
-      }
+      return false;
+    }
+    NodeStatus nodeStatus = datanodeInfo.getNodeStatus();
+    if (nodeStatus.isNodeWritable() &&
+        (hasEnoughSpace(datanodeInfo, metadataSizeRequired,
+            dataSizeRequired))) {
+      LOG.debug("Datanode {} is chosen. Required metadata size is {} and " +
+              "required data size is {} and NodeStatus is {}",
+          datanodeDetails, metadataSizeRequired, dataSizeRequired, nodeStatus);
+      return true;
     }
+    LOG.debug("Datanode {} is not chosen. Required metadata size is {} and " +

Review Comment:
   NIT: Do we need to wrap this one and line 495 above with a `LOG.isDebugEnabled()` check? `toString()` for `NodeStatus` does string concatenation using `+`. I suspect the compiler optimises this by using a `StringBuilder`, but not sure.
   
   ```
     public String toString() {
       return "OperationalState: " + operationalState + " Health: " + health +
           " OperationStateExpiry: " + opStateExpiryEpochSeconds;
     }
   ```
   It's only using existing variables so I don't think this is an expensive computation anway tbh.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org