You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "devmadhuu (via GitHub)" <gi...@apache.org> on 2025/06/23 12:54:30 UTC

[PR] HDDS-13324. Ozone Recon - Optimize memory footprint for listKeys API. [ozone]

devmadhuu opened a new pull request, #8680:
URL: https://github.com/apache/ozone/pull/8680

   ## What changes were proposed in this pull request?
   Optimize memory footprint for listKeys API.
   While constructing full path prefix in constructFullPathPrefix method, below code can be expensive:
   
   `fullPath.insert(0, nsSummary.getDirName() + OM_KEY_PREFIX);`
   Inserting at position 0 in a StringBuilder has O(n) cost because it must shift the entire buffer every time.
   For deeply nested paths (e.g., /a/b/c/d/e/f/g/...), this compounds quickly.
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/HDDS-13324
   
   ## How was this patch tested?
   Tested manually and with existing junit and integration test cases.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13324. Ozone Recon - Optimize memory footprint for listKeys API. [ozone]

Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel commented on code in PR #8680:
URL: https://github.com/apache/ozone/pull/8680#discussion_r2161746631


##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconUtils.java:
##########
@@ -265,26 +266,35 @@ public static StringBuilder constructFullPathPrefix(long initialParentId, String
       }
       // On the last pass, dir-name will be empty and parent will be zero, indicating the loop should end.
       if (!nsSummary.getDirName().isEmpty()) {
-        fullPath.insert(0, nsSummary.getDirName() + OM_KEY_PREFIX);
+        pathSegments.add(nsSummary.getDirName());
       }
 
       // Move to the parent ID of the current directory
       parentId = nsSummary.getParentId();
       isDirectoryPresent = true;
     }
 
-    // Prepend the volume and bucket to the constructed path
-    fullPath.insert(0, volumeName + OM_KEY_PREFIX + bucketName + OM_KEY_PREFIX);
+    Collections.reverse(pathSegments);

Review Comment:
   We can save the reverse and just iterate the array backwards:
   
   ```
   for(int counter=pathSegments.length-1; i >= 0; i--){
   
   }    
   ```    



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13324. Optimize memory footprint for Recon listKeys API [ozone]

Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #8680:
URL: https://github.com/apache/ozone/pull/8680#discussion_r2161881359


##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconUtils.java:
##########
@@ -265,26 +266,35 @@ public static StringBuilder constructFullPathPrefix(long initialParentId, String
       }
       // On the last pass, dir-name will be empty and parent will be zero, indicating the loop should end.
       if (!nsSummary.getDirName().isEmpty()) {
-        fullPath.insert(0, nsSummary.getDirName() + OM_KEY_PREFIX);
+        pathSegments.add(nsSummary.getDirName());
       }
 
       // Move to the parent ID of the current directory
       parentId = nsSummary.getParentId();
       isDirectoryPresent = true;
     }
 
-    // Prepend the volume and bucket to the constructed path
-    fullPath.insert(0, volumeName + OM_KEY_PREFIX + bucketName + OM_KEY_PREFIX);
+    Collections.reverse(pathSegments);

Review Comment:
   Ok



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13324. Optimize memory footprint for Recon listKeys API [ozone]

Posted by "sodonnel (via GitHub)" <gi...@apache.org>.
sodonnel merged PR #8680:
URL: https://github.com/apache/ozone/pull/8680


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org