You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "kerneltime (via GitHub)" <gi...@apache.org> on 2023/05/04 06:36:32 UTC

[GitHub] [ozone] kerneltime opened a new pull request, #4650: HDDS-8289: Speed up FSO ListKeys, skip skipToFirst

kerneltime opened a new pull request, #4650:
URL: https://github.com/apache/ozone/pull/4650

   
   ## What changes were proposed in this pull request?
   When listing keys in FSO, there is no need to seek first if the prefix is provided.
   
   Constructor for RDB Store seeks first if the prefix is not set. ListKeys calls in FSO should forward the prefix to the constructor to avoid wastefully seeks.
   
   ```
     public RDBStoreIterator(ManagedRocksIterator iterator, RDBTable table,
         byte[] prefix) {
       this.rocksDBIterator = iterator;
       this.rocksDBTable = table;
       if (prefix != null) {
         this.prefix = Arrays.copyOf(prefix, prefix.length);
       }
       seekToFirst();
     }
   ```
   
   This is related to https://issues.apache.org/jira/browse/HDDS-8289 but not be sufficient to improve the entire performance issue but should help.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-8289
   
   ## How was this patch tested?
   
   Existing tests should cover this.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] duongkame merged pull request #4650: HDDS-8289. Speed up FSO ListKeys, skip skipToFirst

Posted by "duongkame (via GitHub)" <gi...@apache.org>.
duongkame merged PR #4650:
URL: https://github.com/apache/ozone/pull/4650


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] kerneltime commented on pull request #4650: HDDS-8289: Speed up FSO ListKeys, skip skipToFirst

Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #4650:
URL: https://github.com/apache/ozone/pull/4650#issuecomment-1534161720

   @tanvipenumudy can you please review this as well?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] kerneltime commented on a diff in pull request #4650: HDDS-8289. Speed up FSO ListKeys, skip skipToFirst

Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on code in PR #4650:
URL: https://github.com/apache/ozone/pull/4650#discussion_r1187707635


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneListStatusHelper.java:
##########
@@ -340,7 +340,7 @@ public OzoneFileStatus getStatus(String prefixPath, long scmBlockSize,
             String prefixKey, String startKey) throws IOException {
       this.iterType = iterType;
       this.table = table;
-      this.tableIterator = table.iterator();
+      this.tableIterator = table.iterator(prefixKey);

Review Comment:
   Thanks @duongkame 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] Xushaohong commented on a diff in pull request #4650: HDDS-8289. Speed up FSO ListKeys, skip skipToFirst

Posted by "Xushaohong (via GitHub)" <gi...@apache.org>.
Xushaohong commented on code in PR #4650:
URL: https://github.com/apache/ozone/pull/4650#discussion_r1185773199


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneListStatusHelper.java:
##########
@@ -340,7 +340,7 @@ public OzoneFileStatus getStatus(String prefixPath, long scmBlockSize,
             String prefixKey, String startKey) throws IOException {
       this.iterType = iterType;
       this.table = table;
-      this.tableIterator = table.iterator();
+      this.tableIterator = table.iterator(prefixKey);

Review Comment:
   +1. These seem unnecessary now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] duongkame commented on a diff in pull request #4650: HDDS-8289. Speed up FSO ListKeys, skip skipToFirst

Posted by "duongkame (via GitHub)" <gi...@apache.org>.
duongkame commented on code in PR #4650:
URL: https://github.com/apache/ozone/pull/4650#discussion_r1185598693


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneListStatusHelper.java:
##########
@@ -340,7 +340,7 @@ public OzoneFileStatus getStatus(String prefixPath, long scmBlockSize,
             String prefixKey, String startKey) throws IOException {
       this.iterType = iterType;
       this.table = table;
-      this.tableIterator = table.iterator();
+      this.tableIterator = table.iterator(prefixKey);

Review Comment:
   Does this mean we no longer need the path in the next couple lines.
   ```
         if (!StringUtils.isBlank(prefixKey)) {
           tableIterator.seek(prefixKey);
         }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org