[GitHub] [ozone] rakeshadr commented on pull request #3379: HDDS-6474. Fixed File System Optimized(FSO) bucket list status.

[GitBox <gi...@apache.org>]

rakeshadr commented on PR #3379:
URL: https://github.com/apache/ozone/pull/3379#issuecomment-1117489674

   Thanks a lot @aswinshakil for the continuous efforts in analysing and digging out the root cause.
   
   It looks like OM TableCache is tricky for the FSO or non-FSO listing status code path, which would create inconsistency across batches.
   
   _Please go through the following scenario:_
   
   Say, below keys exists in OM. 
   
   These keys `"x/y/z/a1"...."x/y/z/a1023"` and `"x/y/z/b1"..."x/y/z/b1023"` in DBTable. 
   Assume, `"x/y/z/b1024"` only exists in TableCache.
   
   Now, Ozone client Listing Status with keyName = `"x/y/z/"`:
   
   BatchSize = 1024. It will take two iteration to fetch all the values from OM.
   
   **Iteration-1:**
   keyName = `"x/y/z/"` and startKey = `""`
   
   step-1) Seek all the elements from TableCache if its not a deleted key. 
   Now, the Batch-1 [`"x/y/z/b1024"`]
   step-2) Seek all the elements from 
   Now, the Batch-1 [`"x/y/z/a1"...."x/y/z/a1023"`]
   
   FinalList of Batch-1 will be a combined  values : [`"x/y/z/a1"...."x/y/z/a1023, "x/y/z/b1024"`].
   
   **Iteration-2:**
   keyName = `"x/y/z/"` and startKey = `"x/y/z/b1024"`, which is the last element in the batch-1.
   FinalList of Batch-2 : Empty list because there is no elements after `"x/y/z/b1024".`
   
   Since `"x/y/z/b1"..."x/y/z/b1023"` exists before `"x/y/z/b1024"` start key, all these keys would be skipped and won't get fetched/seeks from DBTable.
   
   **Proposal:** How about to use TableCache only for checking the deleted keys and do not add the cached values into the listing keys ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org