You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/10/14 11:12:00 UTC
[jira] [Updated] (HDDS-16998) Skip set the lastScanned fields for Lifecycle task during cache scan
[ https://issues.apache.org/jira/browse/HDDS-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDDS-16998:
----------------------------------
Labels: pull-request-available (was: )
> Skip set the lastScanned fields for Lifecycle task during cache scan
> --------------------------------------------------------------------
>
> Key: HDDS-16998
> URL: https://issues.apache.org/jira/browse/HDDS-16998
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sammi Chen
> Assignee: rajesh
> Priority: Major
> Labels: pull-request-available
>
> In evaluateKeyAndDirTable (FSO), each directory first iterates keyTable.cacheIterator() and then the RocksDB iterator, and both set lastScannedKey. The file table uses PartialTableCache, which is a ConcurrentHashMap, so the cache pass runs in hash order and lastScannedKey can end up at an arbitrary key. If scan state is saved at that point (either the expired list fills inside the cache loop and handleAndClearFullList sends the state, or the task aborts on shouldRun() at the start of the DB loop), we persist that cache key. On resume we seek(lastScannedKey) in the DB iterator, which skips every key in that directory sorting before it. Those keys aren't evaluated until the next full scan cycle.
> Example: directory has DB keys a, b, c, x, and key m is only in cache. The cache pass sets lastScannedKey = m, the leader changes, state is saved with m, and on resume we seek to m and skip a, b, c.
> Since the cache pass doesn't seek and is fully re-evaluated on every resume anyway, the simple fix is to not update lastScannedKey in the cache loop, so the saved position only comes from the sorted DB iteration. OBS (evaluateBucket) only iterates the DB, so it isn't affected
> It's found by Chandolu Rajesh Kumar.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org