[jira] [Assigned] (HDDS-16195) Avoid recomputing block replicated sizes in the KeyDeletingService reclaimable-key scan

["Chi-Hsuan Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chi-Hsuan Huang reassigned HDDS-16195:
--------------------------------------

    Assignee: Chi-Hsuan Huang

> Avoid recomputing block replicated sizes in the KeyDeletingService reclaimable-key scan
> ---------------------------------------------------------------------------------------
>
>                 Key: HDDS-16195
>                 URL: https://issues.apache.org/jira/browse/HDDS-16195
>             Project: Apache Ozone
>          Issue Type: Improvement
>          Components: OM
>            Reporter: Chi-Hsuan Huang
>            Assignee: Chi-Hsuan Huang
>            Priority: Minor
>
> h3. Problem
> {{KeyManagerImpl}} builds the reclaimable key list for {{KeyDeletingService}} in {{KeyManagerImpl.java:853\-868}}. For every key that passes the reclaim filter it walks the key's blocks twice and computes the replicated size of each block twice:
> {code:java}
> List deletedBlocks = info.getKeyLocationVersions\(\).stream\(\)
>     .flatMap\(versionLocations \-> versionLocations.getLocationList\(\).stream\(\)
>         .map\(b \-> new DeletedBlock\(
>             new BlockID\(b.getContainerID\(\), b.getLocalID\(\)\),
>             b.getLength\(\),
>             QuotaUtil.getReplicatedSize\(b.getLength\(\), info.getReplicationConfig\(\)\),
>             QuotaUtil.getSizePerReplica\(b.getLength\(\), info.getReplicationConfig\(\)\)
>         \)\)\).collect\(Collectors.toList\(\)\);
> ...
> reclaimableKeys.put\(blockGroupName,
>     new PurgedKey\(info.getVolumeName\(\), info.getBucketName\(\), bucketId,
>     keyBlocks, kv.getKey\(\), OMKeyRequest.sumBlockLengths\(info\), info.isDeletedKeyCommitted\(\)\)\);
> {code}
> The first traversal already stores the per\-block replicated size in {{DeletedBlock}} \({{hadoop\-hdds/framework/src/main/java/org/apache/hadoop/ozone/common/DeletedBlock.java}}\), which exposes it through {{getReplicatedSize\(\)}}.
> The second traversal is {{OMKeyRequest.sumBlockLengths\(info\)}} \({{OMKeyRequest.java:887}}\), which sums {{QuotaUtil.getReplicatedSize\(locationInfo.getLength\(\), info.getReplicationConfig\(\)\)}} over the same blocks.
> Both traversals sit in the same filter branch and both start from {{info.getKeyLocationVersions\(\)}}, so they visit an identical set of block objects and apply identical arithmetic. The value {{sumBlockLengths}} recomputes is therefore already in hand:
> {code:java}
> deletedBlocks.stream\(\).mapToLong\(DeletedBlock::getReplicatedSize\).sum\(\)
> {code}
> That removes, per reclaimable key: one full traversal of every block, one {{getLocationList\(\)}} flattening copy per version group, and one {{QuotaUtil.getReplicatedSize}} call per block.
> h3. Why this is separate from HDDS\-16183
> HDDS\-16183 proposes swapping {{getLocationList\(\)}} for {{getLocationLists\(\)}} inside {{sumBlockLengths}}, which benefits every caller of that method. It does not address the duplicate traversal here, because the duplication is independent of which accessor is used. Conversely, the change proposed here removes the {{sumBlockLengths}} call from this loop altogether.
> The two changes touch different files, do not conflict, and can land in either order.
> h3. Scope
> This loop runs per pending\-delete key on every {{KeyDeletingService}} iteration, so the duplicated work repeats across the whole scan.
> h3. Notes
> Behavior is unchanged: the block set and the arithmetic are identical, so the resulting quota value is the same. Existing key deletion and quota tests cover it.
> No benchmark has been run. RocksDB iteration and {{OmKeyInfo}} deserialization plausibly dominate this loop, so this should be treated as removal of provably redundant work rather than a claimed speedup. Unlike an allocation\-shaped observation, the redundancy here is verifiable by reading the code: the two traversals demonstrably cover the same blocks.
> Pinned source commit 4766aa8609. Analysis assisted by AI tooling \(Claude Code, Opus 5\).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org