You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "rich7420 (via GitHub)" <gi...@apache.org> on 2026/08/25 14:50:13 UTC
[PR] HDDS-16279. Cache replica index map in Pipeline [ozone]
rich7420 opened a new pull request, #11114:
URL: https://github.com/apache/ozone/pull/11114
## What changes were proposed in this pull request?
`Pipeline.getReplicaIndexes()` re-derived the `{node -> replicaIndex}` map (stream + `toMap` + boxing) on every call, on the per-block-read command-build path. It is now memoized in a lazy `volatile` field and returned as an unmodifiable view. `reportDatanode()` is the only method that can change the node set, so it drops the cached view. Same contents (all callers only read the map), so behavior is unchanged.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16279
## How was this patch tested?
New `TestPipeline#getReplicaIndexesIsMemoizedAndInvalidatedOnReport` covers the derived contents, the unmodifiable view, memoization, and cache invalidation after `reportDatanode`. Existing `TestPipeline` (9/9) and checkstyle, pmd, findbugs are green on the fork CI.
JMH (out of tree, isolating `getReplicaIndexes()`), JDK 21:
| nodes | ns/op (old -> new) | B/op (old -> new) |
| --- | --- | --- |
| 3 (RATIS) | 54.2 -> 0.47 | 416 -> ~0 |
| 5 (EC) | 66.0 -> 0.46 | 480 -> ~0 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16279. Cache replica index map in Pipeline [ozone]
Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on code in PR #11114:
URL: https://github.com/apache/ozone/pull/11114#discussion_r3878615240
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java:
##########
@@ -245,7 +247,13 @@ public int getReplicaIndex(DatanodeDetails dn) {
* Get the replicaIndex Map.
*/
public Map<DatanodeDetails, Integer> getReplicaIndexes() {
- return this.getNodes().stream().collect(Collectors.toMap(Function.identity(), this::getReplicaIndex));
+ Map<DatanodeDetails, Integer> view = replicaIndexesView;
+ if (view == null) {
+ view = Collections.unmodifiableMap(getNodes().stream()
+ .collect(Collectors.toMap(Function.identity(), this::getReplicaIndex)));
Review Comment:
Goood catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16279. Cache replica index map in Pipeline [ozone]
Posted by "echonesis (via GitHub)" <gi...@apache.org>.
echonesis commented on code in PR #11114:
URL: https://github.com/apache/ozone/pull/11114#discussion_r3877590833
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java:
##########
@@ -245,7 +247,13 @@ public int getReplicaIndex(DatanodeDetails dn) {
* Get the replicaIndex Map.
*/
public Map<DatanodeDetails, Integer> getReplicaIndexes() {
- return this.getNodes().stream().collect(Collectors.toMap(Function.identity(), this::getReplicaIndex));
+ Map<DatanodeDetails, Integer> view = replicaIndexesView;
+ if (view == null) {
+ view = Collections.unmodifiableMap(getNodes().stream()
+ .collect(Collectors.toMap(Function.identity(), this::getReplicaIndex)));
Review Comment:
nit: `ImmutableMap` is already imported. Using ImmutableMap.toImmutableMap(Function.identity(), this::getReplicaIndex) gives an inherently immutable, single-object result with no unmodifiableMap wrapper and makes immutability explicit in the type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16279. Cache replica index map in Pipeline [ozone]
Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 closed pull request #11114: HDDS-16279. Cache replica index map in Pipeline
URL: https://github.com/apache/ozone/pull/11114
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org