You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "smengcl (via GitHub)" <gi...@apache.org> on 2026/10/18 20:55:42 UTC
[PR] HDDS-17113. SCM to prefer a Ratis pipeline on the writer's datanode when allocating blocks [ozone]
smengcl opened a new pull request, #11617:
URL: https://github.com/apache/ozone/pull/11617
<!-- claude-projects-attribution: {"github_login":"smengcl"} -->
Generated-by: Claude Code (Opus 5.5)
## What changes were proposed in this pull request?
When a client runs on a datanode host (for example an HBase RegionServer colocated with a datanode), SCM picks a Ratis pipeline for a new block without looking at where the client is, so often no replica is local to the writer. HDFS writes the first replica on the writer's datanode, which later lets reads of that data (such as HBase compactions) use short circuit reads.
This change adds `hdds.scm.block.allocation.prefer.client.local` (default `false`). When it is set:
- OM passes the client address to SCM on every block allocation, even when OM sorts the datanodes itself.
- SCM first tries open Ratis pipelines led by the client's datanode, then other open pipelines that contain it, and falls back to the pipeline choose policy when none of those has a usable container. A pipeline led by the client's datanode also keeps the client to leader hop of each write on the same host (see HDDS-11000).
EC allocation and the default behavior are unchanged.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17113
## How was this patch tested?
- New unit tests in `TestWritableRatisContainerProvider` (prefers a pipeline led by the client, then one containing it, and falls back when the local pipeline has no container or the client is in no pipeline) and in `TestOMAllocateBlockRequest` (OM sends the client address to SCM when the flag is set).
- Existing `TestSCMBlockProtocolServer`, `TestBlockManager`, `TestOMKeyCreateRequest` and `TestOMFileCreateRequest` pass.
- On a 6 datanode docker compose cluster with the flag set, each writer running on a datanode that leads a pipeline got blocks on that pipeline. hsync latency differences between local leader, local follower and remote writers were within noise there, since all containers share one host. A real network A/B is still needed to measure the latency effect.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17113. SCM to prefer a Ratis pipeline on the writer's datanode when allocating blocks [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #11617:
URL: https://github.com/apache/ozone/pull/11617#discussion_r4308744057
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmConfig.java:
##########
@@ -102,6 +102,17 @@ public class ScmConfig extends ReconfigurableConfig {
)
private String ecPipelineChoosePolicyName;
+ @Config(key = "hdds.scm.block.allocation.prefer.client.local",
+ type = ConfigType.BOOLEAN,
+ defaultValue = "false",
+ tags = { ConfigTag.SCM, ConfigTag.PIPELINE, ConfigTag.PERFORMANCE },
+ description = "If true, when allocating a Ratis block for a client that runs on a datanode host, SCM prefers "
+ + "an open pipeline that contains that datanode, so one replica is local to the writer (like the HDFS "
+ + "first replica). Falls back to the pipeline choose policy when no such pipeline has a usable container. "
+ + "OM reads the same key to pass the client address to SCM on every block allocation."
+ )
+ private boolean preferClientLocalPipeline;
Review Comment:
Open question to reviewers: Should this be made the default?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org