You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "smengcl (via GitHub)" <gi...@apache.org> on 2026/10/19 02:11:25 UTC
[PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
smengcl opened a new pull request, #11619:
URL: https://github.com/apache/ozone/pull/11619
Generated-by: Claude Code (Opus 5.5)
## What changes were proposed in this pull request?
Several production paths pass a host that can be a bare IPv6 literal to `NetUtils.createSocketAddr(target, defaultPort)`, or format it into a URL without brackets. Hadoop parses `target` as a URI authority, so a bare literal such as `2001:db8::10` throws `IllegalArgumentException: Does not contain a valid host:port authority`. Bare literals reach these paths because SCM records a datanode's IP with `InetAddress.getHostAddress()` (and its hostname is the literal when there is no reverse DNS), and `NodeDetails` takes its host address from `InetSocketAddress.getHostName()`.
Affected paths:
- `NodeDetails.getRpcAddress()`: OM bootstrap (`checkRemoteOMConfig`, `addOMToRatisRing`) throws when the remote OM's host address is a bare literal.
- `SCMHANodeDetails.loadSCMHAConfig`: SCM HA fails to start with an unbracketed `ozone.scm.address.<serviceId>.<nodeId>`, although `validateAdvertisedHost` accepts it.
- `XceiverClientManager.getPipelineCacheKey` and `XceiverClientShortCircuit`: with short circuit reads enabled, every non EC read from an IPv6 datanode fails.
- `DataNodeMetricsCollectionTask` (Recon): the datanode JMX URL is malformed, so pending deletion metrics are reported as -1.
- `DiskBalancerSubCommandUtil.getSingleNodeDiskBalancerProxy`: any `:` was taken to mean "port given", so `[2001:db8::1]` and `2001:db8::1` threw instead of using the default port.
- `ozone local`: with `--host ::1`, service addresses, the Recon endpoint and the startup summary became `::1:9860`.
The fix combines host and port with `HddsUtils.getHostPortString`, and parses the DiskBalancer and SCM HA addresses with `HddsUtils.getHostName` / `getHostPort`. These bracket IPv6 literals and accept DNS names, IPv4 literals and already bracketed input. This follows the same pattern as HDDS-15773 and HDDS-16831.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17135
## How was this patch tested?
New unit tests. The bare IPv6 cases fail without the fix; the DNS, IPv4 and bracketed cases guard against regressions:
- `TestOMNodeDetails`: unbracketed IPv6 host address.
- `TestSCMConfiguration`: SCM HA peer with unbracketed and bracketed IPv6 address.
- `TestDataNodeMetricsCollectionTask` (new): JMX URL for a DNS name and an IPv6 literal over HTTP and HTTPS.
- `TestDiskBalancerSubCommandUtil`: address parsing for IPv4, bare IPv6 and bracketed IPv6, with and without a port.
- `TestLocalOzoneCluster` and `TestOzoneLocal`: service addresses, Recon endpoint and startup summary with `--host ::1`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl merged PR #11619:
URL: https://github.com/apache/ozone/pull/11619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #11619:
URL: https://github.com/apache/ozone/pull/11619#discussion_r4311738890
##########
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerSubCommandUtil.java:
##########
@@ -168,14 +168,9 @@ public static DiskBalancerProtocol getSingleNodeDiskBalancerProxy(
UserGroupInformation user = UserGroupInformation.getCurrentUser();
// Parse address and add default port if not specified
- InetSocketAddress nodeAddr;
- if (address.contains(":")) {
- // Port is specified, use NetUtils to parse
- nodeAddr = NetUtils.createSocketAddr(address);
- } else {
- // Port not specified, use default
- nodeAddr = NetUtils.createSocketAddr(address, HDDS_DATANODE_CLIENT_PORT_DEFAULT);
- }
+ int port = HddsUtils.getHostPort(address).orElse(HDDS_DATANODE_CLIENT_PORT_DEFAULT);
+ InetSocketAddress nodeAddr = NetUtils.createSocketAddr(
Review Comment:
A bare IPv6 literal followed by a port **(2001:db8::1:1234)** now parses as host `2001:db8::1:1234` with the default port, instead of failing as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #11619:
URL: https://github.com/apache/ozone/pull/11619#discussion_r4312139640
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMHANodeDetails.java:
##########
@@ -232,7 +232,8 @@ public static SCMHANodeDetails loadSCMHAConfig(OzoneConfiguration conf,
InetSocketAddress addr = null;
try {
- addr = NetUtils.createSocketAddr(rpcAddrStr, ratisPort);
+ addr = NetUtils.createSocketAddr(HddsUtils.getHostPortString(HddsUtils.getHostName(rpcAddrStr).get(),
Review Comment:
Same issue as DiskBalancer can happen here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #11619:
URL: https://github.com/apache/ozone/pull/11619#discussion_r4319246958
##########
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerSubCommandUtil.java:
##########
@@ -168,14 +168,9 @@ public static DiskBalancerProtocol getSingleNodeDiskBalancerProxy(
UserGroupInformation user = UserGroupInformation.getCurrentUser();
// Parse address and add default port if not specified
- InetSocketAddress nodeAddr;
- if (address.contains(":")) {
- // Port is specified, use NetUtils to parse
- nodeAddr = NetUtils.createSocketAddr(address);
- } else {
- // Port not specified, use default
- nodeAddr = NetUtils.createSocketAddr(address, HDDS_DATANODE_CLIENT_PORT_DEFAULT);
- }
+ int port = HddsUtils.getHostPort(address).orElse(HDDS_DATANODE_CLIENT_PORT_DEFAULT);
+ InetSocketAddress nodeAddr = NetUtils.createSocketAddr(
Review Comment:
Thanks, good catch. `getSingleNodeDiskBalancerProxy` now calls `HddsUtils.validateHostPortAuthority` (made public) before parsing, so `2001:db8::1:1234` and a bare `2001:db8::1` are rejected with a message suggesting `[2001:db8::1]:1234` or `[2001:db8::1:1234]`. Added negative cases in `TestDiskBalancerSubCommandUtil`.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
##########
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerSubCommandUtil.java:
##########
@@ -168,14 +168,9 @@ public static DiskBalancerProtocol getSingleNodeDiskBalancerProxy(
UserGroupInformation user = UserGroupInformation.getCurrentUser();
// Parse address and add default port if not specified
- InetSocketAddress nodeAddr;
- if (address.contains(":")) {
- // Port is specified, use NetUtils to parse
- nodeAddr = NetUtils.createSocketAddr(address);
- } else {
- // Port not specified, use default
- nodeAddr = NetUtils.createSocketAddr(address, HDDS_DATANODE_CLIENT_PORT_DEFAULT);
- }
+ int port = HddsUtils.getHostPort(address).orElse(HDDS_DATANODE_CLIENT_PORT_DEFAULT);
+ InetSocketAddress nodeAddr = NetUtils.createSocketAddr(
Review Comment:
Done, using the same check: `HddsUtils.validateHostPortAuthority` is the part of `validateAdvertisedAddress` that rejects unbracketed literals. I did not use `validateAdvertisedAddress` itself because its wildcard and link local checks are about advertised addresses, not a CLI target.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/ha/SCMHANodeDetails.java:
##########
@@ -232,7 +232,8 @@ public static SCMHANodeDetails loadSCMHAConfig(OzoneConfiguration conf,
InetSocketAddress addr = null;
try {
- addr = NetUtils.createSocketAddr(rpcAddrStr, ratisPort);
+ addr = NetUtils.createSocketAddr(HddsUtils.getHostPortString(HddsUtils.getHostName(rpcAddrStr).get(),
Review Comment:
I left this one unchanged. `ozone.scm.address.<serviceId>.<nodeId>` is a host only property: `SCMNodeInfo` (via `validateAdvertisedHost`) appends each SCM port to the whole value, so a bare `2001:db8::1:1234` is already read as the host `2001:db8::1:1234` there, and this path now reads it the same way. Rejecting unbracketed literals only here would make SCM HA refuse a value that clients and datanodes accept. If we want to require brackets for this property, I think it should be done in `validateAdvertisedHost` for all readers in a separate change.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #11619:
URL: https://github.com/apache/ozone/pull/11619#issuecomment-6309449187
Thanks @Gargi-jais11 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #11619:
URL: https://github.com/apache/ozone/pull/11619#discussion_r4312136600
##########
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerSubCommandUtil.java:
##########
@@ -168,14 +168,9 @@ public static DiskBalancerProtocol getSingleNodeDiskBalancerProxy(
UserGroupInformation user = UserGroupInformation.getCurrentUser();
// Parse address and add default port if not specified
- InetSocketAddress nodeAddr;
- if (address.contains(":")) {
- // Port is specified, use NetUtils to parse
- nodeAddr = NetUtils.createSocketAddr(address);
- } else {
- // Port not specified, use default
- nodeAddr = NetUtils.createSocketAddr(address, HDDS_DATANODE_CLIENT_PORT_DEFAULT);
- }
+ int port = HddsUtils.getHostPort(address).orElse(HDDS_DATANODE_CLIENT_PORT_DEFAULT);
+ InetSocketAddress nodeAddr = NetUtils.createSocketAddr(
Review Comment:
same issue is already handled for `ozone.scm.names` / `ozone.recon.address` with `HddsUtils.validateAdvertisedAddress`, which rejects unbracketed literals and tells the user which bracketed form to write. Could we use similar check here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-17135. Bracket IPv6 literals passed to NetUtils.createSocketAddr and in Recon datanode JMX URLs [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #11619:
URL: https://github.com/apache/ozone/pull/11619#issuecomment-6297812888
Thanks @Gargi-jais11 for reviewing this. Would you like to take another look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org