You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "smengcl (via GitHub)" <gi...@apache.org> on 2026/08/11 06:21:48 UTC

[PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

smengcl opened a new pull request, #8669:
URL: https://github.com/apache/hadoop/pull/8669

   ### Description of PR\n\nJira: [HADOOP-19958](https://issues.apache.org/jira/browse/HADOOP-19958)\n\nParent Jira: [HADOOP-11890](https://issues.apache.org/jira/browse/HADOOP-11890)\n\nDelegation token services use . This representation is ambiguous when the host is an IPv6 literal.\n\nThis change:\n\n- adds a shared  helper that formats IPv6 addresses as ;\n- accepts bracketed IPv6 authorities when creating socket addresses;\n- uses the shared format for delegation token services in both token-service modes;\n- verifies token service round trips when  is  or ;\n- rejects ambiguous unbracketed IPv6 authorities; and\n- preserves existing DNS and IPv4 behavior.\n\n### How was this patch tested?\n\n- \n- Result: 76 tests passed with no failures, errors, or skips.\n- \n\n### For code changes:\n\n- [x] Does the title of this PR start with the corresponding JIRA issue id\n      (e.g. 'HADOOP-17799. Your PR title ...')?\n- [ ] Object storage: Have the integration tests been executed and t
 he endpoint\n      declared according to the connector-specific documentation? *Note: Automated CI\n      testing doesn't cover all cases so manual testing with cloud storage is still\n      required.* Not applicable to this change.\n- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion\n      under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? No new dependencies are added.\n- [x] If applicable, have you updated the , ,  files?\n      No license or notice changes are required.\n\n### AI Tooling\n\nContains content generated by Codex.\n\nIf an AI tool was used:\n\n- [x] The PR includes the phrase "Contains content generated by <tool>"\n      where <tool> is the name of the AI tool used.\n- [x] My use of AI contributions follows the ASF legal policy\n      https://www.apache.org/legal/generative-tooling.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r4100668729


##########
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestSecurityUtil.java:
##########
@@ -366,6 +366,14 @@ public void testSocketAddrWithIP() {
     verifyServiceAddr(staticHost, "127.0.0.1");
   }
 
+  @Test
+  public void testSocketAddrWithIPv6() throws Exception {
+    SecurityUtil.setTokenServiceUseIp(false);
+    String host = "::1";
+    InetSocketAddress addr = NetUtils.createSocketAddr("[::1]:123");
+    verifyAddress(addr, host, InetAddress.getByName(host).getHostAddress(), 123);

Review Comment:
   Done. Static resolution keeps the IPv6 hostname stable across systems, while the test still verifies both token-service modes



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r4100680324


##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java:
##########
@@ -491,7 +491,7 @@ public static Text buildTokenService(InetSocketAddress addr) {
     } else {
       host = StringUtils.toLowerCase(addr.getHostName());
     }
-    return new Text(host + ":" + addr.getPort());
+    return new Text(NetUtils.getHostPortString(host, addr.getPort()));

Review Comment:
   Added release note in JIRA



##########
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestSecurityUtil.java:
##########
@@ -366,6 +366,14 @@ public void testSocketAddrWithIP() {
     verifyServiceAddr(staticHost, "127.0.0.1");
   }
 
+  @Test
+  public void testSocketAddrWithIPv6() throws Exception {
+    SecurityUtil.setTokenServiceUseIp(false);

Review Comment:
   Removed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r3755728581


##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java:
##########
@@ -763,10 +766,29 @@ public static String getHostname() {
    * Compose a "host:port" string from the address.
    *
    * @param addr address.
-   * @return hort port string.
+   * @return host port string.
    */
   public static String getHostPortString(InetSocketAddress addr) {
-    return addr.getHostName() + ":" + addr.getPort();
+    return getHostPortString(addr.getHostName(), addr.getPort());
+  }
+
+  /**
+   * Compose a "host:port" string, bracketing IPv6 literals.
+   *
+   * @param host host name or IP address.
+   * @param port port number.
+   * @return host port string.
+   */
+  public static String getHostPortString(String host, int port) {
+    String normalizedHost = host;
+    if (normalizedHost != null && normalizedHost.startsWith("[")
+        && normalizedHost.endsWith("]")) {
+      normalizedHost = normalizedHost.substring(1, normalizedHost.length() - 1);
+    }
+    if (normalizedHost != null && normalizedHost.contains(":")) {
+      return "[" + normalizedHost + "]:" + port;
+    }
+    return normalizedHost + ":" + port;

Review Comment:
   NetUtils.getHostPortString(host, port) now deliberately emits bracketed IPv6 literals (e.g., "[::1]:123"), but NetUtils.getPortFromHostPortString still splits on ":" and rejects anything except exactly one colon. This makes the two helpers inconsistent and will fail for any caller that needs to extract the port from a bracketed IPv6 authority.
   
   Update getPortFromHostPortString() to accept both "<host>:<port>" and "[<ipv6>]:<port>" (and explicitly reject ambiguous unbracketed IPv6).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r4100668729


##########
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestSecurityUtil.java:
##########
@@ -366,6 +366,14 @@ public void testSocketAddrWithIP() {
     verifyServiceAddr(staticHost, "127.0.0.1");
   }
 
+  @Test
+  public void testSocketAddrWithIPv6() throws Exception {
+    SecurityUtil.setTokenServiceUseIp(false);
+    String host = "::1";
+    InetSocketAddress addr = NetUtils.createSocketAddr("[::1]:123");
+    verifyAddress(addr, host, InetAddress.getByName(host).getHostAddress(), 123);

Review Comment:
   Done. Static resolution keeps the IPv6 hostname stable across systems, while the test still verifies both token-service modes



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#issuecomment-5826036064

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m  0s |  |  Docker mode activated.  |
   | -1 :x: |  patch  |   0m 19s |  |  https://github.com/apache/hadoop/pull/8669 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/4/console |
   | versions | git=2.34.1 |
   | Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#issuecomment-5249752596

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m  0s |  |  Docker mode activated.  |
   | -1 :x: |  patch  |   0m 21s |  |  https://github.com/apache/hadoop/pull/8669 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/1/console |
   | versions | git=2.34.1 |
   | Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#issuecomment-5249891811

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m  0s |  |  Docker mode activated.  |
   | -1 :x: |  patch  |   0m 22s |  |  https://github.com/apache/hadoop/pull/8669 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/2/console |
   | versions | git=2.34.1 |
   | Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r4100674416


##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java:
##########
@@ -763,10 +766,29 @@ public static String getHostname() {
    * Compose a "host:port" string from the address.
    *
    * @param addr address.
-   * @return hort port string.
+   * @return host port string.
    */
   public static String getHostPortString(InetSocketAddress addr) {
-    return addr.getHostName() + ":" + addr.getPort();
+    return getHostPortString(addr.getHostName(), addr.getPort());
+  }
+
+  /**
+   * Compose a "host:port" string, bracketing IPv6 literals.
+   *
+   * @param host host name or IP address.
+   * @param port port number.
+   * @return host port string.
+   */
+  public static String getHostPortString(String host, int port) {
+    String normalizedHost = host;
+    if (normalizedHost != null && normalizedHost.startsWith("[")
+        && normalizedHost.endsWith("]")) {
+      normalizedHost = normalizedHost.substring(1, normalizedHost.length() - 1);
+    }
+    if (normalizedHost != null && normalizedHost.contains(":")) {

Review Comment:
   Agreed, fixed in 4fe51ff



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r4100667314


##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java:
##########
@@ -763,10 +766,29 @@ public static String getHostname() {
    * Compose a "host:port" string from the address.
    *
    * @param addr address.
-   * @return hort port string.
+   * @return host port string.
    */
   public static String getHostPortString(InetSocketAddress addr) {
-    return addr.getHostName() + ":" + addr.getPort();
+    return getHostPortString(addr.getHostName(), addr.getPort());
+  }
+
+  /**
+   * Compose a "host:port" string, bracketing IPv6 literals.
+   *
+   * @param host host name or IP address.
+   * @param port port number.
+   * @return host port string.
+   */
+  public static String getHostPortString(String host, int port) {
+    String normalizedHost = host;
+    if (normalizedHost != null && normalizedHost.startsWith("[")
+        && normalizedHost.endsWith("]")) {
+      normalizedHost = normalizedHost.substring(1, normalizedHost.length() - 1);
+    }
+    if (normalizedHost != null && normalizedHost.contains(":")) {
+      return "[" + normalizedHost + "]:" + port;
+    }
+    return normalizedHost + ":" + port;

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r4100675967


##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java:
##########
@@ -491,7 +491,7 @@ public static Text buildTokenService(InetSocketAddress addr) {
     } else {
       host = StringUtils.toLowerCase(addr.getHostName());
     }
-    return new Text(host + ":" + addr.getPort());
+    return new Text(NetUtils.getHostPortString(host, addr.getPort()));

Review Comment:
   Good idea



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #8669:
URL: https://github.com/apache/hadoop/pull/8669


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#issuecomment-5447354640

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   1m  0s |  |  Docker mode activated.  |
   |||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to include 2 new or modified test files.  |
   |||| _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  41m 25s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  16m 25s |  |  trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  compile  |  16m 31s |  |  trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  checkstyle  |   1m 30s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 58s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 30s |  |  trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  javadoc  |   1m 22s |  |  trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  spotbugs  |   3m  8s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  30m 59s |  |  branch has no errors when building and testing our client artifacts.  |
   |||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 16s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  15m 19s |  |  the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  javac  |  15m 19s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  16m 20s |  |  the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  javac  |  16m 20s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 29s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   2m  1s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 28s |  |  the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  javadoc  |   1m 26s |  |  the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu  |
   | +1 :green_heart: |  spotbugs  |   3m 20s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  31m  3s |  |  patch has no errors when building and testing our client artifacts.  |
   |||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |  23m 29s |  |  hadoop-common in the patch passed.  |
   | -1 :x: |  asflicense  |   1m 15s | [/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/3/artifact/out/results-asflicense.txt) |  The patch generated 1 ASF License warnings.  |
   |  |   | 215m 38s |  |  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/3/artifact/out/Dockerfile |
   | GITHUB PR | https://github.com/apache/hadoop/pull/8669 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux e1872fb777e8 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 52966be5e42f0b701ae4c19786b2459f341827a9 |
   | Default Java | Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
   | Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu |
   |  Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/3/testReport/ |
   | Max. process+thread count | 1279 (vs. ulimit of 10000) |
   | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
   | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/3/console |
   | versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
   | Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#discussion_r3876362721


##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java:
##########
@@ -491,7 +491,7 @@ public static Text buildTokenService(InetSocketAddress addr) {
     } else {
       host = StringUtils.toLowerCase(addr.getHostName());
     }
-    return new Text(host + ":" + addr.getPort());
+    return new Text(NetUtils.getHostPortString(host, addr.getPort()));

Review Comment:
   When `hadoop.security.token.service.use_ip=true`, the service string will use whatever canonical form `InetAddress.getHostAddress()` returns (e.g. `0:0:0:0:0:0:0:1` vs `::1`). That is pre-existing JDK behavior, not introduced here, but operators enabling IPv6 should know token service strings may use the expanded form. A brief release note may help.



##########
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestSecurityUtil.java:
##########
@@ -366,6 +366,14 @@ public void testSocketAddrWithIP() {
     verifyServiceAddr(staticHost, "127.0.0.1");
   }
 
+  @Test
+  public void testSocketAddrWithIPv6() throws Exception {
+    SecurityUtil.setTokenServiceUseIp(false);

Review Comment:
   Minor: `SecurityUtil.setTokenServiceUseIp(false)` here is redundant — `verifyAddress()` already exercises both `use_ip=true` and `use_ip=false` via `verifyTokenService()`.



##########
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestSecurityUtil.java:
##########
@@ -366,6 +366,14 @@ public void testSocketAddrWithIP() {
     verifyServiceAddr(staticHost, "127.0.0.1");
   }
 
+  @Test
+  public void testSocketAddrWithIPv6() throws Exception {
+    SecurityUtil.setTokenServiceUseIp(false);
+    String host = "::1";
+    InetSocketAddress addr = NetUtils.createSocketAddr("[::1]:123");
+    verifyAddress(addr, host, InetAddress.getByName(host).getHostAddress(), 123);

Review Comment:
   `testSocketAddrWithIPv6` assumes the resolved hostname for `::1` remains `"::1"`, but on JDK 17 (verified locally) `createSocketAddr("[::1]:123")` yields `getHostName() == "localhost"`. That breaks `verifyValues()` when `use_ip=false` and hostname-mode token service expectations (`"[localhost]:123"` vs `"[::1]:123"`).
   
   The PR description says both `use_ip=true` and `use_ip=false` round-trips are verified; this test may not reliably cover hostname mode depending on `/etc/hosts` and JDK reverse-DNS behavior.
   
   Consider deriving the expected hostname from `addr.getHostName()` after creation, using `NetUtils.addStaticResolution` for a stable name (consistent with other tests in this class), or splitting explicit coverage for `use_ip=true` vs hostname mode.



##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetUtils.java:
##########
@@ -763,10 +766,29 @@ public static String getHostname() {
    * Compose a "host:port" string from the address.
    *
    * @param addr address.
-   * @return hort port string.
+   * @return host port string.
    */
   public static String getHostPortString(InetSocketAddress addr) {
-    return addr.getHostName() + ":" + addr.getPort();
+    return getHostPortString(addr.getHostName(), addr.getPort());
+  }
+
+  /**
+   * Compose a "host:port" string, bracketing IPv6 literals.
+   *
+   * @param host host name or IP address.
+   * @param port port number.
+   * @return host port string.
+   */
+  public static String getHostPortString(String host, int port) {
+    String normalizedHost = host;
+    if (normalizedHost != null && normalizedHost.startsWith("[")
+        && normalizedHost.endsWith("]")) {
+      normalizedHost = normalizedHost.substring(1, normalizedHost.length() - 1);
+    }
+    if (normalizedHost != null && normalizedHost.contains(":")) {

Review Comment:
   This new helper emits bracketed IPv6 authorities (e.g. `"[::1]:123"`), but the existing `getPortFromHostPortString()` (unchanged in this PR) still splits on `":"` and requires exactly one colon — it cannot parse bracketed IPv6 strings.
   
   Token round-trip via `SecurityUtil.getTokenServiceAddr()` → `createSocketAddr()` is fine, but the two helpers are now inconsistent. Any caller that formats with `getHostPortString` and parses with `getPortFromHostPortString` will break on IPv6.
   
   Worth updating `getPortFromHostPortString()` to accept `"[<ipv6>]:<port>"` (and reject ambiguous unbracketed IPv6), with tests, either here or as an immediate follow-up on the same JIRA.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Re: [PR] HADOOP-19958. Support IPv6 delegation token service addresses [hadoop]

Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8669:
URL: https://github.com/apache/hadoop/pull/8669#issuecomment-5900799550

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |:----:|----------:|--------:|:--------:|:-------:|
   | +0 :ok: |  reexec  |   0m  0s |  |  Docker mode activated.  |
   | -1 :x: |  patch  |   0m 25s |  |  https://github.com/apache/hadoop/pull/8669 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.  |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8669/5/console |
   | versions | git=2.34.1 |
   | Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org