[jira] [Created] (YARN-12034) WebAppUtils.getResolvedAddress and RM standby redirect drop IPv6 brackets

["Siyao Meng (Jira)" <ji...@apache.org>]

Siyao Meng created YARN-12034:
---------------------------------

             Summary: WebAppUtils.getResolvedAddress and RM standby redirect drop IPv6 brackets
                 Key: YARN-12034
                 URL: https://issues.apache.org/jira/browse/YARN-12034
             Project: Hadoop YARN
          Issue Type: Bug
          Components: webapp, resourcemanager
            Reporter: Siyao Meng
            Assignee: Siyao Meng


[WebAppUtils.java:343|https://github.com/apache/hadoop/blob/1097744bc490b65cad5ab2fbeafff35687971357/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/util/WebAppUtils.java#L343] appends {{":" + port}} to {{InetSocketAddress.getHostName()}}. [RMWebAppFilter.java:98|https://github.com/apache/hadoop/blob/1097744bc490b65cad5ab2fbeafff35687971357/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebAppFilter.java#L98] and [RMWebApp.java:152|https://github.com/apache/hadoop/blob/1097744bc490b65cad5ab2fbeafff35687971357/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebApp.java#L152] do the same:
{code:java}
path = sock.getHostName() + ":" + sock.getPort();
{code}

With {{yarn.resourcemanager.webapp.address=[2001:db8::10]:8088}} and no PTR record, {{getHostName()}} returns {{2001:db8:0:0:0:0:0:10}}, so the result is {{2001:db8:0:0:0:0:0:10:8088}}. The same happens for a wildcard bind when the local host's canonical name is a literal.

Effects:
* A standby RM sends {{Location: http://2001:db8:0:0:0:0:0:10:8088/...}}, which clients cannot follow.
* Failed and killed applications get an unusable tracking URL ([RMAppAttemptImpl.java:669|https://github.com/apache/hadoop/blob/1097744bc490b65cad5ab2fbeafff35687971357/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java#L669]).
* {{yarn top}} in HA mode fails in {{new URL}} ([TopCLI.java:825|https://github.com/apache/hadoop/blob/1097744bc490b65cad5ab2fbeafff35687971357/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/TopCLI.java#L825]).
* The web proxy's RM and failure-page links are wrong.
* The per-node timeline collector address is wrong ([NodeTimelineCollectorManager.java:329|https://github.com/apache/hadoop/blob/1097744bc490b65cad5ab2fbeafff35687971357/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/collector/NodeTimelineCollectorManager.java#L329]), so {{TimelineV2Client}} builds a URI with no host. That is the symptom reported in MAPREDUCE-7429.

Verified by running a copy of {{getResolvedAddress}} on JDK 21 with a literal IPv6 address and passing the output to {{new URL}}. Not run with HA ResourceManagers.

*Proposed fix:* return {{NetUtils.getHostPortString(host, port)}} from {{getResolvedAddress}}, and build the two RM redirect paths the same way. With this fixed at the source, the consumer-side {{TimelineConnector}} change proposed for MAPREDUCE-7429 is not needed. The proxy host {{split(":")}} in {{ResourceManager}} and the YARN Router is covered by HADOOP-17843.

POC: https://github.com/smengcl/hadoop/commit/d4009c263cd14a6f3ebe6a259996c901eeac4f6e



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org