You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "jialei weng (JIRA)" <ji...@apache.org> on 2018/08/21 04:25:00 UTC
[jira] [Updated] (YARN-8690) Currently path not consistent in
LocalResourceRequest to yarn 2.7
[ https://issues.apache.org/jira/browse/YARN-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jialei weng updated YARN-8690:
------------------------------
Description:
With YARN-1953 change, in yarn 2.9.1, we can not use path, like hdfs://hostname/path, to local resource allocation as it will be resolved to hdfs://hostname{color:#ff0000}:0{color}/path. We have to add the port 443 in path, like hdfs://hostname{color:#ff0000}:443{color}/path, to make it work. It isn't a consistent change. If we can make it consistent without customer's change? [~leftnoteasy]
|Handle resource location path in 2.7|Handle resource location logic in 2.9|
| public static Path getPathFromYarnURL(URL url) throws URISyntaxException {
String scheme = url.getScheme() == null ? "" : url.getScheme();
String authority = "";
if (url.getHost() != null) {
authority = url.getHost();
if (url.getUserInfo() != null) {
authority = url.getUserInfo() + "@" + authority;
}
{color:#d04437} if (url.getPort() > 0) {{color}
{color:#d04437} authority += ":" + url.getPort();{color}
{color:#d04437} }{color}
}
return new Path(
(new URI(scheme, authority, url.getFile(), null, null)).normalize());
}| public Path toPath() throws URISyntaxException {
return new Path(new URI(getScheme(), getUserInfo(),
getHost(), getPort(), getFile(), null, null));
}|
was:With YARN-1953 change, in yarn 2.9.1, we can not use path, like hdfs://hostname/path, to local resource allocation as it will be resolved to hdfs://hostname{color:#FF0000}:0{color}/path. We have to add the port 443 in path, like hdfs://hostname{color:#FF0000}:443{color}/path, to make it work. It isn't a consistent change. If we can make it consistent without customer's change? [~leftnoteasy]
> Currently path not consistent in LocalResourceRequest to yarn 2.7
> -----------------------------------------------------------------
>
> Key: YARN-8690
> URL: https://issues.apache.org/jira/browse/YARN-8690
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.9.1
> Reporter: jialei weng
> Assignee: Wangda Tan
> Priority: Major
>
> With YARN-1953 change, in yarn 2.9.1, we can not use path, like hdfs://hostname/path, to local resource allocation as it will be resolved to hdfs://hostname{color:#ff0000}:0{color}/path. We have to add the port 443 in path, like hdfs://hostname{color:#ff0000}:443{color}/path, to make it work. It isn't a consistent change. If we can make it consistent without customer's change? [~leftnoteasy]
> |Handle resource location path in 2.7|Handle resource location logic in 2.9|
> | public static Path getPathFromYarnURL(URL url) throws URISyntaxException {
> String scheme = url.getScheme() == null ? "" : url.getScheme();
>
> String authority = "";
> if (url.getHost() != null) {
> authority = url.getHost();
> if (url.getUserInfo() != null) {
> authority = url.getUserInfo() + "@" + authority;
> }
> {color:#d04437} if (url.getPort() > 0) {{color}
> {color:#d04437} authority += ":" + url.getPort();{color}
> {color:#d04437} }{color}
> }
>
> return new Path(
> (new URI(scheme, authority, url.getFile(), null, null)).normalize());
> }| public Path toPath() throws URISyntaxException {
> return new Path(new URI(getScheme(), getUserInfo(),
> getHost(), getPort(), getFile(), null, null));
> }|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org