You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Jim Brennan (Jira)" <ji...@apache.org> on 2020/07/16 16:08:00 UTC
[jira] [Commented] (YARN-1741) XInclude support broken for YARN
ResourceManager
[ https://issues.apache.org/jira/browse/YARN-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159326#comment-17159326 ]
Jim Brennan commented on YARN-1741:
-----------------------------------
While reviewing changes we made for our internal branch-2.8, I came across this one.
We have an internal fix for this that we have been running with for quite some time. I have verified that this is still broken in branch-2.8 - if I remove our internal fix, the resourcemanager still fails to load xi:include files.
We are moving to branch-2.10 internally, so I was mainly interested in determining if we still needed this internal change. As far as I can tell, it is not needed in branch-2.10 or later. I believe this is mainly due to these changes: [HADOOP-14216], [HADOOP-14399], [HADOOP-15973], which are all included in branch-2.9 and later.
Since branch-2.8 is EOL, I propose that we close this as Won't Fix. Although if there is interest, I can put up a patch for branch-2.8.
> XInclude support broken for YARN ResourceManager
> ------------------------------------------------
>
> Key: YARN-1741
> URL: https://issues.apache.org/jira/browse/YARN-1741
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Eric Sirianni
> Assignee: Xuan Gong
> Priority: Critical
> Labels: regression
>
> The XInclude support in Hadoop configuration files (introduced via HADOOP-4944) was broken by the recent {{ConfigurationProvider}} changes to YARN ResourceManager. Specifically, YARN-1459 and, more generally, the YARN-1611 family of JIRAs for ResourceManager HA.
> The issue is that {{ConfigurationProvider}} provides a raw {{InputStream}} as a {{Configuration}} resource for what was previously a {{Path}}-based resource.
> For {{Path}} resources, the absolute file path is used as the {{systemId}} for the {{DocumentBuilder.parse()}} call:
> {code}
> } else if (resource instanceof Path) { // a file resource
> ...
> doc = parse(builder, new BufferedInputStream(
> new FileInputStream(file)), ((Path)resource).toString());
> }
> {code}
> The {{systemId}} is used to resolve XIncludes (among other things):
> {code}
> /**
> * Parse the content of the given <code>InputStream</code> as an
> * XML document and return a new DOM Document object.
> ...
> * @param systemId Provide a base for resolving relative URIs.
> ...
> */
> public Document parse(InputStream is, String systemId)
> {code}
> However, for loading raw {{InputStream}} resources, the {{systemId}} is set to {{null}}:
> {code}
> } else if (resource instanceof InputStream) {
> doc = parse(builder, (InputStream) resource, null);
> {code}
> causing XInclude resolution to fail.
> In our particular environment, we make extensive use of XIncludes to standardize common configuration parameters across multiple Hadoop clusters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org