You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Elias Ross (Created) (JIRA)" <ji...@apache.org> on 2012/02/07 21:22:59 UTC
[jira] [Created] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Configuration class fails to find embedded .jar resources; should use URL.openStream()
--------------------------------------------------------------------------------------
Key: HADOOP-8031
URL: https://issues.apache.org/jira/browse/HADOOP-8031
Project: Hadoop Common
Issue Type: Bug
Reporter: Elias Ross
While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
Caused by: java.lang.RuntimeException: core-site.xml not found
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
This is because the URL
jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
The solution is to obtain the resource stream directly from the URL object itself.
That is to say:
{code}
URL url = getResource((String)name);
- if (url != null) {
- if (!quiet) {
- LOG.info("parsing " + url);
- }
- doc = builder.parse(url.toString());
- }
+ doc = builder.parse(url.openStream());
{code}
Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440849#comment-13440849 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2660 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2660/])
HADOOP-8031. Configuration class fails to find embedded .jar resources; should use URL.openStream() (genman via tucu) (Revision 1376772)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1376772
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8031:
--------------------------------
Attachment: hadoop-8031.txt
Patch attached.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins reassigned HADOOP-8031:
-----------------------------------
Assignee: Elias Ross
Thanks for contributing Elias. Can you update TestConfiguration with a case that will fail w/o your patch?
I've rebased your patch on trunk.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444657#comment-13444657 ]
Ahmed Radwan commented on HADOOP-8031:
--------------------------------------
I looked into the implementation of javax.xml.parsers.DocumentBuilder and org.xml.sax.InputSource and there is a difference when the DocumentBuilder parse(String) method is used versus parse(InputStream). Basically we need to use parse(InputStream is, String systemId) which provides a base for resolving relative URIs. Here is a new patch that fixes this issue. It needs to be applied on top of the previously committed patch. I am not sure if we need to create a new ticket since this one is already committed.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon resolved HADOOP-8031.
---------------------------------
Resolution: Fixed
Re-resolving this since Ahmed is addressing my issue in HADOOP-8749
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444582#comment-13444582 ]
Todd Lipcon commented on HADOOP-8031:
-------------------------------------
Hey Tucu. I think this commit broke the way in which relative xincludes are handled in Configuration. I have some development confs which use xinclude with non-absolute paths, and it used to successfully pick up the included files from my conf directory. Now, it seems to be looking in the current working directory instead.
Is it possible to fix the code so that the relative paths are resolved the same as before? I think xinclude is relatively common for deployments.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450587#comment-13450587 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #1158 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1158/])
HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445199#comment-13445199 ]
Ahmed Radwan commented on HADOOP-8031:
--------------------------------------
Thanks Eli! I have created HADOOP-8749.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449956#comment-13449956 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2719 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2719/])
HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated HADOOP-8031:
---------------------------------
Status: Patch Available (was: Open)
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444600#comment-13444600 ]
Ahmed Radwan commented on HADOOP-8031:
--------------------------------------
Hi Todd, It is weird that this patch caused this behavior change. The patch didn't modify the builder or the docBuilderFactory, and it is still docBuilderFactory.setXIncludeAware(true). In essence, the patch is basically using DocumentBuilder#parse(InputStream uri.openStream()) instead of DocumentBuilder#parse(String uri.toString()). Seems there is a change in implementation of both parse methods, which seems weird.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated HADOOP-8031:
---------------------------------
Attachment: HADOOP-8031.patch
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Elias Ross (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Elias Ross updated HADOOP-8031:
-------------------------------
Attachment: 0001-fix-HADOOP-7982-class-loader.patch
Simple patch
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450633#comment-13450633 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1189 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1189/])
HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-8031:
---------------------------------------
Resolution: Fixed
Fix Version/s: 2.2.0-alpha
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Elias. Committed to trunk and branch-2.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440761#comment-13440761 ]
Alejandro Abdelnur commented on HADOOP-8031:
--------------------------------------------
+1
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-8031:
--------------------------------
Component/s: conf
Target Version/s: 2.0.1-alpha
Affects Version/s: 2.0.0-alpha
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated HADOOP-8031:
---------------------------------
Attachment: HADOOP-8031-part2.patch
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444602#comment-13444602 ]
Todd Lipcon commented on HADOOP-8031:
-------------------------------------
Yea, I don't know much about the underlying API, but it definitely changed the behavior. It's still trying to do the xinclude, it's just looking in cwd instead of my conf dir.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440803#comment-13440803 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2696 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2696/])
HADOOP-8031. Configuration class fails to find embedded .jar resources; should use URL.openStream() (genman via tucu) (Revision 1376772)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1376772
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed HADOOP-8031.
---------------------------------
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.0.2-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440794#comment-13440794 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #2632 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2632/])
HADOOP-8031. Configuration class fails to find embedded .jar resources; should use URL.openStream() (genman via tucu) (Revision 1376772)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1376772
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440169#comment-13440169 ]
Hadoop QA commented on HADOOP-8031:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12542098/HADOOP-8031.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1350//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1350//console
This message is automatically generated.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449902#comment-13449902 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #2695 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2695/])
HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445175#comment-13445175 ]
Eli Collins commented on HADOOP-8031:
-------------------------------------
I'd either file a new jira explaining the bug with a new patch with fix/test, or revert this patch and post a new one that includes the fix.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449897#comment-13449897 ]
Hudson commented on HADOOP-8031:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2758 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2758/])
HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031-part2.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Elias Ross (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283897#comment-13283897 ]
Elias Ross commented on HADOOP-8031:
------------------------------------
Eli,
Thanks for your response. I would like to reproduce the problem but I'd have to somehow embed the .xml file inside a .jar and adjust the test classpath to match. I'd likely have to isolate the test from the rest of the existing classpath as well. Maybe you could guide me through this?
Thanks.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Ahmed Radwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440122#comment-13440122 ]
Ahmed Radwan commented on HADOOP-8031:
--------------------------------------
Thanks Elias,
I have noticed that the patch no longer applies due to the recent changes to Configuration.java. I have made the required changes so it can apply cleanly on trunk.
Also changed:
{code}
+ doc = builder.parse((InputStream)name);
{code}
to:
{code}
+ doc = parse(builder, (InputStream) resource);
{code}
to insure the InputStream will be closed.
Regarding testing, I am not sure if there is an easy way to reproduce this issue in a unit test.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-8031) Configuration class fails to find
embedded .jar resources; should use URL.openStream()
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon reopened HADOOP-8031:
---------------------------------
I confirmed that reverting this patch locally restored the old behavior.
If we can't maintain the old behavior, we should at least mark this as an incompatible change. But I bet it's doable to both fix it and have relative xincludes.
> Configuration class fails to find embedded .jar resources; should use URL.openStream()
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-8031
> URL: https://issues.apache.org/jira/browse/HADOOP-8031
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 2.0.0-alpha
> Reporter: Elias Ross
> Assignee: Elias Ross
> Fix For: 2.2.0-alpha
>
> Attachments: 0001-fix-HADOOP-7982-class-loader.patch, HADOOP-8031.patch, hadoop-8031.txt
>
>
> While running a hadoop client within RHQ (monitoring software) using its classloader, I see this:
> 2012-02-07 09:15:25,313 INFO [ResourceContainer.invoker.daemon-2] (org.apache.hadoop.conf.Configuration)- parsing jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> 2012-02-07 09:15:25,318 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to start component for Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com] from synchronized merge.
> org.rhq.core.clientapi.agent.PluginContainerException: Failed to start component for resource Resource[id=16290, type=NameNode, key=NameNode:/usr/lib/hadoop-0.20, name=NameNode, parent=vg61l01ad-hadoop002.apple.com].
> Caused by: java.lang.RuntimeException: core-site.xml not found
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1308)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1228)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1169)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:438)
> This is because the URL
> jar:file:/usr/local/rhq-agent/data/tmp/rhq-hadoop-plugin-4.3.0-SNAPSHOT.jar6856622641102893436.classloader/hadoop-core-0.20.2+737+1.jar7204287718482036191.tmp!/core-default.xml
> cannot be found by DocumentBuilder (doesn't understand it). (Note: the logs are for an old version of Configuration class, but the new version has the same code.)
> The solution is to obtain the resource stream directly from the URL object itself.
> That is to say:
> {code}
> URL url = getResource((String)name);
> - if (url != null) {
> - if (!quiet) {
> - LOG.info("parsing " + url);
> - }
> - doc = builder.parse(url.toString());
> - }
> + doc = builder.parse(url.openStream());
> {code}
> Note: I have a full patch pending approval at Apple for this change, including some cleanup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira