You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Robert Metzger (JIRA)" <ji...@apache.org> on 2015/08/24 16:19:45 UTC
[jira] [Updated] (FLINK-2568) Flink fat jar is missing
org.w3c.dom.ElementTraversal in some environments.
[ https://issues.apache.org/jira/browse/FLINK-2568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Metzger updated FLINK-2568:
----------------------------------
Description:
Error while writing into HDFS reported by a user. The user is using a Hadoop Configuration file using the XML XInclude feature. Maybe that is causing the issue?
{code}
5:04:57,701 WARN org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source)
at org.apache.xerces.xinclude.XIncludeHandler.startDocument(Unknown Source)
at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1989)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1977)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2048)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2001)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1918)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:721)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:740)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:912)
at org.apache.hadoop.hdfs.protocol.HdfsConstants.<clinit>(HdfsConstants.java:67)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1140)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
{code}
I could not find the class in the Flink fat jar.
It seems like we have to include the xml-apis dependency into Flink:
http://stackoverflow.com/questions/17515333/noclassdeffounderror-org-w3c-dom-elementtraversal
because the class is not available on all JVMs: https://issues.apache.org/jira/browse/HADOOP-11831
{code}
<dependency>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.4.01</version>
</dependency>
{code}
I'm opening this issue to track if more users are affected. The user who reported the issue to me was able to resolve the issue by manually providing the xml-apis dependency. If one one user is affected due to environment specific settings, I would not include the xml-apis into our fat jar (for now)
was:
Error while writing into HDFS reported by a user. The user is using a Hadoop Configuration file using the XML XInclude feature. Maybe that is causing the issue?
{code}
5:04:57,701 WARN org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
[3:09:41 PM] Vyacheslav Zholudev: java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source)
at org.apache.xerces.xinclude.XIncludeHandler.startDocument(Unknown Source)
at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1989)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1977)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2048)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2001)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1918)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:721)
at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:740)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:912)
at org.apache.hadoop.hdfs.protocol.HdfsConstants.<clinit>(HdfsConstants.java:67)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1140)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
{code}
I could not find the class in the Flink fat jar.
It seems like we have to include the xml-apis dependency into Flink:
http://stackoverflow.com/questions/17515333/noclassdeffounderror-org-w3c-dom-elementtraversal
because the class is not available on all JVMs: https://issues.apache.org/jira/browse/HADOOP-11831
{code}
<dependency>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.4.01</version>
</dependency>
{code}
I'm opening this issue to track if more users are affected. The user who reported the issue to me was able to resolve the issue by manually providing the xml-apis dependency. If one one user is affected due to environment specific settings, I would not include the xml-apis into our fat jar (for now)
> Flink fat jar is missing org.w3c.dom.ElementTraversal in some environments.
> ---------------------------------------------------------------------------
>
> Key: FLINK-2568
> URL: https://issues.apache.org/jira/browse/FLINK-2568
> Project: Flink
> Issue Type: Bug
> Components: Build System
> Affects Versions: 0.9
> Reporter: Robert Metzger
>
> Error while writing into HDFS reported by a user. The user is using a Hadoop Configuration file using the XML XInclude feature. Maybe that is causing the issue?
> {code}
> 5:04:57,701 WARN org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception
> java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source)
> at org.apache.xerces.xinclude.XIncludeHandler.startDocument(Unknown Source)
> at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source)
> at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown Source)
> at org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source)
> at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
> at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
> at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
> at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
> at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1989)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1977)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2048)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2001)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1918)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:721)
> at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:740)
> at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:912)
> at org.apache.hadoop.hdfs.protocol.HdfsConstants.<clinit>(HdfsConstants.java:67)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1140)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
> {code}
> I could not find the class in the Flink fat jar.
> It seems like we have to include the xml-apis dependency into Flink:
> http://stackoverflow.com/questions/17515333/noclassdeffounderror-org-w3c-dom-elementtraversal
> because the class is not available on all JVMs: https://issues.apache.org/jira/browse/HADOOP-11831
> {code}
> <dependency>
> <groupId>xml-apis</groupId>
> <artifactId>xml-apis</artifactId>
> <version>1.4.01</version>
> </dependency>
> {code}
> I'm opening this issue to track if more users are affected. The user who reported the issue to me was able to resolve the issue by manually providing the xml-apis dependency. If one one user is affected due to environment specific settings, I would not include the xml-apis into our fat jar (for now)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)