[jira] [Created] (FLINK-2568) Flink fat jar is missing org.w3c.dom.ElementTraversal in some environments.

["Robert Metzger (JIRA)" <ji...@apache.org>]

Robert Metzger created FLINK-2568:
-------------------------------------

             Summary: Flink fat jar is missing org.w3c.dom.ElementTraversal in some environments.
                 Key: FLINK-2568
                 URL: https://issues.apache.org/jira/browse/FLINK-2568
             Project: Flink
          Issue Type: Bug
          Components: Build System
    Affects Versions: 0.9
            Reporter: Robert Metzger


Error while writing into HDFS reported by a user. The user is using a Hadoop Configuration file using the XML XInclude feature. Maybe that is causing the issue?

{code}
5:04:57,701 WARN  org.apache.hadoop.hdfs.DFSClient                              - DataStreamer Exception
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
[3:09:41 PM] Vyacheslav Zholudev: java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source)
        at org.apache.xerces.xinclude.XIncludeHandler.startDocument(Unknown Source)
        at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown Source)
        at org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
        at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
        at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
        at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1989)
        at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1977)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2048)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2001)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1918)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:721)
        at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:740)
        at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:912)
        at org.apache.hadoop.hdfs.protocol.HdfsConstants.<clinit>(HdfsConstants.java:67)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1140)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
{code}

I could not find the class in the Flink fat jar.

It seems like we have to include the xml-apis dependency into Flink:
http://stackoverflow.com/questions/17515333/noclassdeffounderror-org-w3c-dom-elementtraversal
because the class is not available on all JVMs: https://issues.apache.org/jira/browse/HADOOP-11831

{code}
<dependency>
	<groupId>xml-apis</groupId>
	<artifactId>xml-apis</artifactId>
	<version>1.4.01</version>
</dependency>
{code}

I'm opening this issue to track if more users are affected. The user who reported the issue to me was able to resolve the issue by manually providing the xml-apis dependency. If one one user is affected due to environment specific settings, I would not include the xml-apis into our fat jar (for now)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)