[jira] [Comment Edited] (FLINK-3338) Kafka deserialization issue - ClassNotFoundException

["Stephan Ewen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132740#comment-15132740 ] 

Stephan Ewen edited comment on FLINK-3338 at 2/4/16 6:40 PM:
-------------------------------------------------------------

The stack trace says that the class is cloned using the {{AppClassLoader}}. 

That should only happen if the class of the specific DeserializationSchema itself is part of the initial classpath, but contains classes that are part of the user code jar. But that case will not be handled correctly.

I think we can fix that by passing explicitly the user code classloader to the clone() function.




I think for a proper solution, we need to pass the user code


was (Author: stephanewen):
The stack trace says that the class is cloned using the {{AppClassLoader}}. 

That should only happen if the class of the specific Deserialization Schame itself is part of the initial classpath, but contains classes that are part of the user code jar. But that case will not be handled correctly.

I think we can fix that by passing explicitly the user code classloader to the clone() function.




I think for a proper solution, we need to pass the user code

> Kafka deserialization issue - ClassNotFoundException
> ----------------------------------------------------
>
>                 Key: FLINK-3338
>                 URL: https://issues.apache.org/jira/browse/FLINK-3338
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Shikhar Bhushan
>
> I get this error when running a JAR submitted using the web UI on a production standalone cluster (single node, 1 TM & 1 JM), but not on a standalone cluster (1 TM & 1 JM) running on my laptop. JDK8 on both.
> {noformat}
> java.lang.ClassNotFoundException: omitted$KafkaDeserializationSchema$
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Class.java:348)
> 	at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:62)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1613)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
> 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
> 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
> 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
> 	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:288)
> 	at org.apache.flink.util.InstantiationUtil.clone(InstantiationUtil.java:314)
> 	at org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher.run(LegacyFetcher.java:203)
> 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08.run(FlinkKafkaConsumer08.java:307)
> 	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58)
> 	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:55)
> 	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:218)
> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
> 	at java.lang.Thread.run(Thread.java:745)
> {noformat}
> [~till.rohrmann] this seems related to your recent change via FLINK-3313? I don't understand why it would happen though.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)