You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Avi Flax <av...@parkassist.com> on 2017/09/08 20:28:43 UTC
Classloading Error with Kotlin and Streams
Hi all, I’m trying to experiment with Kotlin and Streams, and I’m encountering an error:
$ kotlinc -cp kafka-streams-0.11.0.0-cp1.jar:kafka-clients-0.11.0.0-cp1.jar:slf4j-api-1.7.25.jar -jvm-target 1.8
Welcome to Kotlin version 1.1.4-3 (JRE 1.8.0_141-b15)
Type :help for help, :quit for quit
>>> import org.apache.kafka.streams.StreamsConfig
>>> val config = hashMapOf("application.id" to "t2p.normalizer", "bootstrap.servers" to "localhost:9092")
>>> StreamsConfig(config)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
java.lang.ExceptionInInitializerError
Caused by: org.apache.kafka.common.config.ConfigException: Invalid value org.apache.kafka.common.serialization.Serdes$ByteArraySerde for configuration default.key.serde: Class org.apache.kafka.common.serialization.Serdes$ByteArraySerde could not be found.
at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:715)
at org.apache.kafka.common.config.ConfigDef$ConfigKey.<init>(ConfigDef.java:944)
at org.apache.kafka.common.config.ConfigDef.define(ConfigDef.java:137)
at org.apache.kafka.common.config.ConfigDef.define(ConfigDef.java:157)
at org.apache.kafka.common.config.ConfigDef.define(ConfigDef.java:196)
at org.apache.kafka.common.config.ConfigDef.define(ConfigDef.java:358)
at org.apache.kafka.common.config.ConfigDef.define(ConfigDef.java:371)
at org.apache.kafka.streams.StreamsConfig.<clinit>(StreamsConfig.java:270)
at Line_2.<init>(Unknown Source)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.jetbrains.kotlin.cli.common.repl.GenericReplEvaluator$eval$1$scriptInstance$1.invoke(GenericReplEvaluator.kt:93)
<cut>
I’ve put the entire thing into this gist:
https://gist.github.com/aviflax/d33ac2fbd77882fcb80fe8d3fd26feda
I took a look at ConfigDef.java, and I _think_ the error is being thrown from line 706:
https://github.com/apache/kafka/blob/0.11.0.0/clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java#L706
which is:
return Class.forName(trimmed, true, Utils.getContextOrKafkaClassLoader());
so I tried running that in the Kotlin REPL:
>>> import org.apache.kafka.common.utils.Utils
>>> Class.forName("org.apache.kafka.common.serialization.Serdes\$ByteArraySerde", true, Utils.getContextOrKafkaClassLoader())
java.lang.ClassNotFoundException: org.apache.kafka.common.serialization.Serdes$ByteArraySerde
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
(that’s the whole trace)
I thought maybe I was missing a JAR file, but this works fine:
>>> Class.forName("org.apache.kafka.common.serialization.Serdes\$ByteArraySerde")
class org.apache.kafka.common.serialization.Serdes$ByteArraySerde
So I guess this has something to do with the ClassLoader being returned by Utils.getContextOrKafkaClassLoader()
But when I call that, the result looks sane to me:
>>> import org.apache.kafka.common.utils.Utils
>>> Utils.getContextOrKafkaClassLoader()
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
sun.misc.Launcher$AppClassLoader@55f96302
Not that I know much about class loaders, but “AppClassLoader” sounds at least somewhat reasonable.
So if anyone has any ideas on what’s going on here, I’d very much appreciate any help!
Thanks,
Avi
————
Software Architect @ Park Assist » http://tech.parkassist.com/
Re: Classloading Error with Kotlin and Streams
Posted by Avi Flax <av...@parkassist.com>.
> On Sep 15, 2017, at 23:08, Amir Nagri <am...@go-jek.com> wrote:
>
> Were you able to resolve above?
No, not yet. And I haven’t had a chance to open that JIRA ticket… sorry about that. Will try to get to it soon.
————
Software Architect @ Park Assist » http://tech.parkassist.com/
Re: Classloading Error with Kotlin and Streams
Posted by Amir Nagri <am...@go-jek.com>.
Hi Avi,
Were you able to resolve above?
On Tue, Sep 12, 2017 at 12:45 AM, Avi Flax <av...@parkassist.com> wrote:
>
> > On Sep 11, 2017, at 14:31, Bill Bejeck <bi...@confluent.io> wrote:
> >
> > I've done a little digging. Unfortunately, I don't have any answers at
> this point.
> >
> > ...
> >
> > For us to go further, I think you'll need to put a JIRA ticket in, sorry
> this isn't very helpful at this point.
>
> Thanks for putting the time and effort into this! I really appreciate it.
> I’ll open a ticket shortly.
>
> Oh and BTW I cross-posted the problem to the Kotlin forums:
>
> https://discuss.kotlinlang.org/t/classloading-error-with-
> kafka-streams/4547
>
> as I’m thinking that because KafkaConfig works fine in Java and Kotlin is
> aiming for 100% Java compatibility, they might want to consider this a bug,
> even if Streams is doing something weird with class loading.
>
> Thanks!
> Avi
--
Regards,
Amiruddin Nagri
India - +91 9611 9300 33
amir.nagri@go-jek.com
Bangalore
Re: Classloading Error with Kotlin and Streams
Posted by Avi Flax <av...@parkassist.com>.
> On Sep 11, 2017, at 14:31, Bill Bejeck <bi...@confluent.io> wrote:
>
> I've done a little digging. Unfortunately, I don't have any answers at this point.
>
> ...
>
> For us to go further, I think you'll need to put a JIRA ticket in, sorry this isn't very helpful at this point.
Thanks for putting the time and effort into this! I really appreciate it. I’ll open a ticket shortly.
Oh and BTW I cross-posted the problem to the Kotlin forums:
https://discuss.kotlinlang.org/t/classloading-error-with-kafka-streams/4547
as I’m thinking that because KafkaConfig works fine in Java and Kotlin is aiming for 100% Java compatibility, they might want to consider this a bug, even if Streams is doing something weird with class loading.
Thanks!
Avi
Re: Classloading Error with Kotlin and Streams
Posted by Bill Bejeck <bi...@confluent.io>.
Hi Avi,
I've done a little digging. Unfortunately, I don't have any answers at this
point.
Here's what I did:
1. I replicated your CLI command in a standalone class
(TestDeserializerLoader.kt in IntelliJ) and created an uber-jar file. I
can reproduce your error with "kotlin -cp <path-to-jar>
TestDeserializerLoader"
2. If I execute the same command with java, everything works (no big
surprise there!)
3. To see what classes the programs load, I added the
-J-verbose:class/-verbose:class flag to the kotlin/java processes
respectively and captured the results in a file.
4. I grepped for 'serialization, ' and in both files, there is the
following line [Loaded
org.apache.kafka.common.serialization.Serdes$ByteArraySerde from
file:<FileName>]
I've attached the serialization classes loaded from both commands.
For us to go further, I think you'll need to put a JIRA ticket in, sorry
this isn't very helpful at this point.
Thanks,
Bill
On Mon, Sep 11, 2017 at 10:53 AM, Avi Flax <av...@parkassist.com> wrote:
>
> > On Sep 8, 2017, at 17:57, Bill Bejeck <bi...@confluent.io> wrote:
> >
> > I'm not super familiar with Kotiln, but I'm taking a look.
>
> Thanks Bill! I appreciate the help, and I look forward to your findings.
>
> Avi
>
>
> ————
> Software Architect @ Park Assist » http://tech.parkassist.com/
>
Re: Classloading Error with Kotlin and Streams
Posted by Avi Flax <av...@parkassist.com>.
> On Sep 8, 2017, at 17:57, Bill Bejeck <bi...@confluent.io> wrote:
>
> I'm not super familiar with Kotiln, but I'm taking a look.
Thanks Bill! I appreciate the help, and I look forward to your findings.
Avi
————
Software Architect @ Park Assist » http://tech.parkassist.com/
Re: Classloading Error with Kotlin and Streams
Posted by Bill Bejeck <bi...@confluent.io>.
Hi Avi,
I'm not super familiar with Kotiln, but I'm taking a look.
Thanks,
Bill
On Fri, Sep 8, 2017 at 4:28 PM, Avi Flax <av...@parkassist.com> wrote:
> Hi all, I’m trying to experiment with Kotlin and Streams, and I’m
> encountering an error:
>
> $ kotlinc -cp kafka-streams-0.11.0.0-cp1.jar:kafka-clients-0.11.0.0-cp1.jar:slf4j-api-1.7.25.jar
> -jvm-target 1.8
> Welcome to Kotlin version 1.1.4-3 (JRE 1.8.0_141-b15)
> Type :help for help, :quit for quit
> >>> import org.apache.kafka.streams.StreamsConfig
> >>> val config = hashMapOf("application.id" to "t2p.normalizer",
> "bootstrap.servers" to "localhost:9092")
> >>> StreamsConfig(config)
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> java.lang.ExceptionInInitializerError
> Caused by: org.apache.kafka.common.config.ConfigException: Invalid value
> org.apache.kafka.common.serialization.Serdes$ByteArraySerde for
> configuration default.key.serde: Class org.apache.kafka.common.
> serialization.Serdes$ByteArraySerde could not be found.
> at org.apache.kafka.common.config.ConfigDef.parseType(
> ConfigDef.java:715)
> at org.apache.kafka.common.config.ConfigDef$ConfigKey.<
> init>(ConfigDef.java:944)
> at org.apache.kafka.common.config.ConfigDef.define(
> ConfigDef.java:137)
> at org.apache.kafka.common.config.ConfigDef.define(
> ConfigDef.java:157)
> at org.apache.kafka.common.config.ConfigDef.define(
> ConfigDef.java:196)
> at org.apache.kafka.common.config.ConfigDef.define(
> ConfigDef.java:358)
> at org.apache.kafka.common.config.ConfigDef.define(
> ConfigDef.java:371)
> at org.apache.kafka.streams.StreamsConfig.<clinit>(
> StreamsConfig.java:270)
> at Line_2.<init>(Unknown Source)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(
> NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.jetbrains.kotlin.cli.common.repl.
> GenericReplEvaluator$eval$1$scriptInstance$1.invoke(
> GenericReplEvaluator.kt:93)
> <cut>
>
> I’ve put the entire thing into this gist:
>
> https://gist.github.com/aviflax/d33ac2fbd77882fcb80fe8d3fd26feda
>
> I took a look at ConfigDef.java, and I _think_ the error is being thrown
> from line 706:
>
> https://github.com/apache/kafka/blob/0.11.0.0/clients/
> src/main/java/org/apache/kafka/common/config/ConfigDef.java#L706
>
> which is:
>
> return Class.forName(trimmed, true, Utils.getContextOrKafkaClassLoader());
>
> so I tried running that in the Kotlin REPL:
>
> >>> import org.apache.kafka.common.utils.Utils
> >>> Class.forName("org.apache.kafka.common.serialization.Serdes\$ByteArraySerde",
> true, Utils.getContextOrKafkaClassLoader())
> java.lang.ClassNotFoundException: org.apache.kafka.common.
> serialization.Serdes$ByteArraySerde
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
>
> (that’s the whole trace)
>
> I thought maybe I was missing a JAR file, but this works fine:
>
> >>> Class.forName("org.apache.kafka.common.serialization.
> Serdes\$ByteArraySerde")
> class org.apache.kafka.common.serialization.Serdes$ByteArraySerde
>
> So I guess this has something to do with the ClassLoader being returned by
> Utils.getContextOrKafkaClassLoader()
>
> But when I call that, the result looks sane to me:
>
> >>> import org.apache.kafka.common.utils.Utils
> >>> Utils.getContextOrKafkaClassLoader()
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> sun.misc.Launcher$AppClassLoader@55f96302
>
> Not that I know much about class loaders, but “AppClassLoader” sounds at
> least somewhat reasonable.
>
> So if anyone has any ideas on what’s going on here, I’d very much
> appreciate any help!
>
> Thanks,
> Avi
>
> ————
> Software Architect @ Park Assist » http://tech.parkassist.com/
>
>