You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Kashyap Mhaisekar <ka...@gmail.com> on 2014/04/11 04:17:04 UTC
Kafka spout versions
Hi,
What is the version of Kafka spout that works with storm 0.9 and kafka 0.8?
The standard example doesn't throw exceptions but doesn't seem to consume
messages either.
Thanks,
Kashyap
Re: Kafka spout versions
Posted by Andrew Neilson <ar...@gmail.com>.
Based on the jar name you used and the errors you are getting it looks like
you may have not built the topology as a fat jar? The jar you pass to
`storm jar` should contain all of your dependencies. When you run `mvn
package` in that project it will produce two jars in the target dir:
storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar and
storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT-jar-with-dependencies.jar. You
want the "jar-with-dependencies" one. See the instructions in the readme:
run locally
https://github.com/wurstmeister/storm-kafka-0.8-plus-test#build-for-running-locally
deploy to a cluster
https://github.com/wurstmeister/storm-kafka-0.8-plus-test#build-for-running-on-a-storm-cluster
On Wed, Apr 16, 2014 at 10:32 AM, M. Solaimani <so...@gmail.com>wrote:
> Hi,
>
> I have copied all the scala file
> both scala-library-2.9.2.jar, scala-library.jar into storm's lib folder.
> But still get the error :(
>
> Can anyone tell me if there is any version mismatch issue with storm and
> kafka integration?
>
>
>
> Best regards,
> M Solaimani
>
>
> On Tue, Apr 15, 2014 at 8:44 PM, Xing Yong <xy...@gmail.com> wrote:
>
>> maybe you need scala-library.jar, it will work.
>>
>>
>> 2014-04-16 5:37 GMT+08:00 M. Solaimani <so...@gmail.com>:
>>
>> Hi,
>>>
>>> I have copied those jar files in storm lib folder. That errors are gone
>>> but I have new error.
>>>
>>> java.lang.NoClassDefFoundError: scala/runtime/AbstractFunction1$mcLI$sp
>>> at java.lang.ClassLoader.defineClass1(Native Method)
>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:37)
>>> at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:33)
>>> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
>>> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
>>> at scala.collection.immutable.Range.foreach(Range.scala:81)
>>> at scala.collection.TraversableLike$class.map(TraversableLike.scala:233)
>>> at scala.collection.immutable.Range.map(Range.scala:46)
>>> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:33)
>>> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:30)
>>> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
>>> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
>>> at scala.collection.immutable.Range.foreach(Range.scala:81)
>>> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239)
>>> at scala.collection.immutable.Range.flatMap(Range.scala:46)
>>> at kafka.api.OffsetResponse$.readFrom(OffsetResponse.scala:30)
>>>
>>> at
>>> kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:125)
>>>
>>>
>>> It seems that this error is related with scala. I copied the scala 2.9.2
>>> jar files inside the storm lib directory but still have this error. Do you
>>> guys have any idea to resolve it?
>>>
>>> With best regards,
>>> M Solaimani
>>>
>>>
>>> On Mon, Apr 14, 2014 at 8:51 PM, Xing Yong <xy...@gmail.com> wrote:
>>>
>>>> as the Exception "NoClassDefFoundError: storm/kafka/BrokerHos", i
>>>> think the jar of storm-kafka-0.8-plus and jar of kafka should be
>>>> placed in the lib of the storm
>>>>
>>>>
>>>> 2014-04-15 5:21 GMT+08:00 M. Solaimani <so...@gmail.com>:
>>>>
>>>> Hi,
>>>>>
>>>>> I saw this mail and tried to run the Wurstmiesters
>>>>> storm-kafka-0.8-plus test. Here what I did:
>>>>>
>>>>> 1. I downloaded the Wurstmiesters storm-kafka-0.8-plus test and
>>>>> build it by using mvn clean install.
>>>>> 2. I had my storm cluster running. (Storm 0.9.0-rc2, Kakfka 0.8)
>>>>> 3. I had kafka 0.8 and I run kafka servers, with zoo-keeper and
>>>>> created a topic storm-sentence
>>>>>
>>>>> All the above steps running well.
>>>>>
>>>>> But When I run Wurstmiesters storm-kafka-0.8-plus test by the
>>>>> following commands
>>>>>
>>>>> storm jar target/storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar
>>>>> storm.kafka.KafkaSpoutTestTopology 10.176.148.51
>>>>> sentences 10.176.148.51
>>>>>
>>>>> I got the following errors:
>>>>>
>>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>> storm/kafka/BrokerHos
>>>>> ts
>>>>> at java.lang.Class.getDeclaredMethods0(Native Method)
>>>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
>>>>> at java.lang.Class.getMethod0(Class.java:2774)
>>>>> at java.lang.Class.getMethod(Class.java:1663)
>>>>> at
>>>>> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
>>>>> at
>>>>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
>>>>> Caused by: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts
>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>>>> at
>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>>>> ... 6 more
>>>>>
>>>>>
>>>>> At this point I am stuck. I searched the internet and it says that
>>>>> there is some version mismatch issue? Can anyone help me please?
>>>>>
>>>>> With best regards,
>>>>> M Solaimani
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 14, 2014 at 11:58 AM, Kashyap Mhaisekar <
>>>>> kashyap.m@gmail.com> wrote:
>>>>>
>>>>>> Wurstmiesters seems to work for me. I used storm 0.9.0 rc3
>>>>>>
>>>>>> Thanks,
>>>>>> Kashyap
>>>>>>
>>>>>>
>>>>>> On Monday, April 14, 2014, Trevor Madge <tm...@gmail.com> wrote:
>>>>>>
>>>>>>> I didn't have any luck with wurstmeister's but I've found this one (
>>>>>>> https://github.com/HolmesNL/kafka-spout) and it works with storm
>>>>>>> 0.9.1 and kafka 0.8.1.
>>>>>>>
>>>>>>>
>>>>>>> -Trevor
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <
>>>>>>> kashyap.m@gmail.com> wrote:
>>>>>>>
>>>>>>>> Thanks Carlos.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Kashyap,
>>>>>>>>>
>>>>>>>>> The kafka spout provided in the storm-contrib project (
>>>>>>>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>>>>>>>>> work with kafka 0.8
>>>>>>>>>
>>>>>>>>> The version provided by wurstmeister at
>>>>>>>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed
>>>>>>>>> to work with storm 0.9.1 and kafka 0.8.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Carlos
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> What is the version of Kafka spout that works with storm 0.9 and
>>>>>>>>>> kafka 0.8?
>>>>>>>>>> The standard example doesn't throw exceptions but doesn't seem to
>>>>>>>>>> consume messages either.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Kashyap
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Kafka spout versions
Posted by "M. Solaimani" <so...@gmail.com>.
Hi,
I have copied all the scala file
both scala-library-2.9.2.jar, scala-library.jar into storm's lib folder.
But still get the error :(
Can anyone tell me if there is any version mismatch issue with storm and
kafka integration?
Best regards,
M Solaimani
On Tue, Apr 15, 2014 at 8:44 PM, Xing Yong <xy...@gmail.com> wrote:
> maybe you need scala-library.jar, it will work.
>
>
> 2014-04-16 5:37 GMT+08:00 M. Solaimani <so...@gmail.com>:
>
> Hi,
>>
>> I have copied those jar files in storm lib folder. That errors are gone
>> but I have new error.
>>
>> java.lang.NoClassDefFoundError: scala/runtime/AbstractFunction1$mcLI$sp
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:37)
>> at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:33)
>> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
>> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
>> at scala.collection.immutable.Range.foreach(Range.scala:81)
>> at scala.collection.TraversableLike$class.map(TraversableLike.scala:233)
>> at scala.collection.immutable.Range.map(Range.scala:46)
>> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:33)
>> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:30)
>> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
>> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
>> at scala.collection.immutable.Range.foreach(Range.scala:81)
>> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239)
>> at scala.collection.immutable.Range.flatMap(Range.scala:46)
>> at kafka.api.OffsetResponse$.readFrom(OffsetResponse.scala:30)
>>
>> at
>> kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:125)
>>
>> It seems that this error is related with scala. I copied the scala 2.9.2
>> jar files inside the storm lib directory but still have this error. Do you
>> guys have any idea to resolve it?
>>
>> With best regards,
>> M Solaimani
>>
>>
>> On Mon, Apr 14, 2014 at 8:51 PM, Xing Yong <xy...@gmail.com> wrote:
>>
>>> as the Exception "NoClassDefFoundError: storm/kafka/BrokerHos", i think
>>> the jar of storm-kafka-0.8-plus and jar of kafka should be placed in
>>> the lib of the storm
>>>
>>>
>>> 2014-04-15 5:21 GMT+08:00 M. Solaimani <so...@gmail.com>:
>>>
>>> Hi,
>>>>
>>>> I saw this mail and tried to run the Wurstmiesters
>>>> storm-kafka-0.8-plus test. Here what I did:
>>>>
>>>> 1. I downloaded the Wurstmiesters storm-kafka-0.8-plus test and build
>>>> it by using mvn clean install.
>>>> 2. I had my storm cluster running. (Storm 0.9.0-rc2, Kakfka 0.8)
>>>> 3. I had kafka 0.8 and I run kafka servers, with zoo-keeper and created
>>>> a topic storm-sentence
>>>>
>>>> All the above steps running well.
>>>>
>>>> But When I run Wurstmiesters storm-kafka-0.8-plus test by the
>>>> following commands
>>>>
>>>> storm jar target/storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar
>>>> storm.kafka.KafkaSpoutTestTopology 10.176.148.51
>>>> sentences 10.176.148.51
>>>>
>>>> I got the following errors:
>>>>
>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>> storm/kafka/BrokerHos
>>>> ts
>>>> at java.lang.Class.getDeclaredMethods0(Native Method)
>>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
>>>> at java.lang.Class.getMethod0(Class.java:2774)
>>>> at java.lang.Class.getMethod(Class.java:1663)
>>>> at
>>>> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
>>>> at
>>>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
>>>> Caused by: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>>> ... 6 more
>>>>
>>>>
>>>> At this point I am stuck. I searched the internet and it says that
>>>> there is some version mismatch issue? Can anyone help me please?
>>>>
>>>> With best regards,
>>>> M Solaimani
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Apr 14, 2014 at 11:58 AM, Kashyap Mhaisekar <
>>>> kashyap.m@gmail.com> wrote:
>>>>
>>>>> Wurstmiesters seems to work for me. I used storm 0.9.0 rc3
>>>>>
>>>>> Thanks,
>>>>> Kashyap
>>>>>
>>>>>
>>>>> On Monday, April 14, 2014, Trevor Madge <tm...@gmail.com> wrote:
>>>>>
>>>>>> I didn't have any luck with wurstmeister's but I've found this one (
>>>>>> https://github.com/HolmesNL/kafka-spout) and it works with storm
>>>>>> 0.9.1 and kafka 0.8.1.
>>>>>>
>>>>>>
>>>>>> -Trevor
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <
>>>>>> kashyap.m@gmail.com> wrote:
>>>>>>
>>>>>>> Thanks Carlos.
>>>>>>>
>>>>>>>
>>>>>>> On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Kashyap,
>>>>>>>>
>>>>>>>> The kafka spout provided in the storm-contrib project (
>>>>>>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>>>>>>>> work with kafka 0.8
>>>>>>>>
>>>>>>>> The version provided by wurstmeister at
>>>>>>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed
>>>>>>>> to work with storm 0.9.1 and kafka 0.8.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Carlos
>>>>>>>>
>>>>>>>>
>>>>>>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> What is the version of Kafka spout that works with storm 0.9 and
>>>>>>>>> kafka 0.8?
>>>>>>>>> The standard example doesn't throw exceptions but doesn't seem to
>>>>>>>>> consume messages either.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Kashyap
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>>
>>
>
Re: Kafka spout versions
Posted by Xing Yong <xy...@gmail.com>.
maybe you need scala-library.jar, it will work.
2014-04-16 5:37 GMT+08:00 M. Solaimani <so...@gmail.com>:
> Hi,
>
> I have copied those jar files in storm lib folder. That errors are gone
> but I have new error.
>
> java.lang.NoClassDefFoundError: scala/runtime/AbstractFunction1$mcLI$sp
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:37)
> at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:33)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
> at scala.collection.immutable.Range.foreach(Range.scala:81)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:233)
> at scala.collection.immutable.Range.map(Range.scala:46)
> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:33)
> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:30)
> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
> at scala.collection.immutable.Range.foreach(Range.scala:81)
> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239)
> at scala.collection.immutable.Range.flatMap(Range.scala:46)
> at kafka.api.OffsetResponse$.readFrom(OffsetResponse.scala:30)
>
> at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:125)
>
>
> It seems that this error is related with scala. I copied the scala 2.9.2
> jar files inside the storm lib directory but still have this error. Do you
> guys have any idea to resolve it?
>
> With best regards,
> M Solaimani
>
>
> On Mon, Apr 14, 2014 at 8:51 PM, Xing Yong <xy...@gmail.com> wrote:
>
>> as the Exception "NoClassDefFoundError: storm/kafka/BrokerHos", i think
>> the jar of storm-kafka-0.8-plus and jar of kafka should be placed in the
>> lib of the storm
>>
>>
>> 2014-04-15 5:21 GMT+08:00 M. Solaimani <so...@gmail.com>:
>>
>> Hi,
>>>
>>> I saw this mail and tried to run the Wurstmiesters storm-kafka-0.8-plus
>>> test. Here what I did:
>>>
>>> 1. I downloaded the Wurstmiesters storm-kafka-0.8-plus test and build
>>> it by using mvn clean install.
>>> 2. I had my storm cluster running. (Storm 0.9.0-rc2, Kakfka 0.8)
>>> 3. I had kafka 0.8 and I run kafka servers, with zoo-keeper and created
>>> a topic storm-sentence
>>>
>>> All the above steps running well.
>>>
>>> But When I run Wurstmiesters storm-kafka-0.8-plus test by the following
>>> commands
>>>
>>> storm jar target/storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar
>>> storm.kafka.KafkaSpoutTestTopology 10.176.148.51
>>> sentences 10.176.148.51
>>>
>>> I got the following errors:
>>>
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> storm/kafka/BrokerHos
>>> ts
>>> at java.lang.Class.getDeclaredMethods0(Native Method)
>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
>>> at java.lang.Class.getMethod0(Class.java:2774)
>>> at java.lang.Class.getMethod(Class.java:1663)
>>> at
>>> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
>>> at
>>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
>>> Caused by: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> ... 6 more
>>>
>>>
>>> At this point I am stuck. I searched the internet and it says that there
>>> is some version mismatch issue? Can anyone help me please?
>>>
>>> With best regards,
>>> M Solaimani
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Apr 14, 2014 at 11:58 AM, Kashyap Mhaisekar <kashyap.m@gmail.com
>>> > wrote:
>>>
>>>> Wurstmiesters seems to work for me. I used storm 0.9.0 rc3
>>>>
>>>> Thanks,
>>>> Kashyap
>>>>
>>>>
>>>> On Monday, April 14, 2014, Trevor Madge <tm...@gmail.com> wrote:
>>>>
>>>>> I didn't have any luck with wurstmeister's but I've found this one (
>>>>> https://github.com/HolmesNL/kafka-spout) and it works with storm
>>>>> 0.9.1 and kafka 0.8.1.
>>>>>
>>>>>
>>>>> -Trevor
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <
>>>>> kashyap.m@gmail.com> wrote:
>>>>>
>>>>>> Thanks Carlos.
>>>>>>
>>>>>>
>>>>>> On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Kashyap,
>>>>>>>
>>>>>>> The kafka spout provided in the storm-contrib project (
>>>>>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>>>>>>> work with kafka 0.8
>>>>>>>
>>>>>>> The version provided by wurstmeister at
>>>>>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to
>>>>>>> work with storm 0.9.1 and kafka 0.8.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Carlos
>>>>>>>
>>>>>>>
>>>>>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> What is the version of Kafka spout that works with storm 0.9 and
>>>>>>>> kafka 0.8?
>>>>>>>> The standard example doesn't throw exceptions but doesn't seem to
>>>>>>>> consume messages either.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Kashyap
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
>>
>
Re: Kafka spout versions
Posted by "M. Solaimani" <so...@gmail.com>.
Hi,
I have copied those jar files in storm lib folder. That errors are gone but
I have new error.
java.lang.NoClassDefFoundError: scala/runtime/AbstractFunction1$mcLI$sp
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:37)
at kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:33)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
at scala.collection.immutable.Range.foreach(Range.scala:81)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:233)
at scala.collection.immutable.Range.map(Range.scala:46)
at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:33)
at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:30)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239)
at scala.collection.immutable.Range.foreach(Range.scala:81)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239)
at scala.collection.immutable.Range.flatMap(Range.scala:46)
at kafka.api.OffsetResponse$.readFrom(OffsetResponse.scala:30)
at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:125)
It seems that this error is related with scala. I copied the scala 2.9.2
jar files inside the storm lib directory but still have this error. Do you
guys have any idea to resolve it?
With best regards,
M Solaimani
On Mon, Apr 14, 2014 at 8:51 PM, Xing Yong <xy...@gmail.com> wrote:
> as the Exception "NoClassDefFoundError: storm/kafka/BrokerHos", i think
> the jar of storm-kafka-0.8-plus and jar of kafka should be placed in the
> lib of the storm
>
>
> 2014-04-15 5:21 GMT+08:00 M. Solaimani <so...@gmail.com>:
>
> Hi,
>>
>> I saw this mail and tried to run the Wurstmiesters storm-kafka-0.8-plus
>> test. Here what I did:
>>
>> 1. I downloaded the Wurstmiesters storm-kafka-0.8-plus test and build
>> it by using mvn clean install.
>> 2. I had my storm cluster running. (Storm 0.9.0-rc2, Kakfka 0.8)
>> 3. I had kafka 0.8 and I run kafka servers, with zoo-keeper and created a
>> topic storm-sentence
>>
>> All the above steps running well.
>>
>> But When I run Wurstmiesters storm-kafka-0.8-plus test by the following
>> commands
>>
>> storm jar target/storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar
>> storm.kafka.KafkaSpoutTestTopology 10.176.148.51
>> sentences 10.176.148.51
>>
>> I got the following errors:
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> storm/kafka/BrokerHos
>> ts
>> at java.lang.Class.getDeclaredMethods0(Native Method)
>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
>> at java.lang.Class.getMethod0(Class.java:2774)
>> at java.lang.Class.getMethod(Class.java:1663)
>> at
>> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
>> at
>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
>> Caused by: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> ... 6 more
>>
>>
>> At this point I am stuck. I searched the internet and it says that there
>> is some version mismatch issue? Can anyone help me please?
>>
>> With best regards,
>> M Solaimani
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Apr 14, 2014 at 11:58 AM, Kashyap Mhaisekar <ka...@gmail.com>wrote:
>>
>>> Wurstmiesters seems to work for me. I used storm 0.9.0 rc3
>>>
>>> Thanks,
>>> Kashyap
>>>
>>>
>>> On Monday, April 14, 2014, Trevor Madge <tm...@gmail.com> wrote:
>>>
>>>> I didn't have any luck with wurstmeister's but I've found this one (
>>>> https://github.com/HolmesNL/kafka-spout) and it works with storm 0.9.1
>>>> and kafka 0.8.1.
>>>>
>>>>
>>>> -Trevor
>>>>
>>>>
>>>>
>>>> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <kashyap.m@gmail.com
>>>> > wrote:
>>>>
>>>>> Thanks Carlos.
>>>>>
>>>>>
>>>>> On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com> wrote:
>>>>>
>>>>>> Hi Kashyap,
>>>>>>
>>>>>> The kafka spout provided in the storm-contrib project (
>>>>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>>>>>> work with kafka 0.8
>>>>>>
>>>>>> The version provided by wurstmeister at
>>>>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to
>>>>>> work with storm 0.9.1 and kafka 0.8.
>>>>>>
>>>>>> Regards,
>>>>>> Carlos
>>>>>>
>>>>>>
>>>>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>>>>>
>>>>>>> Hi,
>>>>>>> What is the version of Kafka spout that works with storm 0.9 and
>>>>>>> kafka 0.8?
>>>>>>> The standard example doesn't throw exceptions but doesn't seem to
>>>>>>> consume messages either.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Kashyap
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>
>
Re: Kafka spout versions
Posted by Xing Yong <xy...@gmail.com>.
as the Exception "NoClassDefFoundError: storm/kafka/BrokerHos", i think the
jar of storm-kafka-0.8-plus and jar of kafka should be placed in the lib of
the storm
2014-04-15 5:21 GMT+08:00 M. Solaimani <so...@gmail.com>:
> Hi,
>
> I saw this mail and tried to run the Wurstmiesters storm-kafka-0.8-plus
> test. Here what I did:
>
> 1. I downloaded the Wurstmiesters storm-kafka-0.8-plus test and build it
> by using mvn clean install.
> 2. I had my storm cluster running. (Storm 0.9.0-rc2, Kakfka 0.8)
> 3. I had kafka 0.8 and I run kafka servers, with zoo-keeper and created a
> topic storm-sentence
>
> All the above steps running well.
>
> But When I run Wurstmiesters storm-kafka-0.8-plus test by the following
> commands
>
> storm jar target/storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar
> storm.kafka.KafkaSpoutTestTopology 10.176.148.51
> sentences 10.176.148.51
>
> I got the following errors:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> storm/kafka/BrokerHos
> ts
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
> at java.lang.Class.getMethod0(Class.java:2774)
> at java.lang.Class.getMethod(Class.java:1663)
> at
> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
> at
> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
> Caused by: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 6 more
>
>
> At this point I am stuck. I searched the internet and it says that there
> is some version mismatch issue? Can anyone help me please?
>
> With best regards,
> M Solaimani
>
>
>
>
>
>
>
> On Mon, Apr 14, 2014 at 11:58 AM, Kashyap Mhaisekar <ka...@gmail.com>wrote:
>
>> Wurstmiesters seems to work for me. I used storm 0.9.0 rc3
>>
>> Thanks,
>> Kashyap
>>
>>
>> On Monday, April 14, 2014, Trevor Madge <tm...@gmail.com> wrote:
>>
>>> I didn't have any luck with wurstmeister's but I've found this one (
>>> https://github.com/HolmesNL/kafka-spout) and it works with storm 0.9.1
>>> and kafka 0.8.1.
>>>
>>>
>>> -Trevor
>>>
>>>
>>>
>>> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <ka...@gmail.com>wrote:
>>>
>>>> Thanks Carlos.
>>>>
>>>>
>>>> On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com> wrote:
>>>>
>>>>> Hi Kashyap,
>>>>>
>>>>> The kafka spout provided in the storm-contrib project (
>>>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>>>>> work with kafka 0.8
>>>>>
>>>>> The version provided by wurstmeister at
>>>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to
>>>>> work with storm 0.9.1 and kafka 0.8.
>>>>>
>>>>> Regards,
>>>>> Carlos
>>>>>
>>>>>
>>>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>>>>
>>>>>> Hi,
>>>>>> What is the version of Kafka spout that works with storm 0.9 and
>>>>>> kafka 0.8?
>>>>>> The standard example doesn't throw exceptions but doesn't seem to
>>>>>> consume messages either.
>>>>>>
>>>>>> Thanks,
>>>>>> Kashyap
>>>>>>
>>>>>
>>>>>
>>>
>
Re: Kafka spout versions
Posted by "M. Solaimani" <so...@gmail.com>.
Hi,
I saw this mail and tried to run the Wurstmiesters storm-kafka-0.8-plus
test. Here what I did:
1. I downloaded the Wurstmiesters storm-kafka-0.8-plus test and build it
by using mvn clean install.
2. I had my storm cluster running. (Storm 0.9.0-rc2, Kakfka 0.8)
3. I had kafka 0.8 and I run kafka servers, with zoo-keeper and created a
topic storm-sentence
All the above steps running well.
But When I run Wurstmiesters storm-kafka-0.8-plus test by the following
commands
storm jar target/storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar
storm.kafka.KafkaSpoutTestTopology 10.176.148.51
sentences 10.176.148.51
I got the following errors:
Exception in thread "main" java.lang.NoClassDefFoundError:
storm/kafka/BrokerHos
ts
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
at java.lang.Class.getMethod0(Class.java:2774)
at java.lang.Class.getMethod(Class.java:1663)
at
sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at
sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
At this point I am stuck. I searched the internet and it says that there is
some version mismatch issue? Can anyone help me please?
With best regards,
M Solaimani
On Mon, Apr 14, 2014 at 11:58 AM, Kashyap Mhaisekar <ka...@gmail.com>wrote:
> Wurstmiesters seems to work for me. I used storm 0.9.0 rc3
>
> Thanks,
> Kashyap
>
>
> On Monday, April 14, 2014, Trevor Madge <tm...@gmail.com> wrote:
>
>> I didn't have any luck with wurstmeister's but I've found this one (
>> https://github.com/HolmesNL/kafka-spout) and it works with storm 0.9.1
>> and kafka 0.8.1.
>>
>>
>> -Trevor
>>
>>
>>
>> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <ka...@gmail.com>wrote:
>>
>>> Thanks Carlos.
>>>
>>>
>>> On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com> wrote:
>>>
>>>> Hi Kashyap,
>>>>
>>>> The kafka spout provided in the storm-contrib project (
>>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>>>> work with kafka 0.8
>>>>
>>>> The version provided by wurstmeister at
>>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to
>>>> work with storm 0.9.1 and kafka 0.8.
>>>>
>>>> Regards,
>>>> Carlos
>>>>
>>>>
>>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>>>
>>>>> Hi,
>>>>> What is the version of Kafka spout that works with storm 0.9 and kafka
>>>>> 0.8?
>>>>> The standard example doesn't throw exceptions but doesn't seem to
>>>>> consume messages either.
>>>>>
>>>>> Thanks,
>>>>> Kashyap
>>>>>
>>>>
>>>>
>>
Re: Kafka spout versions
Posted by Kashyap Mhaisekar <ka...@gmail.com>.
Wurstmiesters seems to work for me. I used storm 0.9.0 rc3
Thanks,
Kashyap
On Monday, April 14, 2014, Trevor Madge <tm...@gmail.com> wrote:
> I didn't have any luck with wurstmeister's but I've found this one (
> https://github.com/HolmesNL/kafka-spout) and it works with storm 0.9.1
> and kafka 0.8.1.
>
>
> -Trevor
>
>
>
> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <kashyap.m@gmail.com<javascript:_e(%7B%7D,'cvml','kashyap.m@gmail.com');>
> > wrote:
>
>> Thanks Carlos.
>>
>>
>> On Friday, April 11, 2014, Carlos Rodriguez <crresse@gmail.com<javascript:_e(%7B%7D,'cvml','crresse@gmail.com');>>
>> wrote:
>>
>>> Hi Kashyap,
>>>
>>> The kafka spout provided in the storm-contrib project (
>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>>> work with kafka 0.8
>>>
>>> The version provided by wurstmeister at
>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to
>>> work with storm 0.9.1 and kafka 0.8.
>>>
>>> Regards,
>>> Carlos
>>>
>>>
>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>>
>>>> Hi,
>>>> What is the version of Kafka spout that works with storm 0.9 and kafka
>>>> 0.8?
>>>> The standard example doesn't throw exceptions but doesn't seem to
>>>> consume messages either.
>>>>
>>>> Thanks,
>>>> Kashyap
>>>>
>>>
>>>
>
Re: Kafka spout versions
Posted by Trevor Madge <tm...@gmail.com>.
I didn't have any luck with wurstmeister's but I've found this one (
https://github.com/HolmesNL/kafka-spout) and it works with storm 0.9.1 and
kafka 0.8.1.
-Trevor
On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar <ka...@gmail.com>wrote:
> Thanks Carlos.
>
>
> On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com> wrote:
>
>> Hi Kashyap,
>>
>> The kafka spout provided in the storm-contrib project (
>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
>> work with kafka 0.8
>>
>> The version provided by wurstmeister at
>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to work
>> with storm 0.9.1 and kafka 0.8.
>>
>> Regards,
>> Carlos
>>
>>
>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
>>
>>> Hi,
>>> What is the version of Kafka spout that works with storm 0.9 and kafka
>>> 0.8?
>>> The standard example doesn't throw exceptions but doesn't seem to
>>> consume messages either.
>>>
>>> Thanks,
>>> Kashyap
>>>
>>
>>
Re: Kafka spout versions
Posted by Kashyap Mhaisekar <ka...@gmail.com>.
Thanks Carlos.
On Friday, April 11, 2014, Carlos Rodriguez <cr...@gmail.com> wrote:
> Hi Kashyap,
>
> The kafka spout provided in the storm-contrib project (
> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
> work with kafka 0.8
>
> The version provided by wurstmeister at
> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to work
> with storm 0.9.1 and kafka 0.8.
>
> Regards,
> Carlos
>
>
> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <kashyap.m@gmail.com<javascript:_e(%7B%7D,'cvml','kashyap.m@gmail.com');>
> >:
>
>> Hi,
>> What is the version of Kafka spout that works with storm 0.9 and kafka
>> 0.8?
>> The standard example doesn't throw exceptions but doesn't seem to consume
>> messages either.
>>
>> Thanks,
>> Kashyap
>>
>
>
Re: Kafka spout versions
Posted by Carlos Rodriguez <cr...@gmail.com>.
Hi Kashyap,
The kafka spout provided in the storm-contrib project (
https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) doesnt
work with kafka 0.8
The version provided by wurstmeister at
https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed to work
with storm 0.9.1 and kafka 0.8.
Regards,
Carlos
2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <ka...@gmail.com>:
> Hi,
> What is the version of Kafka spout that works with storm 0.9 and kafka 0.8?
> The standard example doesn't throw exceptions but doesn't seem to consume
> messages either.
>
> Thanks,
> Kashyap
>