You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Abhishek Bhattacharjee <ab...@gmail.com> on 2014/06/06 01:40:23 UTC
Kafka-Storm Run-time Exception
I am using kafka with storm. I am using maven to build my topology and I am
using scala 2.9.2 same as I am using kafka_2.9.2_0.8.1.
Topology build perfectly using maven. But hwn I submit the topology to
storm I get the following Exception:
java.lang.NoSuchMethodError: scala.Predef$.int2Integer(I)Ljava/lang/Integer;
at kafka.api.OffsetRequest$.(OffsetRequest.scala:28)
at kafka.api.OffsetRequest$.(OffsetRequest.scala)
at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala:36)
at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala)
at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
at storm.ubiquitous.spouts.KafkaConsumer.fetchdata(KafkaConsumer.java:41)
at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:92)
at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:68)
at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter$1.init(PartitionedTransactionalSpoutExecutor.java:77)
at backtype.storm.transactional.state.RotatingTransactionalState.getState(RotatingTransactionalState.java:82)
at backtype.storm.transactional.state.RotatingTransactionalState.getStateOrCreate(RotatingTransactionalState.java:103)
at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:73)
at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:50)
at backtype.storm.transactional.TransactionalSpoutBatchExecutor.execute(TransactionalSpoutBatchExecutor.java:48)
at backtype.storm.coordination.CoordinatedBolt.execute(CoordinatedBolt.java:308)
at backtype.storm.daemon.executor$fn__3498$tuple_action_fn__3500.invoke(executor.clj:615)
at backtype.storm.daemon.executor$mk_task_receiver$fn__3421.invoke(executor.clj:383)
at backtype.storm.disruptor$clojure_handler$reify__2962.onEvent(disruptor.clj:43)
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87)
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:61)
at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
at backtype.storm.daemon.executor$fn__3498$fn__3510$fn__3557.invoke(executor.clj:730)
at backtype.storm.util$async_loop$fn__444.invoke(util.clj:403)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:724)
I searched around and realized that its version related error. But both my
compile-time and run-time Scala versions are same(2.9.2) still I am getting
the error.
Any help is appreciated !
Thanks
--
*Abhishek Bhattacharjee*
*Pune Institute of Computer Technology*
Re: Kafka-Storm Run-time Exception
Posted by Abhishek Bhattacharjee <ab...@gmail.com>.
hi, thanks for the reply.
I tried the solution although it is not working I am getting the same
Exception.
here's the output of my: >> mvn dependency:tree | grep "scala"
[INFO] | +- org.scala-lang:scala-compiler:jar:2.9.2:compile
[INFO] +- org.scala-lang:scala-library:jar:2.9.2:compile
I don't know what's going wrong !
Thanks
On Fri, Jun 6, 2014 at 6:50 AM, Andrew Neilson <ar...@gmail.com> wrote:
> it's possible you have some other dependency using an earlier version of
> Scala. A common one to check for when using Kafka is jline 0.9.94, which
> comes through the zookeeper 3.3.4 dependency included with
> kafka_2.9.2-0.8.1 and has more than one dependency that uses scala 2.8.x.
>
> If this is where it's coming from, a solution is to just exclude jline from
> the kafka/storm-kafka dependency in your pom.xml. ex:
>
> <dependency>
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_2.9.2</artifactId>
> <version>0.8.1</version>
> <exclusions>
> <exclusion>
> <groupId>jline</groupId>
> <artifactId>jline</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
>
> You can also check your dependency tree and look for other references to
> other scala versions (mvn dependency:tree).
>
>
> On Thu, Jun 5, 2014 at 4:40 PM, Abhishek Bhattacharjee <
> abhishek.bhattacharjee11@gmail.com> wrote:
>
> > I am using kafka with storm. I am using maven to build my topology and I
> > am using scala 2.9.2 same as I am using kafka_2.9.2_0.8.1.
> >
> > Topology build perfectly using maven. But hwn I submit the topology to
> > storm I get the following Exception:
> >
> > java.lang.NoSuchMethodError:
> scala.Predef$.int2Integer(I)Ljava/lang/Integer;
> > at kafka.api.OffsetRequest$.(OffsetRequest.scala:28)
> > at kafka.api.OffsetRequest$.(OffsetRequest.scala)
> > at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala:36)
> > at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala)
> > at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> > at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> > at
> storm.ubiquitous.spouts.KafkaConsumer.fetchdata(KafkaConsumer.java:41)
> > at
> storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:92)
> > at
> storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:68)
> > at
> backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter$1.init(PartitionedTransactionalSpoutExecutor.java:77)
> > at
> backtype.storm.transactional.state.RotatingTransactionalState.getState(RotatingTransactionalState.java:82)
> > at
> backtype.storm.transactional.state.RotatingTransactionalState.getStateOrCreate(RotatingTransactionalState.java:103)
> > at
> backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:73)
> > at
> backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:50)
> > at
> backtype.storm.transactional.TransactionalSpoutBatchExecutor.execute(TransactionalSpoutBatchExecutor.java:48)
> > at
> backtype.storm.coordination.CoordinatedBolt.execute(CoordinatedBolt.java:308)
> > at
> backtype.storm.daemon.executor$fn__3498$tuple_action_fn__3500.invoke(executor.clj:615)
> > at
> backtype.storm.daemon.executor$mk_task_receiver$fn__3421.invoke(executor.clj:383)
> > at
> backtype.storm.disruptor$clojure_handler$reify__2962.onEvent(disruptor.clj:43)
> > at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87)
> > at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:61)
> > at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
> > at
> backtype.storm.daemon.executor$fn__3498$fn__3510$fn__3557.invoke(executor.clj:730)
> > at backtype.storm.util$async_loop$fn__444.invoke(util.clj:403)
> > at clojure.lang.AFn.run(AFn.java:24)
> > at java.lang.Thread.run(Thread.java:724)
> >
> > I searched around and realized that its version related error. But both
> my
> > compile-time and run-time Scala versions are same(2.9.2) still I am
> getting
> > the error.
> >
> > Any help is appreciated !
> >
> > Thanks
> > --
> > *Abhishek Bhattacharjee*
> > *Pune Institute of Computer Technology*
> >
>
--
*Abhishek Bhattacharjee*
*Pune Institute of Computer Technology*
Re: Kafka-Storm Run-time Exception
Posted by Andrew Neilson <ar...@gmail.com>.
it's possible you have some other dependency using an earlier version of
Scala. A common one to check for when using Kafka is jline 0.9.94, which
comes through the zookeeper 3.3.4 dependency included with
kafka_2.9.2-0.8.1 and has more than one dependency that uses scala 2.8.x.
If this is where it's coming from, a solution is to just exclude jline from
the kafka/storm-kafka dependency in your pom.xml. ex:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.9.2</artifactId>
<version>0.8.1</version>
<exclusions>
<exclusion>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
</exclusion>
</exclusions>
</dependency>
You can also check your dependency tree and look for other references to
other scala versions (mvn dependency:tree).
On Thu, Jun 5, 2014 at 4:40 PM, Abhishek Bhattacharjee <
abhishek.bhattacharjee11@gmail.com> wrote:
> I am using kafka with storm. I am using maven to build my topology and I
> am using scala 2.9.2 same as I am using kafka_2.9.2_0.8.1.
>
> Topology build perfectly using maven. But hwn I submit the topology to
> storm I get the following Exception:
>
> java.lang.NoSuchMethodError: scala.Predef$.int2Integer(I)Ljava/lang/Integer;
> at kafka.api.OffsetRequest$.(OffsetRequest.scala:28)
> at kafka.api.OffsetRequest$.(OffsetRequest.scala)
> at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala:36)
> at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala)
> at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> at storm.ubiquitous.spouts.KafkaConsumer.fetchdata(KafkaConsumer.java:41)
> at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:92)
> at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:68)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter$1.init(PartitionedTransactionalSpoutExecutor.java:77)
> at backtype.storm.transactional.state.RotatingTransactionalState.getState(RotatingTransactionalState.java:82)
> at backtype.storm.transactional.state.RotatingTransactionalState.getStateOrCreate(RotatingTransactionalState.java:103)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:73)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:50)
> at backtype.storm.transactional.TransactionalSpoutBatchExecutor.execute(TransactionalSpoutBatchExecutor.java:48)
> at backtype.storm.coordination.CoordinatedBolt.execute(CoordinatedBolt.java:308)
> at backtype.storm.daemon.executor$fn__3498$tuple_action_fn__3500.invoke(executor.clj:615)
> at backtype.storm.daemon.executor$mk_task_receiver$fn__3421.invoke(executor.clj:383)
> at backtype.storm.disruptor$clojure_handler$reify__2962.onEvent(disruptor.clj:43)
> at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87)
> at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:61)
> at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
> at backtype.storm.daemon.executor$fn__3498$fn__3510$fn__3557.invoke(executor.clj:730)
> at backtype.storm.util$async_loop$fn__444.invoke(util.clj:403)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:724)
>
> I searched around and realized that its version related error. But both my
> compile-time and run-time Scala versions are same(2.9.2) still I am getting
> the error.
>
> Any help is appreciated !
>
> Thanks
> --
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>
Re: Kafka-Storm Run-time Exception
Posted by Andrew Neilson <ar...@gmail.com>.
it's possible you have some other dependency using an earlier version of
Scala. A common one to check for when using Kafka is jline 0.9.94, which
comes through the zookeeper 3.3.4 dependency included with
kafka_2.9.2-0.8.1 and has more than one dependency that uses scala 2.8.x.
If this is where it's coming from, a solution is to just exclude jline from
the kafka/storm-kafka dependency in your pom.xml. ex:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.9.2</artifactId>
<version>0.8.1</version>
<exclusions>
<exclusion>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
</exclusion>
</exclusions>
</dependency>
You can also check your dependency tree and look for other references to
other scala versions (mvn dependency:tree).
On Thu, Jun 5, 2014 at 4:40 PM, Abhishek Bhattacharjee <
abhishek.bhattacharjee11@gmail.com> wrote:
> I am using kafka with storm. I am using maven to build my topology and I
> am using scala 2.9.2 same as I am using kafka_2.9.2_0.8.1.
>
> Topology build perfectly using maven. But hwn I submit the topology to
> storm I get the following Exception:
>
> java.lang.NoSuchMethodError: scala.Predef$.int2Integer(I)Ljava/lang/Integer;
> at kafka.api.OffsetRequest$.(OffsetRequest.scala:28)
> at kafka.api.OffsetRequest$.(OffsetRequest.scala)
> at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala:36)
> at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala)
> at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> at storm.ubiquitous.spouts.KafkaConsumer.fetchdata(KafkaConsumer.java:41)
> at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:92)
> at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:68)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter$1.init(PartitionedTransactionalSpoutExecutor.java:77)
> at backtype.storm.transactional.state.RotatingTransactionalState.getState(RotatingTransactionalState.java:82)
> at backtype.storm.transactional.state.RotatingTransactionalState.getStateOrCreate(RotatingTransactionalState.java:103)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:73)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:50)
> at backtype.storm.transactional.TransactionalSpoutBatchExecutor.execute(TransactionalSpoutBatchExecutor.java:48)
> at backtype.storm.coordination.CoordinatedBolt.execute(CoordinatedBolt.java:308)
> at backtype.storm.daemon.executor$fn__3498$tuple_action_fn__3500.invoke(executor.clj:615)
> at backtype.storm.daemon.executor$mk_task_receiver$fn__3421.invoke(executor.clj:383)
> at backtype.storm.disruptor$clojure_handler$reify__2962.onEvent(disruptor.clj:43)
> at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87)
> at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:61)
> at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
> at backtype.storm.daemon.executor$fn__3498$fn__3510$fn__3557.invoke(executor.clj:730)
> at backtype.storm.util$async_loop$fn__444.invoke(util.clj:403)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:724)
>
> I searched around and realized that its version related error. But both my
> compile-time and run-time Scala versions are same(2.9.2) still I am getting
> the error.
>
> Any help is appreciated !
>
> Thanks
> --
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>
Re: Kafka-Storm Run-time Exception
Posted by Andrew Neilson <ar...@gmail.com>.
it's possible you have some other dependency using an earlier version of
Scala. A common one to check for when using Kafka is jline 0.9.94, which
comes through the zookeeper 3.3.4 dependency included with
kafka_2.9.2-0.8.1 and has more than one dependency that uses scala 2.8.x.
If this is where it's coming from, a solution is to just exclude jline from
the kafka/storm-kafka dependency in your pom.xml. ex:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.9.2</artifactId>
<version>0.8.1</version>
<exclusions>
<exclusion>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
</exclusion>
</exclusions>
</dependency>
You can also check your dependency tree and look for other references to
other scala versions (mvn dependency:tree).
On Thu, Jun 5, 2014 at 4:40 PM, Abhishek Bhattacharjee <
abhishek.bhattacharjee11@gmail.com> wrote:
> I am using kafka with storm. I am using maven to build my topology and I
> am using scala 2.9.2 same as I am using kafka_2.9.2_0.8.1.
>
> Topology build perfectly using maven. But hwn I submit the topology to
> storm I get the following Exception:
>
> java.lang.NoSuchMethodError: scala.Predef$.int2Integer(I)Ljava/lang/Integer;
> at kafka.api.OffsetRequest$.(OffsetRequest.scala:28)
> at kafka.api.OffsetRequest$.(OffsetRequest.scala)
> at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala:36)
> at kafka.consumer.ConsumerConfig$.(ConsumerConfig.scala)
> at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:35)
> at storm.ubiquitous.spouts.KafkaConsumer.fetchdata(KafkaConsumer.java:41)
> at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:92)
> at storm.ubiquitous.spouts.KafkaSpoutTransaction$KafkaPartitionedEmitter.emitPartitionBatchNew(KafkaSpoutTransaction.java:68)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter$1.init(PartitionedTransactionalSpoutExecutor.java:77)
> at backtype.storm.transactional.state.RotatingTransactionalState.getState(RotatingTransactionalState.java:82)
> at backtype.storm.transactional.state.RotatingTransactionalState.getStateOrCreate(RotatingTransactionalState.java:103)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:73)
> at backtype.storm.transactional.partitioned.PartitionedTransactionalSpoutExecutor$Emitter.emitBatch(PartitionedTransactionalSpoutExecutor.java:50)
> at backtype.storm.transactional.TransactionalSpoutBatchExecutor.execute(TransactionalSpoutBatchExecutor.java:48)
> at backtype.storm.coordination.CoordinatedBolt.execute(CoordinatedBolt.java:308)
> at backtype.storm.daemon.executor$fn__3498$tuple_action_fn__3500.invoke(executor.clj:615)
> at backtype.storm.daemon.executor$mk_task_receiver$fn__3421.invoke(executor.clj:383)
> at backtype.storm.disruptor$clojure_handler$reify__2962.onEvent(disruptor.clj:43)
> at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87)
> at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:61)
> at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
> at backtype.storm.daemon.executor$fn__3498$fn__3510$fn__3557.invoke(executor.clj:730)
> at backtype.storm.util$async_loop$fn__444.invoke(util.clj:403)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:724)
>
> I searched around and realized that its version related error. But both my
> compile-time and run-time Scala versions are same(2.9.2) still I am getting
> the error.
>
> Any help is appreciated !
>
> Thanks
> --
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>