Kafka exception on Toplogy start

[abe oppenheim <ab...@gmail.com>]

I realize this is a little vague, but any advice as to how I can
troubleshoot this would be very helpful.

My Topology uses a KafkaSpout. When I start the Topology I see the below
exception and all my Executors die. Then the Worker tries to start again,
the exception occurs, and the Executors die. This continues with no
resolution, the Worker's uptime never exceeds a few seconds.


java.lang.IndexOutOfBoundsException: null
    at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_11]
    at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:359) ~[na:1.7.0_11]
    at kafka.message.Message.sliceDelimited(Message.scala:229)
~[stormjar.jar:na]
    at kafka.message.Message.payload(Message.scala:218) ~[stormjar.jar:na]
    at storm.kafka.KafkaUtils.generateTuples(KafkaUtils.java:201)
~[stormjar.jar:na]
    at storm.kafka.PartitionManager.next(PartitionManager.java:131)
~[stormjar.jar:na]
    at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:141) ~[stormjar.jar:na]
    at backtype.storm.daemon.executor$fn__4654$fn__4669$fn__4698.invoke(executor.clj:565)
~[storm-core-0.9.4.jar:0.9.4]
    at backtype.storm.util$async_loop$fn__458.invoke(util.clj:463)
~[storm-core-0.9.4.jar:0.9.4]
    at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
    at java.lang.Thread.run(Thread.java:722) [na:1.7.0_11]

Re: Kafka exception on Toplogy start

[임정택 <ka...@gmail.com>]

Hi Abe,

Having a look into stack trace and trace some source codes, I feel that
Message itself could be broken.
(Or maybe offset is not valid. I'm just using Kafka so I couldn't know the
details.)

Message class is from Kafka, and storm-kafka just seems to fetch them from
Kafka, and doesn't modify it.
(Also Message seems to be immutable.
https://github.com/apache/kafka/blob/0.8.1.1/core/src/main/scala/kafka/message/Message.scala
)

Could you try consume same offset without KafkaSpout, and call
Message.isValid() to check its validity?

Thanks,
Jungtaek Lim (HeartSaVioR)


2015-10-30 21:47 GMT+09:00 abe oppenheim <ab...@gmail.com>:

> I realize this is a little vague, but any advice as to how I can
> troubleshoot this would be very helpful.
>
> My Topology uses a KafkaSpout. When I start the Topology I see the below
> exception and all my Executors die. Then the Worker tries to start again,
> the exception occurs, and the Executors die. This continues with no
> resolution, the Worker's uptime never exceeds a few seconds.
>
>
> java.lang.IndexOutOfBoundsException: null
>     at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_11]
>     at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:359)
> ~[na:1.7.0_11]
>     at kafka.message.Message.sliceDelimited(Message.scala:229)
> ~[stormjar.jar:na]
>     at kafka.message.Message.payload(Message.scala:218) ~[stormjar.jar:na]
>     at storm.kafka.KafkaUtils.generateTuples(KafkaUtils.java:201)
> ~[stormjar.jar:na]
>     at storm.kafka.PartitionManager.next(PartitionManager.java:131)
> ~[stormjar.jar:na]
>     at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:141)
> ~[stormjar.jar:na]
>     at
> backtype.storm.daemon.executor$fn__4654$fn__4669$fn__4698.invoke(executor.clj:565)
> ~[storm-core-0.9.4.jar:0.9.4]
>     at backtype.storm.util$async_loop$fn__458.invoke(util.clj:463)
> ~[storm-core-0.9.4.jar:0.9.4]
>     at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
>     at java.lang.Thread.run(Thread.java:722) [na:1.7.0_11]
>



-- 
Name : 임 정택
Blog : http://www.heartsavior.net / http://dev.heartsavior.net
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior