You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Irek Khasyanov <qu...@gmail.com> on 2014/05/21 13:24:40 UTC
Sometimes topology crashed with internal exception
Hello.
I have strange problem with by topology, sometimes everything crashed with
exception:
java.lang.RuntimeException: java.lang.NullPointerException
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
at
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
at
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
at
backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at
backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
at
backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
at backtype.storm.util$fast_list_map.invoke(util.clj:832)
at
backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
at
backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
at
backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
... 6 more
First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus
0.4.0, zookeeper 3.3.6, oracle java 1.6
I have toplogy: kafka spout -> database bolt -> database bolt. Second
database bolt commits to another table, with data emmited from first bolt.
I saw wiki page
https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-stormbut
I can't say I'm doing everything wrong. I'm ack'ing right after
execute() call, for example:
@Override
public void execute(Tuple tuple) {
this.collector.ack(tuple);
this.queue.offer(tuple);
}
And then queue in different thread will be commited to database.
Everything works well for different time, topology can crash after few
minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
What can be a problem? Where I should look into?
--
With best regards, Irek Khasyanov.
Re: Sometimes topology crashed with internal exception
Posted by Irek Khasyanov <qu...@gmail.com>.
Nathan, it works well for longer time than before. About 2m tuples recieved
and 20m tuples emitted, when I see exception again. I will try to change
code and remove synchronized block and see what happens.
Is it possible what kafka 0.8.1 has some bugs and I should upgrade to
0.8.1.1 ? Or bugs only with my code?
On 21 May 2014 20:41, Nathan Leung <nc...@gmail.com> wrote:
> You can also synchronize access to the OutputCollector so that only 1
> thread is using it at a time.
>
>
> On Wed, May 21, 2014 at 12:39 PM, Irek Khasyanov <qu...@gmail.com> wrote:
>
>> Hm, yes, firs bolt emitting from different thread, I did't realize that
>> this will be problem. Thanks! I'll try to change everything and see errors.
>>
>>
>>
>> On 21 May 2014 18:01, P. Taylor Goetz <pt...@gmail.com> wrote:
>>
>>> Are you using the OutputCollector to emit in a separate thread (i.e.
>>> Outside of the execute() method.)?
>>>
>>> As the wiki states, this will cause the problem you are seeing?
>>>
>>> -Taylor
>>>
>>> On May 21, 2014, at 7:24 AM, Irek Khasyanov <qu...@gmail.com> wrote:
>>>
>>> Hello.
>>>
>>> I have strange problem with by topology, sometimes everything crashed
>>> with exception:
>>>
>>> java.lang.RuntimeException: java.lang.NullPointerException
>>> at
>>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
>>> at
>>> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
>>> at
>>> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
>>> at
>>> backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
>>> at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
>>> at clojure.lang.AFn.run(AFn.java:24)
>>> at java.lang.Thread.run(Thread.java:662)
>>> Caused by: java.lang.NullPointerException
>>> at
>>> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
>>> at
>>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
>>> at backtype.storm.util$fast_list_map.invoke(util.clj:832)
>>> at
>>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
>>> at
>>> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
>>> at
>>> backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
>>> at
>>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
>>> ... 6 more
>>>
>>> First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus
>>> 0.4.0, zookeeper 3.3.6, oracle java 1.6
>>>
>>> I have toplogy: kafka spout -> database bolt -> database bolt. Second
>>> database bolt commits to another table, with data emmited from first bolt.
>>>
>>> I saw wiki page
>>> https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-stormbut I can't say I'm doing everything wrong. I'm ack'ing right after
>>> execute() call, for example:
>>>
>>> @Override
>>> public void execute(Tuple tuple) {
>>> this.collector.ack(tuple);
>>> this.queue.offer(tuple);
>>> }
>>>
>>>
>>>
>>> And then queue in different thread will be commited to database.
>>>
>>> Everything works well for different time, topology can crash after few
>>> minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
>>>
>>> What can be a problem? Where I should look into?
>>>
>>> --
>>> With best regards, Irek Khasyanov.
>>>
>>>
>>
>>
>> --
>> With best regards, Irek Khasyanov.
>>
>
>
--
With best regards, Irek Khasyanov.
Re: Sometimes topology crashed with internal exception
Posted by Nathan Leung <nc...@gmail.com>.
You can also synchronize access to the OutputCollector so that only 1
thread is using it at a time.
On Wed, May 21, 2014 at 12:39 PM, Irek Khasyanov <qu...@gmail.com> wrote:
> Hm, yes, firs bolt emitting from different thread, I did't realize that
> this will be problem. Thanks! I'll try to change everything and see errors.
>
>
>
> On 21 May 2014 18:01, P. Taylor Goetz <pt...@gmail.com> wrote:
>
>> Are you using the OutputCollector to emit in a separate thread (i.e.
>> Outside of the execute() method.)?
>>
>> As the wiki states, this will cause the problem you are seeing?
>>
>> -Taylor
>>
>> On May 21, 2014, at 7:24 AM, Irek Khasyanov <qu...@gmail.com> wrote:
>>
>> Hello.
>>
>> I have strange problem with by topology, sometimes everything crashed
>> with exception:
>>
>> java.lang.RuntimeException: java.lang.NullPointerException
>> at
>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
>> at
>> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
>> at
>> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
>> at
>> backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
>> at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
>> at clojure.lang.AFn.run(AFn.java:24)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.lang.NullPointerException
>> at
>> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
>> at
>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
>> at backtype.storm.util$fast_list_map.invoke(util.clj:832)
>> at
>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
>> at
>> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
>> at
>> backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
>> at
>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
>> ... 6 more
>>
>> First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus
>> 0.4.0, zookeeper 3.3.6, oracle java 1.6
>>
>> I have toplogy: kafka spout -> database bolt -> database bolt. Second
>> database bolt commits to another table, with data emmited from first bolt.
>>
>> I saw wiki page
>> https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-stormbut I can't say I'm doing everything wrong. I'm ack'ing right after
>> execute() call, for example:
>>
>> @Override
>> public void execute(Tuple tuple) {
>> this.collector.ack(tuple);
>> this.queue.offer(tuple);
>> }
>>
>>
>>
>> And then queue in different thread will be commited to database.
>>
>> Everything works well for different time, topology can crash after few
>> minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
>>
>> What can be a problem? Where I should look into?
>>
>> --
>> With best regards, Irek Khasyanov.
>>
>>
>
>
> --
> With best regards, Irek Khasyanov.
>
Re: Sometimes topology crashed with internal exception
Posted by Irek Khasyanov <qu...@gmail.com>.
Hm, yes, firs bolt emitting from different thread, I did't realize that
this will be problem. Thanks! I'll try to change everything and see errors.
On 21 May 2014 18:01, P. Taylor Goetz <pt...@gmail.com> wrote:
> Are you using the OutputCollector to emit in a separate thread (i.e.
> Outside of the execute() method.)?
>
> As the wiki states, this will cause the problem you are seeing?
>
> -Taylor
>
> On May 21, 2014, at 7:24 AM, Irek Khasyanov <qu...@gmail.com> wrote:
>
> Hello.
>
> I have strange problem with by topology, sometimes everything crashed with
> exception:
>
> java.lang.RuntimeException: java.lang.NullPointerException
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
> at
> backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
> at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.NullPointerException
> at
> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
> at
> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
> at backtype.storm.util$fast_list_map.invoke(util.clj:832)
> at
> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
> at
> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
> at
> backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
> ... 6 more
>
> First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus
> 0.4.0, zookeeper 3.3.6, oracle java 1.6
>
> I have toplogy: kafka spout -> database bolt -> database bolt. Second
> database bolt commits to another table, with data emmited from first bolt.
>
> I saw wiki page
> https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-stormbut I can't say I'm doing everything wrong. I'm ack'ing right after
> execute() call, for example:
>
> @Override
> public void execute(Tuple tuple) {
> this.collector.ack(tuple);
> this.queue.offer(tuple);
> }
>
>
>
> And then queue in different thread will be commited to database.
>
> Everything works well for different time, topology can crash after few
> minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
>
> What can be a problem? Where I should look into?
>
> --
> With best regards, Irek Khasyanov.
>
>
--
With best regards, Irek Khasyanov.
Re: Sometimes topology crashed with internal exception
Posted by "P. Taylor Goetz" <pt...@gmail.com>.
Are you using the OutputCollector to emit in a separate thread (i.e. Outside of the execute() method.)?
As the wiki states, this will cause the problem you are seeing?
-Taylor
> On May 21, 2014, at 7:24 AM, Irek Khasyanov <qu...@gmail.com> wrote:
>
> Hello.
>
> I have strange problem with by topology, sometimes everything crashed with exception:
>
> java.lang.RuntimeException: java.lang.NullPointerException
> at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
> at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
> at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
> at backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
> at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.NullPointerException
> at backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
> at backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
> at backtype.storm.util$fast_list_map.invoke(util.clj:832)
> at backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
> at backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
> at backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
> at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
> ... 6 more
>
> First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus 0.4.0, zookeeper 3.3.6, oracle java 1.6
>
> I have toplogy: kafka spout -> database bolt -> database bolt. Second database bolt commits to another table, with data emmited from first bolt.
>
> I saw wiki page https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-storm but I can't say I'm doing everything wrong. I'm ack'ing right after execute() call, for example:
>
> @Override
> public void execute(Tuple tuple) {
> this.collector.ack(tuple);
> this.queue.offer(tuple);
> }
>
>
>
> And then queue in different thread will be commited to database.
>
> Everything works well for different time, topology can crash after few minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
>
> What can be a problem? Where I should look into?
>
> --
> With best regards, Irek Khasyanov.
Re: Sometimes topology crashed with internal exception
Posted by Irek Khasyanov <qu...@gmail.com>.
With null value topology crashed with different exception:
java.lang.NullPointerException: null
at backtype.storm.utils.Utils.toByteArray(Utils.java:383)
~[storm-core-0.9.0.1.jar:na]
at storm.kafka.KafkaUtils.generateTuples(KafkaUtils.java:177)
~[storm-kafka-0.8-plus-0.4.0.jar:na]
at storm.kafka.PartitionManager.next(PartitionManager.java:115)
~[storm-kafka-0.8-plus-0.4.0.jar:na]
at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:123)
~[storm-kafka-0.8-plus-0.4.0.jar:na]
at
backtype.storm.daemon.executor$fn__3430$fn__3445$fn__3474.invoke(executor.clj:547)
~[storm-core-0.9.0.1.jar:na]
at backtype.storm.util$async_loop$fn__444.invoke(util.clj:403)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65]
442383 [Thread-25-event] ERROR backtype.storm.daemon.executor -
java.lang.NullPointerException: null
at backtype.storm.utils.Utils.toByteArray(Utils.java:383)
~[storm-core-0.9.0.1.jar:na]
at storm.kafka.KafkaUtils.generateTuples(KafkaUtils.java:177)
~[storm-kafka-0.8-plus-0.4.0.jar:na]
at storm.kafka.PartitionManager.next(PartitionManager.java:115)
~[storm-kafka-0.8-plus-0.4.0.jar:na]
at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:123)
~[storm-kafka-0.8-plus-0.4.0.jar:na]
at
backtype.storm.daemon.executor$fn__3430$fn__3445$fn__3474.invoke(executor.clj:547)
~[storm-core-0.9.0.1.jar:na]
at backtype.storm.util$async_loop$fn__444.invoke(util.clj:403)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65]
On 21 May 2014 17:05, Irek Khasyanov <qu...@gmail.com> wrote:
> Interesting question, I did't look into this direction. I will check it.
>
>
> On 21 May 2014 16:49, Nathan Leung <nc...@gmail.com> wrote:
>
>> Are you sure that you're not passing a null value in your tuple?
>> On May 21, 2014 7:25 AM, "Irek Khasyanov" <qu...@gmail.com> wrote:
>>
>>> Hello.
>>>
>>> I have strange problem with by topology, sometimes everything crashed
>>> with exception:
>>>
>>> java.lang.RuntimeException: java.lang.NullPointerException
>>> at
>>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
>>> at
>>> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
>>> at
>>> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
>>> at
>>> backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
>>> at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
>>> at clojure.lang.AFn.run(AFn.java:24)
>>> at java.lang.Thread.run(Thread.java:662)
>>> Caused by: java.lang.NullPointerException
>>> at
>>> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
>>> at
>>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
>>> at backtype.storm.util$fast_list_map.invoke(util.clj:832)
>>> at
>>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
>>> at
>>> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
>>> at
>>> backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
>>> at
>>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
>>> ... 6 more
>>>
>>> First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus
>>> 0.4.0, zookeeper 3.3.6, oracle java 1.6
>>>
>>> I have toplogy: kafka spout -> database bolt -> database bolt. Second
>>> database bolt commits to another table, with data emmited from first bolt.
>>>
>>> I saw wiki page
>>> https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-stormbut I can't say I'm doing everything wrong. I'm ack'ing right after
>>> execute() call, for example:
>>>
>>> @Override
>>> public void execute(Tuple tuple) {
>>> this.collector.ack(tuple);
>>> this.queue.offer(tuple);
>>> }
>>>
>>>
>>>
>>> And then queue in different thread will be commited to database.
>>>
>>> Everything works well for different time, topology can crash after few
>>> minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
>>>
>>> What can be a problem? Where I should look into?
>>>
>>> --
>>> With best regards, Irek Khasyanov.
>>>
>>
>
>
> --
> With best regards, Irek Khasyanov.
>
--
With best regards, Irek Khasyanov.
Re: Sometimes topology crashed with internal exception
Posted by Irek Khasyanov <qu...@gmail.com>.
Interesting question, I did't look into this direction. I will check it.
On 21 May 2014 16:49, Nathan Leung <nc...@gmail.com> wrote:
> Are you sure that you're not passing a null value in your tuple?
> On May 21, 2014 7:25 AM, "Irek Khasyanov" <qu...@gmail.com> wrote:
>
>> Hello.
>>
>> I have strange problem with by topology, sometimes everything crashed
>> with exception:
>>
>> java.lang.RuntimeException: java.lang.NullPointerException
>> at
>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
>> at
>> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
>> at
>> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
>> at
>> backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
>> at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
>> at clojure.lang.AFn.run(AFn.java:24)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.lang.NullPointerException
>> at
>> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
>> at
>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
>> at backtype.storm.util$fast_list_map.invoke(util.clj:832)
>> at
>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
>> at
>> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
>> at
>> backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
>> at
>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
>> ... 6 more
>>
>> First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus
>> 0.4.0, zookeeper 3.3.6, oracle java 1.6
>>
>> I have toplogy: kafka spout -> database bolt -> database bolt. Second
>> database bolt commits to another table, with data emmited from first bolt.
>>
>> I saw wiki page
>> https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-stormbut I can't say I'm doing everything wrong. I'm ack'ing right after
>> execute() call, for example:
>>
>> @Override
>> public void execute(Tuple tuple) {
>> this.collector.ack(tuple);
>> this.queue.offer(tuple);
>> }
>>
>>
>>
>> And then queue in different thread will be commited to database.
>>
>> Everything works well for different time, topology can crash after few
>> minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
>>
>> What can be a problem? Where I should look into?
>>
>> --
>> With best regards, Irek Khasyanov.
>>
>
--
With best regards, Irek Khasyanov.
Re: Sometimes topology crashed with internal exception
Posted by Nathan Leung <nc...@gmail.com>.
Are you sure that you're not passing a null value in your tuple?
On May 21, 2014 7:25 AM, "Irek Khasyanov" <qu...@gmail.com> wrote:
> Hello.
>
> I have strange problem with by topology, sometimes everything crashed with
> exception:
>
> java.lang.RuntimeException: java.lang.NullPointerException
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
> at
> backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
> at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.NullPointerException
> at
> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:41)
> at
> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217$fn__4221.invoke(worker.clj:123)
> at backtype.storm.util$fast_list_map.invoke(util.clj:832)
> at
> backtype.storm.daemon.worker$mk_transfer_fn$fn__4217.invoke(worker.clj:123)
> at
> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3746.invoke(executor.clj:255)
> at
> backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
> ... 6 more
>
> First of all, I'm using storm 0.9.1, kafka 0.8.1, storm-kafka-0.8-plus
> 0.4.0, zookeeper 3.3.6, oracle java 1.6
>
> I have toplogy: kafka spout -> database bolt -> database bolt. Second
> database bolt commits to another table, with data emmited from first bolt.
>
> I saw wiki page
> https://github.com/nathanmarz/storm/wiki/Troubleshooting#nullpointerexception-from-deep-inside-stormbut I can't say I'm doing everything wrong. I'm ack'ing right after
> execute() call, for example:
>
> @Override
> public void execute(Tuple tuple) {
> this.collector.ack(tuple);
> this.queue.offer(tuple);
> }
>
>
>
> And then queue in different thread will be commited to database.
>
> Everything works well for different time, topology can crash after few
> minutes or after 10 hours. I did't see anything wrong in ZK, kafka logs.
>
> What can be a problem? Where I should look into?
>
> --
> With best regards, Irek Khasyanov.
>