[Issue] OutOfMemoryError when disable ackers

["jamesweb3@yahoo.com.tw" <ja...@yahoo.com.tw>]

Hi all,
I use storm 0.9.2 now, and do real-time processing by basic storm. It's normal to use in reliable way. But the speed is extremely slow about 6000 tuples/s. So I try to disable acker feature by conf.setNumAckers(0). However, this leads to following error:

java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.reflect.Method.copy(Method.java:151) at java.lang.reflect.ReflectAccess.copyMethod(ReflectAccess.java:136) at sun.reflect.ReflectionFactory.copyMethod(ReflectionFactory.java:300) at java.lang.Class.copyMethods(Class.java:2891) at java.lang.Class.getMethods(Class.java:1467) at clojure.lang.Reflector.getMethods(Reflector.java:357) at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:27) at backtype.storm.daemon.worker$mk_transfer_fn$fn__5748.invoke(worker.clj:128) at backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__5483.invoke(executor.clj:256) at backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) at backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94) at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)


Can anyone tell me why this happens?
Thank you very much.

Best regards,
James Fu

Re: [Issue] OutOfMemoryError when disable ackers

["Cody A. Ray" <co...@gmail.com>]

My hunch would be that you're anchoring your tuples and without acker tasks
these tuple trees are never destroyed. So all the tuples from each batch
are kept in memory until you run out of memory, i.e., a very, very fast
memory leak.

If you disable ackers, you should probably not anchor your tuples either.

-Cody


On Mon, Jun 30, 2014 at 9:56 PM, jamesweb3@yahoo.com.tw <
jamesweb3@yahoo.com.tw> wrote:

>  Hi all,
>
> I use storm 0.9.2 now, and do real-time processing by basic storm. It's
> normal to use in reliable way. But the speed is extremely slow about 6000
> tuples/s. So I try to disable acker feature by conf.setNumAckers(0).
> However, this leads to following error:
>
> java.lang.OutOfMemoryError: GC overhead limit exceeded at
> java.lang.reflect.Method.copy(Method.java:151) at
> java.lang.reflect.ReflectAccess.copyMethod(ReflectAccess.java:136) at
> sun.reflect.ReflectionFactory.copyMethod(ReflectionFactory.java:300) at
> java.lang.Class.copyMethods(Class.java:2891) at
> java.lang.Class.getMethods(Class.java:1467) at
> clojure.lang.Reflector.getMethods(Reflector.java:357) at
> clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:27) at
> backtype.storm.daemon.worker$mk_transfer_fn$fn__5748.invoke(worker.clj:128)
> at
> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__5483.invoke(executor.clj:256)
> at
> backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
> at
> backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94)
> at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) at
> clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:745)
>
>
> Can anyone tell me why this happens?
> Thank you very much.
>
> Best regards,
> James Fu
>
>


-- 
Cody A. Ray, LEED AP
cody.a.ray@gmail.com
215.501.7891