You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by "Rick Kellogg (JIRA)" <ji...@apache.org> on 2015/10/05 04:18:26 UTC
[jira] [Updated] (STORM-299) tick tuples stop in a long running
topology
[ https://issues.apache.org/jira/browse/STORM-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rick Kellogg updated STORM-299:
-------------------------------
Component/s: storm-core
> tick tuples stop in a long running topology
> -------------------------------------------
>
> Key: STORM-299
> URL: https://issues.apache.org/jira/browse/STORM-299
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 0.9.0.1
> Reporter: Sushant Kumar
>
> We are running a topology that processes about 10K tuples a second on a single worker. There are about 80 executor threads across 3 different Bolt types, each receiving a tick tuple at a frequency of 1 second. We observe that the tick tuples stop coming in after few hours or sometime even after a day. The nimbus UI also shows that the tick tuples count has frozen whereas the same set of bolts continue to process tuples from other spouts. I have taken a stack trace and it shows following:
> "Thread-8" daemon prio=10 tid=0x00007f1a0fed2000 nid=0x4438 runnable [0x00007f19a5e01000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> at com.lmax.disruptor.AbstractMultithreadedClaimStrategy.waitForFreeSlotAt(AbstractMultithreadedClaimStrategy.java:99)
> at com.lmax.disruptor.AbstractMultithreadedClaimStrategy.incrementAndGet(AbstractMultithreadedClaimStrategy.java:49)
> at com.lmax.disruptor.Sequencer.next(Sequencer.java:127)
> at backtype.storm.utils.DisruptorQueue.publish(DisruptorQueue.java:113)
> at backtype.storm.disruptor$publish.invoke(disruptor.clj:51)
> at backtype.storm.disruptor$publish.invoke(disruptor.clj:53)
> at backtype.storm.daemon.executor$setup_ticks_BANG_$fn__3885.invoke(executor.clj:299)
> at backtype.storm.timer$schedule_recurring$this__1839.invoke(timer.clj:79)
> at backtype.storm.timer$mk_timer$thread_fn__1822$fn__1823.invoke(timer.clj:32)
> at backtype.storm.timer$mk_timer$thread_fn__1822.invoke(timer.clj:25)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:744)
>
> Locked ownable synchronizers:
> - None
>
> "Thread-268-__system" prio=10 tid=0x00007f1a0ef45000 nid=0x46b5 runnable [0x00007f17ef574000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000073d9a2060> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2176)
> at com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:87)
> at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:54)
> at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:56)
> at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
> at backtype.storm.daemon.executor$eval4023$fn__4024$fn__4038$fn__4089.invoke(executor.clj:749)
> at backtype.storm.util$async_loop$fn__364.invoke(util.clj:400)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:744)
>
> Locked ownable synchronizers:
> - None
> We are running storm-0.9.0_wip21 and I would be glad to provide any other information.
> Also see https://issues.apache.org/jira/browse/STORM-106.
> Thanks,
> Sushant
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)