You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Jie Li <ja...@gmail.com> on 2014/06/04 21:17:51 UTC
storm topology stuck at exactly 26 emits and 17 acks with trident-redis
Hi all,
Not sure if any came across this. I have a trident topology running with
TransactionalTridentKafkaSpout and MemoryMapState for 20 days, but after I
switched to RedisState (tried both transactional and non-transactional), it
consistenly got stuck at exactly 26 emits and 17 acks, no matter how many
time I re-summited it.
Here's the versions I use:
storm - 0.9.1-incubating
kafka: 0.8.0.1
storm-kafka-0.8-plus 0.3
trident-redis <https://github.com/kstyrc/trident-redis>: 1.1-SNAPSHOT
Thanks!
Jie
Re: storm topology stuck at exactly 26 emits and 17 acks with trident-redis
Posted by Jie Li <ja...@gmail.com>.
Likely the issue is inside the trident-redis implementation. I tried
storm-hbase and it worked.
On Wed, Jun 4, 2014 at 1:40 PM, Jie Li <ja...@gmail.com> wrote:
> Thanks for the tip. The log is saying it's re-emitting batches.
>
> But my topology is only using one worker, so seems it's not related to
> ZMQ/Netty which are used for messages between workers?
>
>
> On Wed, Jun 4, 2014 at 12:25 PM, Danijel Schiavuzzi <
> danijel@schiavuzzi.com> wrote:
>
>> Hi,
>>
>> What do you mean by "stuck"? What do the logs say? Is the Kafka spout
>> still running and re-emitting batches?
>>
>> Could you try to revert Storm to use ZMQ transport instead of Netty (the
>> new default)? In my case, using Netty seems to block my topology
>> sometimes (like in your case, it's a Trident transactional Kafka topology
>> too), i.e. the spout is continuously re-emitting batches which never seem
>> to get ack-ed.
>>
>>
>> On Wednesday, June 4, 2014, Jie Li <ja...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> Not sure if any came across this. I have a trident topology running
>>> with TransactionalTridentKafkaSpout and MemoryMapState for 20 days, but
>>> after I switched to RedisState (tried both transactional and
>>> non-transactional), it consistenly got stuck at exactly 26 emits and 17
>>> acks, no matter how many time I re-summited it.
>>>
>>> Here's the versions I use:
>>> storm - 0.9.1-incubating
>>> kafka: 0.8.0.1
>>> storm-kafka-0.8-plus 0.3
>>> trident-redis <https://github.com/kstyrc/trident-redis>: 1.1-SNAPSHOT
>>>
>>> Thanks!
>>>
>>> Jie
>>>
>>
>>
>> --
>> Danijel Schiavuzzi
>>
>> E: danijel@schiavuzzi.com
>> W: www.schiavuzzi.com
>> T: +385989035562
>> Skype: danijels7
>>
>
>
Re: storm topology stuck at exactly 26 emits and 17 acks with trident-redis
Posted by Jie Li <ja...@gmail.com>.
Thanks for the tip. The log is saying it's re-emitting batches.
But my topology is only using one worker, so seems it's not related to
ZMQ/Netty which are used for messages between workers?
On Wed, Jun 4, 2014 at 12:25 PM, Danijel Schiavuzzi <da...@schiavuzzi.com>
wrote:
> Hi,
>
> What do you mean by "stuck"? What do the logs say? Is the Kafka spout
> still running and re-emitting batches?
>
> Could you try to revert Storm to use ZMQ transport instead of Netty (the
> new default)? In my case, using Netty seems to block my topology
> sometimes (like in your case, it's a Trident transactional Kafka topology
> too), i.e. the spout is continuously re-emitting batches which never seem
> to get ack-ed.
>
>
> On Wednesday, June 4, 2014, Jie Li <ja...@gmail.com> wrote:
>
>> Hi all,
>>
>> Not sure if any came across this. I have a trident topology running with
>> TransactionalTridentKafkaSpout and MemoryMapState for 20 days, but after I
>> switched to RedisState (tried both transactional and non-transactional), it
>> consistenly got stuck at exactly 26 emits and 17 acks, no matter how many
>> time I re-summited it.
>>
>> Here's the versions I use:
>> storm - 0.9.1-incubating
>> kafka: 0.8.0.1
>> storm-kafka-0.8-plus 0.3
>> trident-redis <https://github.com/kstyrc/trident-redis>: 1.1-SNAPSHOT
>>
>> Thanks!
>>
>> Jie
>>
>
>
> --
> Danijel Schiavuzzi
>
> E: danijel@schiavuzzi.com
> W: www.schiavuzzi.com
> T: +385989035562
> Skype: danijels7
>
Re: storm topology stuck at exactly 26 emits and 17 acks with trident-redis
Posted by Danijel Schiavuzzi <da...@schiavuzzi.com>.
Hi,
What do you mean by "stuck"? What do the logs say? Is the Kafka spout still
running and re-emitting batches?
Could you try to revert Storm to use ZMQ transport instead of Netty (the
new default)? In my case, using Netty seems to block my topology
sometimes (like in your case, it's a Trident transactional Kafka topology
too), i.e. the spout is continuously re-emitting batches which never seem
to get ack-ed.
On Wednesday, June 4, 2014, Jie Li <ja...@gmail.com> wrote:
> Hi all,
>
> Not sure if any came across this. I have a trident topology running with
> TransactionalTridentKafkaSpout and MemoryMapState for 20 days, but after I
> switched to RedisState (tried both transactional and non-transactional), it
> consistenly got stuck at exactly 26 emits and 17 acks, no matter how many
> time I re-summited it.
>
> Here's the versions I use:
> storm - 0.9.1-incubating
> kafka: 0.8.0.1
> storm-kafka-0.8-plus 0.3
> trident-redis <https://github.com/kstyrc/trident-redis>: 1.1-SNAPSHOT
>
> Thanks!
>
> Jie
>
--
Danijel Schiavuzzi
E: danijel@schiavuzzi.com
W: www.schiavuzzi.com
T: +385989035562
Skype: danijels7