You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@storm.apache.org by pradeep s <sr...@gmail.com> on 2017/08/15 18:28:32 UTC

Strom Kafka Spout stuck (Storm version 1.1.0)

Hi,
I have a storm kafka spout which listens to a group of topics. Max number
of partitions in a topic is 50. There are 50 spout threads and 7 worker
nodes.
Issue is observed when there is a bigger load . All the partitions except
two or three will be processed .
When i check kafka manager , same consumer id is stuck for multiple topics
in that group.
Is this issue related to https://issues.apache.org/jira/browse/STORM-2343

Thanks
Pradeep

Re: Strom Kafka Spout stuck (Storm version 1.1.0)

Posted by Stig Rohde Døssing <sr...@apache.org>.

Yes, I believe so (Storm tries to follow semantic versioning, so the APIs
shouldn't change within minor/patch versions).

2017-08-15 21:12 GMT+02:00 pradeep s <sr...@gmail.com>:

> Thanks Stig for the quick response .Current dependency is
>
>
> <artifactId>storm-kafka-client</artifactId>
>
> <version>1.1.0</version>
>
> Can i upgrade this to 1.1.1 without storm cluster version change
>
> On Tue, Aug 15, 2017 at 11:58 AM, Stig Rohde Døssing <sr...@apache.org>
> wrote:
>
>> Hi,
>>
>> It's very likely. Try upgrading your storm-kafka-client dependency and
>> see if the issue goes away. If not we can look at debugging it.
>>
>>
>> 2017-08-15 20:28 GMT+02:00 pradeep s <sr...@gmail.com>:
>>
>>> Hi,
>>> I have a storm kafka spout which listens to a group of topics. Max
>>> number of partitions in a topic is 50. There are 50 spout threads and 7
>>> worker nodes.
>>> Issue is observed when there is a bigger load . All the partitions
>>> except two or three will be processed .
>>> When i check kafka manager , same consumer id is stuck for multiple
>>> topics in that group.
>>> Is this issue related to https://issues.apache.org/j
>>> ira/browse/STORM-2343
>>>
>>> Thanks
>>> Pradeep
>>>
>>
>>
>

Re: Strom Kafka Spout stuck (Storm version 1.1.0)

Posted by pradeep s <sr...@gmail.com>.

Thanks Stig for the quick response .Current dependency is


<artifactId>storm-kafka-client</artifactId>

<version>1.1.0</version>

Can i upgrade this to 1.1.1 without storm cluster version change

On Tue, Aug 15, 2017 at 11:58 AM, Stig Rohde Døssing <sr...@apache.org>
wrote:

> Hi,
>
> It's very likely. Try upgrading your storm-kafka-client dependency and see
> if the issue goes away. If not we can look at debugging it.
>
>
> 2017-08-15 20:28 GMT+02:00 pradeep s <sr...@gmail.com>:
>
>> Hi,
>> I have a storm kafka spout which listens to a group of topics. Max number
>> of partitions in a topic is 50. There are 50 spout threads and 7 worker
>> nodes.
>> Issue is observed when there is a bigger load . All the partitions except
>> two or three will be processed .
>> When i check kafka manager , same consumer id is stuck for multiple
>> topics in that group.
>> Is this issue related to https://issues.apache.org/jira/browse/STORM-2343
>>
>> Thanks
>> Pradeep
>>
>
>

Re: Strom Kafka Spout stuck (Storm version 1.1.0)

Posted by Stig Rohde Døssing <sr...@apache.org>.

Hi,

It's very likely. Try upgrading your storm-kafka-client dependency and see
if the issue goes away. If not we can look at debugging it.


2017-08-15 20:28 GMT+02:00 pradeep s <sr...@gmail.com>:

> Hi,
> I have a storm kafka spout which listens to a group of topics. Max number
> of partitions in a topic is 50. There are 50 spout threads and 7 worker
> nodes.
> Issue is observed when there is a bigger load . All the partitions except
> two or three will be processed .
> When i check kafka manager , same consumer id is stuck for multiple topics
> in that group.
> Is this issue related to https://issues.apache.org/jira/browse/STORM-2343
>
> Thanks
> Pradeep
>