KafkaSpout not committing to some kafka partitions and stop consuming

[Zhechao Ma <ma...@gmail.com>]

Hi

We are using KafkaSpout to read messages from kafka servers. We've
frequently come across the situation that KafkaSpout stops consuming from
some kafka partitions. We check logs and find the following log.

2017-11-07 11:35:03.170 o.a.s.k.s.KafkaSpout Thread-11-spout-executor[121
121] [DEBUG] Not polling. [10000083] uncommitted offsets across all topic
partitions has reached the threshold of [10000000]

When it reaches to the maxUncommittedOffsets (10000000), KafkaSpout stops
committing to
those partitions, and never consumes again. We also find that this issue
usually happens after partition reassignment.

Storm 1.0.2
storm-kafka-clients 1.1.0
Kafka 0.10.1.1

Could anyone give me help?

-- 
Thanks
Zhechao Ma

Re: KafkaSpout not committing to some kafka partitions and stop consuming

[Stig Rohde Døssing <sr...@apache.org>]

Hi,

There are a few bugs related to this in 1.1.0. One cause could be that the
message is right, and you have some failed tuples that haven't been
committed which are preventing the spout from progressing. You could also
be hitting either of the issues fixed in
https://github.com/apache/storm/pull/2156 and
https://github.com/apache/storm/pull/2307/files. The second fix is only
relevant if you're using topic compaction in Kafka.

You might want to try checking out and building storm-kafka-client from one
of these branches and see if that fixes your issue. You should be able to
use the upgraded storm-kafka-client with your current Storm version.

2017-11-07 4:44 GMT+01:00 Zhechao Ma <ma...@gmail.com>:

> Hi
>
> We are using KafkaSpout to read messages from kafka servers. We've
> frequently come across the situation that KafkaSpout stops consuming from
> some kafka partitions. We check logs and find the following log.
>
> 2017-11-07 11:35:03.170 o.a.s.k.s.KafkaSpout Thread-11-spout-executor[121
> 121] [DEBUG] Not polling. [10000083] uncommitted offsets across all topic
> partitions has reached the threshold of [10000000]
>
> When it reaches to the maxUncommittedOffsets (10000000), KafkaSpout stops
> committing to
> those partitions, and never consumes again. We also find that this issue
> usually happens after partition reassignment.
>
> Storm 1.0.2
> storm-kafka-clients 1.1.0
> Kafka 0.10.1.1
>
> Could anyone give me help?
>
> --
> Thanks
> Zhechao Ma
>