You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Paramesh Nc <pa...@gmail.com> on 2016/07/12 17:35:24 UTC
Error ILLEGAL_GENERATION occurred while committing offsets for group
Hi ,
I am getting the below error messages when consuming the records from the
kafka topic .
o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION
occurred while committing offsets for group test-group.
2016-07-11 22:56:50.314 ERROR 8699 --- [ Thread]
o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION
occurred while committing offsets for group test-group
2016-07-11 22:56:50.315 WARN 8699 --- [Thread]
o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed:
Commit cannot be completed due to group rebalance
2016-07-11 22:56:50.320 ERROR 8699 --- [ Thread]
o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION
occurred while committing offsets for group test-group
2016-07-11 22:56:50.321 WARN 8699 --- [- Thread]
o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed:
2016-07-11 22:56:50.323 INFO 8699 --- [Thread]
o.a.k.c.c.internals.AbstractCoordinator : Attempt to join group test-group
failed due to unknown member id, resetting and retrying.
And I am creating the consumer as follows :
KafkaConsumer<byte[],byte[]> consumer=new KafkaConsumer<>(props);
consumer.subscribe(Arrays.asList(topic).
Where props is set with the properties :
props.put("security.protocol", "SASL_PLAINTEXT");
props.put("group.id", groupId);
props.put("bootstrap.servers", "pqrs:9092,xyz:9092");
And kafka client I am using is 0.9.0.1.
Could you please help me to resolve this issue. I have gone through so many
links available over the internet but did not find the useful answers.
Thanks in Advance.
Param.
Re: Error ILLEGAL_GENERATION occurred while committing offsets for group
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
You're probably spending too long processing some messages. The
ILLEGAL_GENERATION message indicates that a new generation was created,
which probably means you didn't call poll() frequently enough to send
heartbeats indicating you are still an active member of the group.
We're currently working on implementing KIP-62 (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread)
to support background heartbeats during some timeout period so you can
spend more than the session timeout processing data without causing you to
fall out of the group. This helps decouple the amount of time spent
processing messages from the timeout for tracking when a consumer has died
and won't return.
In the mean time, if you can, you might want to set max.poll.records to a
small value to avoid hitting these timeouts.
-Ewen
On Tue, Jul 12, 2016 at 10:35 AM, Paramesh Nc <pa...@gmail.com>
wrote:
> Hi ,
>
> I am getting the below error messages when consuming the records from the
> kafka topic .
>
> o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION
> occurred while committing offsets for group test-group.
>
> 2016-07-11 22:56:50.314 ERROR 8699 --- [ Thread]
> o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION
> occurred while committing offsets for group test-group
> 2016-07-11 22:56:50.315 WARN 8699 --- [Thread]
> o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed:
> Commit cannot be completed due to group rebalance
> 2016-07-11 22:56:50.320 ERROR 8699 --- [ Thread]
> o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION
> occurred while committing offsets for group test-group
> 2016-07-11 22:56:50.321 WARN 8699 --- [- Thread]
> o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed:
> 2016-07-11 22:56:50.323 INFO 8699 --- [Thread]
> o.a.k.c.c.internals.AbstractCoordinator : Attempt to join group test-group
> failed due to unknown member id, resetting and retrying.
>
> And I am creating the consumer as follows :
> KafkaConsumer<byte[],byte[]> consumer=new KafkaConsumer<>(props);
> consumer.subscribe(Arrays.asList(topic).
> Where props is set with the properties :
> props.put("security.protocol", "SASL_PLAINTEXT");
> props.put("group.id", groupId);
> props.put("bootstrap.servers", "pqrs:9092,xyz:9092");
> And kafka client I am using is 0.9.0.1.
>
>
>
> Could you please help me to resolve this issue. I have gone through so many
> links available over the internet but did not find the useful answers.
>
> Thanks in Advance.
> Param.
>
--
Thanks,
Ewen