You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by eb...@smartertravel.com, eb...@smartertravel.com on 2018/03/05 14:30:38 UTC

Offset auto-commit stops after timeout

In our kafka consumer logs, we're seeing the following messages:

2018-03-05 03:57:03,350 INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator  - Marking the coordinator kafka08:9092 (id: 2147483639 rack: null) dead for group mygroup
2018-03-05 03:57:03,350 WARN  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator  - Auto-commit of offsets {topic1-11=OffsetAndMetadata{offset=64888089, metadata=''}} failed for group mygroup: Offset commit failed with a retriable exception. You should retry committing offsets. The underlying error was: The request timed out.

After this message appears, the consumer no longer auto-commits offsets to the broker, even though the timeout does not repeat (and the consumer continues to consume messages from the topic partitions). 

Is this expected behavior that once the coordinator is marked dead from a single timeout, it no longer attempts to send auto-commit offsets? Shouldn't the timeout apply only to a single attempt at ConsumerCoordinator.sendOffsetCommitRequest(), and subsequent offset commits would still go through? 

We are using kafka client version 0.11.0.2 and kafka server version kafka-1.0.0.