You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "David Jacot (Jira)" <ji...@apache.org> on 2019/10/18 07:58:00 UTC
[jira] [Updated] (KAFKA-9065) Loading offsets and group metadata
loops forever
[ https://issues.apache.org/jira/browse/KAFKA-9065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Jacot updated KAFKA-9065:
-------------------------------
Description:
When the metadata manager loads the groups and the offsets of a partition of the __consumer-offsets topic, `GroupMetadataManager.doLoadGroupsAndOffsets` could loop forever if the start offset of the partition is smaller than the end offset and no records are effectively read from the partition.
While the conditions leading to this issue are not clear, I've got the case where a partition was having two segments which were both empty in a cluster. This could theoretically happen when all the tombstones in the first are expired and the second is truncated.
As a side effect, the `doLoadGroupsAndOffsets` spins forever, blocks the single thread of the scheduler, blocks the loading of all the groups and offsets which are after in the queue, and blocks the expiration of the offsets.
was:
When the metadata manager loads the groups and the offsets of a partition of the __consumer-offsets topic, `GroupMetadataManager.doLoadGroupsAndOffsets` could loop forever if the start offset of the partition is smaller than the end offset and no records are effectively read from the partition.
While the conditions leading to this issue are not clear, I've got the case where a partition was having two segments which were both empty in a cluster. This could theoretically happen when all the tombstones in the first are expired and the second is truncated.
As a side effect, the `doLoadGroupsAndOffsets` spins forever, blocks the single thread of the scheduler, blocks the loading of all the groups and offsets which are after in the queue, and blocks the expiration of the offsets.
> Loading offsets and group metadata loops forever
> ------------------------------------------------
>
> Key: KAFKA-9065
> URL: https://issues.apache.org/jira/browse/KAFKA-9065
> Project: Kafka
> Issue Type: Bug
> Reporter: David Jacot
> Priority: Major
>
> When the metadata manager loads the groups and the offsets of a partition of the __consumer-offsets topic, `GroupMetadataManager.doLoadGroupsAndOffsets` could loop forever if the start offset of the partition is smaller than the end offset and no records are effectively read from the partition.
> While the conditions leading to this issue are not clear, I've got the case where a partition was having two segments which were both empty in a cluster. This could theoretically happen when all the tombstones in the first are expired and the second is truncated.
> As a side effect, the `doLoadGroupsAndOffsets` spins forever, blocks the single thread of the scheduler, blocks the loading of all the groups and offsets which are after in the queue, and blocks the expiration of the offsets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)