You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Wouter Bancken (JIRA)" <ji...@apache.org> on 2018/02/13 12:06:00 UTC
[jira] [Updated] (KAFKA-6557) Kafka Consumer Offsets are
unavailable during rebalancing
[ https://issues.apache.org/jira/browse/KAFKA-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wouter Bancken updated KAFKA-6557:
----------------------------------
Description:
Using the kafka-consumer-groups tool to fetch the consumer offsets, the offsets seem to be unavailable during rebalancing. During the rebalancing, the consumer group has no partition assignments.
*Scenario sketch*
# I have a consumer group 'X' subscribed to topics 'A' and 'B' with partition assignments and lag information. Consumer group 'X' is "Stable".
# a. Topic 'C' is (being) created.
b. During this creation, I do not have a partition assignment for consumer group 'X' for topics 'A' and 'B' but the consumer group is still "Stable".
# A second later: I have a partition assignment for consumer group 'X' for topics 'A' and 'B' again and the consumer group is still "Stable".
I at least expected the state of consumer group 'X' during step 2b to be "PreparingRebalance" or "AwaitingSync". Otherwise there is no way to detect that the consumer is rebalancing.
See the [discussion on the 'users' mailinglist|https://mail-archives.apache.org/mod_mbox/kafka-users/201801.mbox/%3CCAL5nZgS1im_AVVCdknLjLCFK4CkCEaqH9%3DqctgZJZKB%3Dt3hxwg%40mail.gmail.com%3E] for additional information.
was:
Using the kafka-consumer-groups tool to fetch the consumer offsets, the offsets seem to be unavailable during rebalancing. During the rebalancing, the consumer group has no partition assignments.
*Scenario sketch*
1. I have a consumer group 'X' subscribed to topics 'A' and 'B' with partition assignments and lag information. Consumer group 'X' is "Stable".
2a. Topic 'C' is (being) created.2b. During this creation, I do not have a partition assignment for consumer group 'X' for topics 'A' and 'B' but the consumer group is still "Stable".
3. A second later: I have a partition assignment for consumer group 'X' for topics 'A' and 'B' again and the consumer group is still "Stable".
I at least expected the state of consumer group 'X' during step 2b to be "PreparingRebalance" or "AwaitingSync". Otherwise there is no way to detect that the consumer is rebalancing.
See the [discussion on the 'users' mailinglist|https://mail-archives.apache.org/mod_mbox/kafka-users/201801.mbox/%3CCAL5nZgS1im_AVVCdknLjLCFK4CkCEaqH9%3DqctgZJZKB%3Dt3hxwg%40mail.gmail.com%3E] for additional information.
> Kafka Consumer Offsets are unavailable during rebalancing
> ---------------------------------------------------------
>
> Key: KAFKA-6557
> URL: https://issues.apache.org/jira/browse/KAFKA-6557
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Wouter Bancken
> Priority: Major
>
> Using the kafka-consumer-groups tool to fetch the consumer offsets, the offsets seem to be unavailable during rebalancing. During the rebalancing, the consumer group has no partition assignments.
>
> *Scenario sketch*
> # I have a consumer group 'X' subscribed to topics 'A' and 'B' with partition assignments and lag information. Consumer group 'X' is "Stable".
> # a. Topic 'C' is (being) created.
> b. During this creation, I do not have a partition assignment for consumer group 'X' for topics 'A' and 'B' but the consumer group is still "Stable".
> # A second later: I have a partition assignment for consumer group 'X' for topics 'A' and 'B' again and the consumer group is still "Stable".
>
> I at least expected the state of consumer group 'X' during step 2b to be "PreparingRebalance" or "AwaitingSync". Otherwise there is no way to detect that the consumer is rebalancing.
> See the [discussion on the 'users' mailinglist|https://mail-archives.apache.org/mod_mbox/kafka-users/201801.mbox/%3CCAL5nZgS1im_AVVCdknLjLCFK4CkCEaqH9%3DqctgZJZKB%3Dt3hxwg%40mail.gmail.com%3E] for additional information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)