Re: Kafka Consumers not rebalancing.

[satyajit vegesna <sa...@gmail.com>]

Any help would be appreciated!

Hi All,

I was experimenting on the new consumer API and have a question regarding
the rebalance process.

I start a consumer group with single thread and make the Thread sleep while
processing the records retrieved from the first consumer.poll call, i was
making sure the Thread.sleep time goes beyond the session timeout and was
expecting to see a rebalance on the consumer group.

But when i try to get the state of the consumer group using the below
command,

/opt/confluent-4.0.0/bin/kafka-consumer-groups  --group
"consumer-grievances-group15" --bootstrap-server xxxxxxx:9092,xxxxxxxx:9092,
xxxxxxxx:9092  --describe

i get below result ,

Consumer group 'consumer-grievances-group15' has no active members.


TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET
LAG        CONSUMER-ID                                       HOST
                CLIENT-ID

TELMATEQA.grievances.grievances 0          125855          152037
26182      -                                                 -
                -

The same happens with multiple thread in the consumer group scenario,
and *going
further one step i was able to make the thread get into running state, from
sleep state and could see that the consumer group started off from where it
left.*

My only question is , why isn't the rebalancing happening in this scenario.
My expectation was that the threads rebalance and start from the committed
offset.

Regards,
Satyajit.


On Mon, Jan 29, 2018 at 3:36 PM, satyajit vegesna <sa...@gmail.com>
wrote:

> Hi All,
>
> I was experimenting on the new consumer API and have a question regarding
> the rebalance process.
>
> I start a consumer group with single thread and make the Thread sleep
> while processing the records retrieved from the first consumer.poll call, i
> was making sure the Thread.sleep time goes beyond the session timeout and
> was expecting to see a rebalance on the consumer group.
>
> But when i try to get the state of the consumer group using the below
> command,
>
> /opt/confluent-4.0.0/bin/kafka-consumer-groups  --group
> "consumer-grievances-group15" --bootstrap-server xxxxxxx:9092,xxxxxxxx:9092,
> xxxxxxxx:9092  --describe
>
> i get below result ,
>
> Consumer group 'consumer-grievances-group15' has no active members.
>
>
> TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET
> LAG        CONSUMER-ID                                       HOST
>                   CLIENT-ID
>
> TELMATEQA.grievances.grievances 0          125855          152037
>   26182      -                                                 -
>                     -
>
> The same happens with multiple thread in the consumer group scenario, and *going
> further one step i was able to make the thread get into running state, from
> sleep state and could see that the consumer group started off from where it
> left.*
>
> My only question is , why isn't the rebalancing happening in this
> scenario. My expectation was that the threads rebalance and start from the
> committed offset.
>
> Regards,
> Satyajit.
>
>