You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by "Bateman, Matt" <ma...@ebay.com> on 2012/06/16 02:08:44 UTC

KAFKA-264

Hi,

How likely is the scenario where a rebalance might leave some partitions without a consumer? Described here https://cwiki.apache.org/confluence/display/KAFKA/Central+Consumer+Coordination.

It looks like KAFKA-264 would address this?

Thanks,

Matt

Re: KAFKA-264

Posted by Neha Narkhede <ne...@gmail.com>.

Matt,

If a consumer rebalances successfully (does not run out of rebalancing
attempts) and there is no bug in the rebalancing logic, a partition
should always have exactly one consumer/group. Currently, the only way
of verifying this is by running the VerifyConsumerRebalance tool in
kafka.tools package.

KAFKA-264 would improve this since a consumer would always rebalance
successfully and the co-ordinator will ensure that each partition has
exactly one consumer/group.

Thanks,
Neha

On Wed, Jun 20, 2012 at 3:16 PM, Bateman, Matt <ma...@ebay.com> wrote:
> Okay, let me ask this another way: How can I detect if a partition has no consumer? Is there any way to tell outside of manually checking data post-consumer?
>
> Thanks,
>
> Matt
>
> -----Original Message-----
> From: Bateman, Matt [mailto:mabateman@ebay.com]
> Sent: Friday, June 15, 2012 5:09 PM
> To: kafka-users@incubator.apache.org
> Subject: KAFKA-264
>
> Hi,
>
> How likely is the scenario where a rebalance might leave some partitions without a consumer? Described here https://cwiki.apache.org/confluence/display/KAFKA/Central+Consumer+Coordination.
>
> It looks like KAFKA-264 would address this?
>
> Thanks,
>
> Matt

RE: KAFKA-264

Posted by "Bateman, Matt" <ma...@ebay.com>.

Okay, let me ask this another way: How can I detect if a partition has no consumer? Is there any way to tell outside of manually checking data post-consumer?

Thanks,

Matt

-----Original Message-----
From: Bateman, Matt [mailto:mabateman@ebay.com] 
Sent: Friday, June 15, 2012 5:09 PM
To: kafka-users@incubator.apache.org
Subject: KAFKA-264

Hi,

How likely is the scenario where a rebalance might leave some partitions without a consumer? Described here https://cwiki.apache.org/confluence/display/KAFKA/Central+Consumer+Coordination.

It looks like KAFKA-264 would address this?

Thanks,

Matt