You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by sunil kalva <ka...@gmail.com> on 2017/08/29 18:27:05 UTC
kafka consumer rebalance
Hi
Is there any change in consumer rebalancing algorithm from 0.8.2 and 0.10.2
versions. If yes how do i do rolling upgrades of consumer client
applications, while doing rolling upgrade few of clients might be with
older algo and few might be with newer algo then rebalancing might not be
working well. Which can cause duplicates.
Note: All the consumer client applications are reading from the same topic
with same group name.
T
SunilKalva
Re: kafka consumer rebalance
Posted by sunil kalva <ka...@gmail.com>.
thanks manikumar, this helped us a lot.
On Wed, Aug 30, 2017 at 11:33 AM, Manikumar <ma...@gmail.com>
wrote:
> In old consumer, group coordination is based on zookeeper and new consumer
> uses inbuilt
> (not depending on ZK) group coordinator. As of now, auto-migration
> of migration of old consumers
> to new consumers is not available. More details here:
> https://issues.apache.org/jira/browse/KAFKA-4513
>
> On Wed, Aug 30, 2017 at 11:05 AM, sunil kalva <ka...@gmail.com>
> wrote:
>
> > Hi
> > Can someone help me here ..!
> >
> > On Tue, Aug 29, 2017 at 11:57 PM, sunil kalva <ka...@gmail.com>
> > wrote:
> >
> > > Hi
> > > Is there any change in consumer rebalancing algorithm from 0.8.2 and
> > > 0.10.2 versions. If yes how do i do rolling upgrades of consumer client
> > > applications, while doing rolling upgrade few of clients might be with
> > > older algo and few might be with newer algo then rebalancing might not
> be
> > > working well. Which can cause duplicates.
> > >
> > > Note: All the consumer client applications are reading from the same
> > topic
> > > with same group name.
> > >
> > > T
> > > SunilKalva
> > >
> >
>
Re: kafka consumer rebalance
Posted by Manikumar <ma...@gmail.com>.
In old consumer, group coordination is based on zookeeper and new consumer
uses inbuilt
(not depending on ZK) group coordinator. As of now, auto-migration
of migration of old consumers
to new consumers is not available. More details here:
https://issues.apache.org/jira/browse/KAFKA-4513
On Wed, Aug 30, 2017 at 11:05 AM, sunil kalva <ka...@gmail.com> wrote:
> Hi
> Can someone help me here ..!
>
> On Tue, Aug 29, 2017 at 11:57 PM, sunil kalva <ka...@gmail.com>
> wrote:
>
> > Hi
> > Is there any change in consumer rebalancing algorithm from 0.8.2 and
> > 0.10.2 versions. If yes how do i do rolling upgrades of consumer client
> > applications, while doing rolling upgrade few of clients might be with
> > older algo and few might be with newer algo then rebalancing might not be
> > working well. Which can cause duplicates.
> >
> > Note: All the consumer client applications are reading from the same
> topic
> > with same group name.
> >
> > T
> > SunilKalva
> >
>
Re: kafka consumer rebalance
Posted by sunil kalva <ka...@gmail.com>.
Hi
Can someone help me here ..!
On Tue, Aug 29, 2017 at 11:57 PM, sunil kalva <ka...@gmail.com> wrote:
> Hi
> Is there any change in consumer rebalancing algorithm from 0.8.2 and
> 0.10.2 versions. If yes how do i do rolling upgrades of consumer client
> applications, while doing rolling upgrade few of clients might be with
> older algo and few might be with newer algo then rebalancing might not be
> working well. Which can cause duplicates.
>
> Note: All the consumer client applications are reading from the same topic
> with same group name.
>
> T
> SunilKalva
>