You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Kane Kane <ka...@gmail.com> on 2013/10/01 09:20:31 UTC

as i understand rebalance happens on client side

But it looks like some clients don't implement it?

Re: as i understand rebalance happens on client side

Posted by Keith Bourgoin <ke...@gmail.com>.
To be honest, 0.8 it's a pretty big change from how things are currently
managed in samsa. I don't have much of an ETA at the moment, but I'm hoping
to find the time in the next month or two.

Keith.


On Thu, Oct 3, 2013 at 11:59 AM, Kane Kane <ka...@gmail.com> wrote:

> Hi Keith, thanks for update! Interestingly i've found your library
> yesterday and was going to borrow your zookeeper code for partition
> management. Do you have any ETA for the 0.8 support?
>
> Thanks!
>
>
> On Thu, Oct 3, 2013 at 8:09 AM, Keith Bourgoin <keith.bourgoin@gmail.com
> >wrote:
>
> > Hi Kane,
> >
> > I just wanted to chime in as well. I currently maintain samsa (
> > https://github.com/getsamsa/samsa or PyPI), which does implement the
> > rebalancing logic, but currently doesn't support 0.8. If you're on 0.7.x
> > still, it might be helpful. If not, we're working on 0.8 support, but it
> > probably won't be ready for a bit.
> >
> > Cheers,
> >
> > Keith.
> >
> >
> > On Tue, Oct 1, 2013 at 1:39 PM, Kane Kane <ka...@gmail.com> wrote:
> >
> > > Thanks for reply, David, your library is great and indeed the
> rebalancing
> > > is currently somewhat quirky and complicated. And I guess it doesn't
> make
> > > sense to implement it considering 0.9 is planned relatively soon.
> > >
> > >
> > > On Tue, Oct 1, 2013 at 10:09 AM, David Arthur <mu...@gmail.com>
> wrote:
> > >
> > > > Kane,
> > > >
> > > > I'm the creator of kafka-python, just thought I'd give some insight.
> > > >
> > > > Consumer rebalancing is actually pretty tricky to get right. It
> > requires
> > > > interaction with ZooKeeper which (though possible via kazoo) is
> > something
> > > > I've tried to avoid in kafka-python. It also seems a little strange
> to
> > me
> > > > to mix your consumers between Java/Scala and Python. If you really
> need
> > > > rebalancing between Python consumers, you'd have to implement that on
> > top
> > > > of kafka-python.
> > > >
> > > > Once the coordinator API is finalized for 0.9, I (or someone) will
> work
> > > on
> > > > implementing it in kafka-python
> > > >
> > > > Cheers
> > > > -David
> > > >
> > > >
> > > > On 10/1/13 11:56 AM, Kane Kane wrote:
> > > >
> > > >> The reason i was asking is that this library seems to have support
> > only
> > > >> for
> > > >> SimpleConsumer https://github.com/mumrah/**kafka-python/<
> > > https://github.com/mumrah/kafka-python/>,
> > > >> i was curious if
> > > >> all should be implemented on client or kafka has some rebalancing
> > logic
> > > >> and
> > > >> prevent consuming from the same queue on server side in case of
> > > >> SimpleConsumer api, but I see now that everything should implemented
> > on
> > > >> the
> > > >> client side.
> > > >>
> > > >> Thanks.
> > > >>
> > > >>
> > > >> On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com>
> > > wrote:
> > > >>
> > > >>  I do not understand your question, what are you trying to
> implement?
> > > >>>
> > > >>>
> > > >>> On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com>
> > > wrote:
> > > >>>
> > > >>>  So essentially you can't do "queue" pattern, unless you somehow
> > > >>>> implement
> > > >>>> locking on the client?
> > > >>>>
> > > >>>>
> > > >>>> On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wangguoz@gmail.com
> >
> > > >>>>
> > > >>> wrote:
> > > >>>
> > > >>>> SimpleConsumer do not have any concept of group management, only
> the
> > > >>>>> high-level consumers have. So multiple simple consumers can
> > > >>>>>
> > > >>>> independently
> > > >>>
> > > >>>> consume from the same partition(s).
> > > >>>>>
> > > >>>>> Guozhang
> > > >>>>>
> > > >>>>>
> > > >>>>> On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <kane.isturm@gmail.com
> >
> > > >>>>>
> > > >>>> wrote:
> > > >>>
> > > >>>> Yeah, I noticed that, i'm curious how balancing happens if
> > > >>>>>>
> > > >>>>> SimpleConsumer
> > > >>>>
> > > >>>>> is used. I.e. i can provide a partition to read from if i use
> > > >>>>>> SimpleConsumer, but what if someone else already attached to
> that
> > > >>>>>> partition, what would happen? Also what would happen if one
> > > >>>>>>
> > > >>>>> SimpleConsumer
> > > >>>>>
> > > >>>>>> attached to all partitions? Noone would be able to join?
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> > > >>>>>>
> > > >>>>> neha.narkhede@gmail.com
> > > >>>
> > > >>>>  wrote:
> > > >>>>>>> There are 2 types of consumer clients in Kafka -
> > > >>>>>>>
> > > >>>>>> ZookeeperConsumerConnector
> > > >>>>>>
> > > >>>>>>> and SimpleConsumer. Only the former has the re balancing logic.
> > > >>>>>>>
> > > >>>>>>> Thanks,
> > > >>>>>>> Neha
> > > >>>>>>> On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com>
> > wrote:
> > > >>>>>>>
> > > >>>>>>>  But it looks like some clients don't implement it?
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> -- Guozhang
> > > >>>>>
> > > >>>>>
> > > >>>
> > > >>> --
> > > >>> -- Guozhang
> > > >>>
> > > >>>
> > > >
> > >
> >
>

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
Hi Keith, thanks for update! Interestingly i've found your library
yesterday and was going to borrow your zookeeper code for partition
management. Do you have any ETA for the 0.8 support?

Thanks!


On Thu, Oct 3, 2013 at 8:09 AM, Keith Bourgoin <ke...@gmail.com>wrote:

> Hi Kane,
>
> I just wanted to chime in as well. I currently maintain samsa (
> https://github.com/getsamsa/samsa or PyPI), which does implement the
> rebalancing logic, but currently doesn't support 0.8. If you're on 0.7.x
> still, it might be helpful. If not, we're working on 0.8 support, but it
> probably won't be ready for a bit.
>
> Cheers,
>
> Keith.
>
>
> On Tue, Oct 1, 2013 at 1:39 PM, Kane Kane <ka...@gmail.com> wrote:
>
> > Thanks for reply, David, your library is great and indeed the rebalancing
> > is currently somewhat quirky and complicated. And I guess it doesn't make
> > sense to implement it considering 0.9 is planned relatively soon.
> >
> >
> > On Tue, Oct 1, 2013 at 10:09 AM, David Arthur <mu...@gmail.com> wrote:
> >
> > > Kane,
> > >
> > > I'm the creator of kafka-python, just thought I'd give some insight.
> > >
> > > Consumer rebalancing is actually pretty tricky to get right. It
> requires
> > > interaction with ZooKeeper which (though possible via kazoo) is
> something
> > > I've tried to avoid in kafka-python. It also seems a little strange to
> me
> > > to mix your consumers between Java/Scala and Python. If you really need
> > > rebalancing between Python consumers, you'd have to implement that on
> top
> > > of kafka-python.
> > >
> > > Once the coordinator API is finalized for 0.9, I (or someone) will work
> > on
> > > implementing it in kafka-python
> > >
> > > Cheers
> > > -David
> > >
> > >
> > > On 10/1/13 11:56 AM, Kane Kane wrote:
> > >
> > >> The reason i was asking is that this library seems to have support
> only
> > >> for
> > >> SimpleConsumer https://github.com/mumrah/**kafka-python/<
> > https://github.com/mumrah/kafka-python/>,
> > >> i was curious if
> > >> all should be implemented on client or kafka has some rebalancing
> logic
> > >> and
> > >> prevent consuming from the same queue on server side in case of
> > >> SimpleConsumer api, but I see now that everything should implemented
> on
> > >> the
> > >> client side.
> > >>
> > >> Thanks.
> > >>
> > >>
> > >> On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com>
> > wrote:
> > >>
> > >>  I do not understand your question, what are you trying to implement?
> > >>>
> > >>>
> > >>> On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com>
> > wrote:
> > >>>
> > >>>  So essentially you can't do "queue" pattern, unless you somehow
> > >>>> implement
> > >>>> locking on the client?
> > >>>>
> > >>>>
> > >>>> On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> > >>>>
> > >>> wrote:
> > >>>
> > >>>> SimpleConsumer do not have any concept of group management, only the
> > >>>>> high-level consumers have. So multiple simple consumers can
> > >>>>>
> > >>>> independently
> > >>>
> > >>>> consume from the same partition(s).
> > >>>>>
> > >>>>> Guozhang
> > >>>>>
> > >>>>>
> > >>>>> On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
> > >>>>>
> > >>>> wrote:
> > >>>
> > >>>> Yeah, I noticed that, i'm curious how balancing happens if
> > >>>>>>
> > >>>>> SimpleConsumer
> > >>>>
> > >>>>> is used. I.e. i can provide a partition to read from if i use
> > >>>>>> SimpleConsumer, but what if someone else already attached to that
> > >>>>>> partition, what would happen? Also what would happen if one
> > >>>>>>
> > >>>>> SimpleConsumer
> > >>>>>
> > >>>>>> attached to all partitions? Noone would be able to join?
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> > >>>>>>
> > >>>>> neha.narkhede@gmail.com
> > >>>
> > >>>>  wrote:
> > >>>>>>> There are 2 types of consumer clients in Kafka -
> > >>>>>>>
> > >>>>>> ZookeeperConsumerConnector
> > >>>>>>
> > >>>>>>> and SimpleConsumer. Only the former has the re balancing logic.
> > >>>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Neha
> > >>>>>>> On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com>
> wrote:
> > >>>>>>>
> > >>>>>>>  But it looks like some clients don't implement it?
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>
> > >>>>> --
> > >>>>> -- Guozhang
> > >>>>>
> > >>>>>
> > >>>
> > >>> --
> > >>> -- Guozhang
> > >>>
> > >>>
> > >
> >
>

Re: as i understand rebalance happens on client side

Posted by Keith Bourgoin <ke...@gmail.com>.
Hi Kane,

I just wanted to chime in as well. I currently maintain samsa (
https://github.com/getsamsa/samsa or PyPI), which does implement the
rebalancing logic, but currently doesn't support 0.8. If you're on 0.7.x
still, it might be helpful. If not, we're working on 0.8 support, but it
probably won't be ready for a bit.

Cheers,

Keith.


On Tue, Oct 1, 2013 at 1:39 PM, Kane Kane <ka...@gmail.com> wrote:

> Thanks for reply, David, your library is great and indeed the rebalancing
> is currently somewhat quirky and complicated. And I guess it doesn't make
> sense to implement it considering 0.9 is planned relatively soon.
>
>
> On Tue, Oct 1, 2013 at 10:09 AM, David Arthur <mu...@gmail.com> wrote:
>
> > Kane,
> >
> > I'm the creator of kafka-python, just thought I'd give some insight.
> >
> > Consumer rebalancing is actually pretty tricky to get right. It requires
> > interaction with ZooKeeper which (though possible via kazoo) is something
> > I've tried to avoid in kafka-python. It also seems a little strange to me
> > to mix your consumers between Java/Scala and Python. If you really need
> > rebalancing between Python consumers, you'd have to implement that on top
> > of kafka-python.
> >
> > Once the coordinator API is finalized for 0.9, I (or someone) will work
> on
> > implementing it in kafka-python
> >
> > Cheers
> > -David
> >
> >
> > On 10/1/13 11:56 AM, Kane Kane wrote:
> >
> >> The reason i was asking is that this library seems to have support only
> >> for
> >> SimpleConsumer https://github.com/mumrah/**kafka-python/<
> https://github.com/mumrah/kafka-python/>,
> >> i was curious if
> >> all should be implemented on client or kafka has some rebalancing logic
> >> and
> >> prevent consuming from the same queue on server side in case of
> >> SimpleConsumer api, but I see now that everything should implemented on
> >> the
> >> client side.
> >>
> >> Thanks.
> >>
> >>
> >> On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >>
> >>  I do not understand your question, what are you trying to implement?
> >>>
> >>>
> >>> On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com>
> wrote:
> >>>
> >>>  So essentially you can't do "queue" pattern, unless you somehow
> >>>> implement
> >>>> locking on the client?
> >>>>
> >>>>
> >>>> On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> >>>>
> >>> wrote:
> >>>
> >>>> SimpleConsumer do not have any concept of group management, only the
> >>>>> high-level consumers have. So multiple simple consumers can
> >>>>>
> >>>> independently
> >>>
> >>>> consume from the same partition(s).
> >>>>>
> >>>>> Guozhang
> >>>>>
> >>>>>
> >>>>> On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
> >>>>>
> >>>> wrote:
> >>>
> >>>> Yeah, I noticed that, i'm curious how balancing happens if
> >>>>>>
> >>>>> SimpleConsumer
> >>>>
> >>>>> is used. I.e. i can provide a partition to read from if i use
> >>>>>> SimpleConsumer, but what if someone else already attached to that
> >>>>>> partition, what would happen? Also what would happen if one
> >>>>>>
> >>>>> SimpleConsumer
> >>>>>
> >>>>>> attached to all partitions? Noone would be able to join?
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> >>>>>>
> >>>>> neha.narkhede@gmail.com
> >>>
> >>>>  wrote:
> >>>>>>> There are 2 types of consumer clients in Kafka -
> >>>>>>>
> >>>>>> ZookeeperConsumerConnector
> >>>>>>
> >>>>>>> and SimpleConsumer. Only the former has the re balancing logic.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Neha
> >>>>>>> On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
> >>>>>>>
> >>>>>>>  But it looks like some clients don't implement it?
> >>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>> --
> >>>>> -- Guozhang
> >>>>>
> >>>>>
> >>>
> >>> --
> >>> -- Guozhang
> >>>
> >>>
> >
>

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
Thanks for reply, David, your library is great and indeed the rebalancing
is currently somewhat quirky and complicated. And I guess it doesn't make
sense to implement it considering 0.9 is planned relatively soon.


On Tue, Oct 1, 2013 at 10:09 AM, David Arthur <mu...@gmail.com> wrote:

> Kane,
>
> I'm the creator of kafka-python, just thought I'd give some insight.
>
> Consumer rebalancing is actually pretty tricky to get right. It requires
> interaction with ZooKeeper which (though possible via kazoo) is something
> I've tried to avoid in kafka-python. It also seems a little strange to me
> to mix your consumers between Java/Scala and Python. If you really need
> rebalancing between Python consumers, you'd have to implement that on top
> of kafka-python.
>
> Once the coordinator API is finalized for 0.9, I (or someone) will work on
> implementing it in kafka-python
>
> Cheers
> -David
>
>
> On 10/1/13 11:56 AM, Kane Kane wrote:
>
>> The reason i was asking is that this library seems to have support only
>> for
>> SimpleConsumer https://github.com/mumrah/**kafka-python/<https://github.com/mumrah/kafka-python/>,
>> i was curious if
>> all should be implemented on client or kafka has some rebalancing logic
>> and
>> prevent consuming from the same queue on server side in case of
>> SimpleConsumer api, but I see now that everything should implemented on
>> the
>> client side.
>>
>> Thanks.
>>
>>
>> On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com> wrote:
>>
>>  I do not understand your question, what are you trying to implement?
>>>
>>>
>>> On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com> wrote:
>>>
>>>  So essentially you can't do "queue" pattern, unless you somehow
>>>> implement
>>>> locking on the client?
>>>>
>>>>
>>>> On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
>>>>
>>> wrote:
>>>
>>>> SimpleConsumer do not have any concept of group management, only the
>>>>> high-level consumers have. So multiple simple consumers can
>>>>>
>>>> independently
>>>
>>>> consume from the same partition(s).
>>>>>
>>>>> Guozhang
>>>>>
>>>>>
>>>>> On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
>>>>>
>>>> wrote:
>>>
>>>> Yeah, I noticed that, i'm curious how balancing happens if
>>>>>>
>>>>> SimpleConsumer
>>>>
>>>>> is used. I.e. i can provide a partition to read from if i use
>>>>>> SimpleConsumer, but what if someone else already attached to that
>>>>>> partition, what would happen? Also what would happen if one
>>>>>>
>>>>> SimpleConsumer
>>>>>
>>>>>> attached to all partitions? Noone would be able to join?
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
>>>>>>
>>>>> neha.narkhede@gmail.com
>>>
>>>>  wrote:
>>>>>>> There are 2 types of consumer clients in Kafka -
>>>>>>>
>>>>>> ZookeeperConsumerConnector
>>>>>>
>>>>>>> and SimpleConsumer. Only the former has the re balancing logic.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Neha
>>>>>>> On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
>>>>>>>
>>>>>>>  But it looks like some clients don't implement it?
>>>>>>>>
>>>>>>>>
>>>>>
>>>>> --
>>>>> -- Guozhang
>>>>>
>>>>>
>>>
>>> --
>>> -- Guozhang
>>>
>>>
>

Re: as i understand rebalance happens on client side

Posted by David Arthur <mu...@gmail.com>.
Kane,

I'm the creator of kafka-python, just thought I'd give some insight.

Consumer rebalancing is actually pretty tricky to get right. It requires 
interaction with ZooKeeper which (though possible via kazoo) is 
something I've tried to avoid in kafka-python. It also seems a little 
strange to me to mix your consumers between Java/Scala and Python. If 
you really need rebalancing between Python consumers, you'd have to 
implement that on top of kafka-python.

Once the coordinator API is finalized for 0.9, I (or someone) will work 
on implementing it in kafka-python

Cheers
-David

On 10/1/13 11:56 AM, Kane Kane wrote:
> The reason i was asking is that this library seems to have support only for
> SimpleConsumer https://github.com/mumrah/kafka-python/, i was curious if
> all should be implemented on client or kafka has some rebalancing logic and
> prevent consuming from the same queue on server side in case of
> SimpleConsumer api, but I see now that everything should implemented on the
> client side.
>
> Thanks.
>
>
> On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
>> I do not understand your question, what are you trying to implement?
>>
>>
>> On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com> wrote:
>>
>>> So essentially you can't do "queue" pattern, unless you somehow implement
>>> locking on the client?
>>>
>>>
>>> On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
>> wrote:
>>>> SimpleConsumer do not have any concept of group management, only the
>>>> high-level consumers have. So multiple simple consumers can
>> independently
>>>> consume from the same partition(s).
>>>>
>>>> Guozhang
>>>>
>>>>
>>>> On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
>> wrote:
>>>>> Yeah, I noticed that, i'm curious how balancing happens if
>>> SimpleConsumer
>>>>> is used. I.e. i can provide a partition to read from if i use
>>>>> SimpleConsumer, but what if someone else already attached to that
>>>>> partition, what would happen? Also what would happen if one
>>>> SimpleConsumer
>>>>> attached to all partitions? Noone would be able to join?
>>>>>
>>>>>
>>>>> On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
>> neha.narkhede@gmail.com
>>>>>> wrote:
>>>>>> There are 2 types of consumer clients in Kafka -
>>>>> ZookeeperConsumerConnector
>>>>>> and SimpleConsumer. Only the former has the re balancing logic.
>>>>>>
>>>>>> Thanks,
>>>>>> Neha
>>>>>> On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
>>>>>>
>>>>>>> But it looks like some clients don't implement it?
>>>>>>>
>>>>
>>>>
>>>> --
>>>> -- Guozhang
>>>>
>>
>>
>> --
>> -- Guozhang
>>


Re: as i understand rebalance happens on client side

Posted by Neha Narkhede <ne...@gmail.com>.
Currently we are preparing for the 0.8-final release. But 0.9 is not slated
to release until early next year.

Thanks,
Neha


On Tue, Oct 1, 2013 at 9:26 AM, Kane Kane <ka...@gmail.com> wrote:

> Btw, is it expected to be released on Oct 31?
>
> Thanks!
>
>
> On Tue, Oct 1, 2013 at 9:01 AM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > We do plan to move the group membership over to the server side and have
> a
> > very thin consumer client. The proposal is here -
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPIand
> > this is being planned for the 0.9 release. Once this is complete, the
> > non-java clients can easily leverage the group management feature.
> >
> > Thanks,
> > Neha
> >
> >
> > On Tue, Oct 1, 2013 at 8:56 AM, Kane Kane <ka...@gmail.com> wrote:
> >
> > > The reason i was asking is that this library seems to have support only
> > for
> > > SimpleConsumer https://github.com/mumrah/kafka-python/, i was curious
> if
> > > all should be implemented on client or kafka has some rebalancing logic
> > and
> > > prevent consuming from the same queue on server side in case of
> > > SimpleConsumer api, but I see now that everything should implemented on
> > the
> > > client side.
> > >
> > > Thanks.
> > >
> > >
> > > On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com>
> > wrote:
> > >
> > > > I do not understand your question, what are you trying to implement?
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com>
> > wrote:
> > > >
> > > > > So essentially you can't do "queue" pattern, unless you somehow
> > > implement
> > > > > locking on the client?
> > > > >
> > > > >
> > > > > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > SimpleConsumer do not have any concept of group management, only
> > the
> > > > > > high-level consumers have. So multiple simple consumers can
> > > > independently
> > > > > > consume from the same partition(s).
> > > > > >
> > > > > > Guozhang
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <kane.isturm@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > Yeah, I noticed that, i'm curious how balancing happens if
> > > > > SimpleConsumer
> > > > > > > is used. I.e. i can provide a partition to read from if i use
> > > > > > > SimpleConsumer, but what if someone else already attached to
> that
> > > > > > > partition, what would happen? Also what would happen if one
> > > > > > SimpleConsumer
> > > > > > > attached to all partitions? Noone would be able to join?
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> > > > neha.narkhede@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > There are 2 types of consumer clients in Kafka -
> > > > > > > ZookeeperConsumerConnector
> > > > > > > > and SimpleConsumer. Only the former has the re balancing
> logic.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Neha
> > > > > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com>
> > > wrote:
> > > > > > > >
> > > > > > > > > But it looks like some clients don't implement it?
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > -- Guozhang
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > > >
> > >
> >
>

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
Btw, is it expected to be released on Oct 31?

Thanks!


On Tue, Oct 1, 2013 at 9:01 AM, Neha Narkhede <ne...@gmail.com>wrote:

> We do plan to move the group membership over to the server side and have a
> very thin consumer client. The proposal is here -
>
> https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPIand
> this is being planned for the 0.9 release. Once this is complete, the
> non-java clients can easily leverage the group management feature.
>
> Thanks,
> Neha
>
>
> On Tue, Oct 1, 2013 at 8:56 AM, Kane Kane <ka...@gmail.com> wrote:
>
> > The reason i was asking is that this library seems to have support only
> for
> > SimpleConsumer https://github.com/mumrah/kafka-python/, i was curious if
> > all should be implemented on client or kafka has some rebalancing logic
> and
> > prevent consuming from the same queue on server side in case of
> > SimpleConsumer api, but I see now that everything should implemented on
> the
> > client side.
> >
> > Thanks.
> >
> >
> > On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> > > I do not understand your question, what are you trying to implement?
> > >
> > >
> > > On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com>
> wrote:
> > >
> > > > So essentially you can't do "queue" pattern, unless you somehow
> > implement
> > > > locking on the client?
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> > > wrote:
> > > >
> > > > > SimpleConsumer do not have any concept of group management, only
> the
> > > > > high-level consumers have. So multiple simple consumers can
> > > independently
> > > > > consume from the same partition(s).
> > > > >
> > > > > Guozhang
> > > > >
> > > > >
> > > > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Yeah, I noticed that, i'm curious how balancing happens if
> > > > SimpleConsumer
> > > > > > is used. I.e. i can provide a partition to read from if i use
> > > > > > SimpleConsumer, but what if someone else already attached to that
> > > > > > partition, what would happen? Also what would happen if one
> > > > > SimpleConsumer
> > > > > > attached to all partitions? Noone would be able to join?
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> > > neha.narkhede@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > There are 2 types of consumer clients in Kafka -
> > > > > > ZookeeperConsumerConnector
> > > > > > > and SimpleConsumer. Only the former has the re balancing logic.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Neha
> > > > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > > But it looks like some clients don't implement it?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
Thanks! Direction in that proposal looks very good, I wish that would be
implemented already


On Tue, Oct 1, 2013 at 9:01 AM, Neha Narkhede <ne...@gmail.com>wrote:

> We do plan to move the group membership over to the server side and have a
> very thin consumer client. The proposal is here -
>
> https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPIand
> this is being planned for the 0.9 release. Once this is complete, the
> non-java clients can easily leverage the group management feature.
>
> Thanks,
> Neha
>
>
> On Tue, Oct 1, 2013 at 8:56 AM, Kane Kane <ka...@gmail.com> wrote:
>
> > The reason i was asking is that this library seems to have support only
> for
> > SimpleConsumer https://github.com/mumrah/kafka-python/, i was curious if
> > all should be implemented on client or kafka has some rebalancing logic
> and
> > prevent consuming from the same queue on server side in case of
> > SimpleConsumer api, but I see now that everything should implemented on
> the
> > client side.
> >
> > Thanks.
> >
> >
> > On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> > > I do not understand your question, what are you trying to implement?
> > >
> > >
> > > On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com>
> wrote:
> > >
> > > > So essentially you can't do "queue" pattern, unless you somehow
> > implement
> > > > locking on the client?
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> > > wrote:
> > > >
> > > > > SimpleConsumer do not have any concept of group management, only
> the
> > > > > high-level consumers have. So multiple simple consumers can
> > > independently
> > > > > consume from the same partition(s).
> > > > >
> > > > > Guozhang
> > > > >
> > > > >
> > > > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Yeah, I noticed that, i'm curious how balancing happens if
> > > > SimpleConsumer
> > > > > > is used. I.e. i can provide a partition to read from if i use
> > > > > > SimpleConsumer, but what if someone else already attached to that
> > > > > > partition, what would happen? Also what would happen if one
> > > > > SimpleConsumer
> > > > > > attached to all partitions? Noone would be able to join?
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> > > neha.narkhede@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > There are 2 types of consumer clients in Kafka -
> > > > > > ZookeeperConsumerConnector
> > > > > > > and SimpleConsumer. Only the former has the re balancing logic.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Neha
> > > > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > > But it looks like some clients don't implement it?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>

Re: as i understand rebalance happens on client side

Posted by Neha Narkhede <ne...@gmail.com>.
We do plan to move the group membership over to the server side and have a
very thin consumer client. The proposal is here -
https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPIand
this is being planned for the 0.9 release. Once this is complete, the
non-java clients can easily leverage the group management feature.

Thanks,
Neha


On Tue, Oct 1, 2013 at 8:56 AM, Kane Kane <ka...@gmail.com> wrote:

> The reason i was asking is that this library seems to have support only for
> SimpleConsumer https://github.com/mumrah/kafka-python/, i was curious if
> all should be implemented on client or kafka has some rebalancing logic and
> prevent consuming from the same queue on server side in case of
> SimpleConsumer api, but I see now that everything should implemented on the
> client side.
>
> Thanks.
>
>
> On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > I do not understand your question, what are you trying to implement?
> >
> >
> > On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com> wrote:
> >
> > > So essentially you can't do "queue" pattern, unless you somehow
> implement
> > > locking on the client?
> > >
> > >
> > > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> > wrote:
> > >
> > > > SimpleConsumer do not have any concept of group management, only the
> > > > high-level consumers have. So multiple simple consumers can
> > independently
> > > > consume from the same partition(s).
> > > >
> > > > Guozhang
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
> > wrote:
> > > >
> > > > > Yeah, I noticed that, i'm curious how balancing happens if
> > > SimpleConsumer
> > > > > is used. I.e. i can provide a partition to read from if i use
> > > > > SimpleConsumer, but what if someone else already attached to that
> > > > > partition, what would happen? Also what would happen if one
> > > > SimpleConsumer
> > > > > attached to all partitions? Noone would be able to join?
> > > > >
> > > > >
> > > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> > neha.narkhede@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > There are 2 types of consumer clients in Kafka -
> > > > > ZookeeperConsumerConnector
> > > > > > and SimpleConsumer. Only the former has the re balancing logic.
> > > > > >
> > > > > > Thanks,
> > > > > > Neha
> > > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com>
> wrote:
> > > > > >
> > > > > > > But it looks like some clients don't implement it?
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
The reason i was asking is that this library seems to have support only for
SimpleConsumer https://github.com/mumrah/kafka-python/, i was curious if
all should be implemented on client or kafka has some rebalancing logic and
prevent consuming from the same queue on server side in case of
SimpleConsumer api, but I see now that everything should implemented on the
client side.

Thanks.


On Tue, Oct 1, 2013 at 8:52 AM, Guozhang Wang <wa...@gmail.com> wrote:

> I do not understand your question, what are you trying to implement?
>
>
> On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com> wrote:
>
> > So essentially you can't do "queue" pattern, unless you somehow implement
> > locking on the client?
> >
> >
> > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> > > SimpleConsumer do not have any concept of group management, only the
> > > high-level consumers have. So multiple simple consumers can
> independently
> > > consume from the same partition(s).
> > >
> > > Guozhang
> > >
> > >
> > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
> wrote:
> > >
> > > > Yeah, I noticed that, i'm curious how balancing happens if
> > SimpleConsumer
> > > > is used. I.e. i can provide a partition to read from if i use
> > > > SimpleConsumer, but what if someone else already attached to that
> > > > partition, what would happen? Also what would happen if one
> > > SimpleConsumer
> > > > attached to all partitions? Noone would be able to join?
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> neha.narkhede@gmail.com
> > > > >wrote:
> > > >
> > > > > There are 2 types of consumer clients in Kafka -
> > > > ZookeeperConsumerConnector
> > > > > and SimpleConsumer. Only the former has the re balancing logic.
> > > > >
> > > > > Thanks,
> > > > > Neha
> > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
> > > > >
> > > > > > But it looks like some clients don't implement it?
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>
>
>
> --
> -- Guozhang
>

Re: as i understand rebalance happens on client side

Posted by Guozhang Wang <wa...@gmail.com>.
I do not understand your question, what are you trying to implement?


On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com> wrote:

> So essentially you can't do "queue" pattern, unless you somehow implement
> locking on the client?
>
>
> On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > SimpleConsumer do not have any concept of group management, only the
> > high-level consumers have. So multiple simple consumers can independently
> > consume from the same partition(s).
> >
> > Guozhang
> >
> >
> > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com> wrote:
> >
> > > Yeah, I noticed that, i'm curious how balancing happens if
> SimpleConsumer
> > > is used. I.e. i can provide a partition to read from if i use
> > > SimpleConsumer, but what if someone else already attached to that
> > > partition, what would happen? Also what would happen if one
> > SimpleConsumer
> > > attached to all partitions? Noone would be able to join?
> > >
> > >
> > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <neha.narkhede@gmail.com
> > > >wrote:
> > >
> > > > There are 2 types of consumer clients in Kafka -
> > > ZookeeperConsumerConnector
> > > > and SimpleConsumer. Only the former has the re balancing logic.
> > > >
> > > > Thanks,
> > > > Neha
> > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
> > > >
> > > > > But it looks like some clients don't implement it?
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>



-- 
-- Guozhang

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
Yes, and I actually want to use high level api, i just didn't understand
how kafka works inside. For some reason i thought in case of
SimpleConsumer, kafka would implement locking and rebalancing on server
side, letting client know that it can't attach to specific partition, etc.
But i see now that all management should happen on the client.

Thanks.


On Tue, Oct 1, 2013 at 8:48 AM, Neha Narkhede <ne...@gmail.com>wrote:

> I think you want locking to prevent two processes from consuming the same
> partition. You can just use the high level consumer to get this group
> management functionality. It ensures that a partition is only consumed by
> one high level consumer at any given time. Is there any particular reason
> why you want to use SimpleConsumer instead?
>
> Thanks,
> Neha
>
>
> On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com> wrote:
>
> > So essentially you can't do "queue" pattern, unless you somehow implement
> > locking on the client?
> >
> >
> > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com>
> wrote:
> >
> > > SimpleConsumer do not have any concept of group management, only the
> > > high-level consumers have. So multiple simple consumers can
> independently
> > > consume from the same partition(s).
> > >
> > > Guozhang
> > >
> > >
> > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com>
> wrote:
> > >
> > > > Yeah, I noticed that, i'm curious how balancing happens if
> > SimpleConsumer
> > > > is used. I.e. i can provide a partition to read from if i use
> > > > SimpleConsumer, but what if someone else already attached to that
> > > > partition, what would happen? Also what would happen if one
> > > SimpleConsumer
> > > > attached to all partitions? Noone would be able to join?
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <
> neha.narkhede@gmail.com
> > > > >wrote:
> > > >
> > > > > There are 2 types of consumer clients in Kafka -
> > > > ZookeeperConsumerConnector
> > > > > and SimpleConsumer. Only the former has the re balancing logic.
> > > > >
> > > > > Thanks,
> > > > > Neha
> > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
> > > > >
> > > > > > But it looks like some clients don't implement it?
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>

Re: as i understand rebalance happens on client side

Posted by Neha Narkhede <ne...@gmail.com>.
I think you want locking to prevent two processes from consuming the same
partition. You can just use the high level consumer to get this group
management functionality. It ensures that a partition is only consumed by
one high level consumer at any given time. Is there any particular reason
why you want to use SimpleConsumer instead?

Thanks,
Neha


On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <ka...@gmail.com> wrote:

> So essentially you can't do "queue" pattern, unless you somehow implement
> locking on the client?
>
>
> On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
> > SimpleConsumer do not have any concept of group management, only the
> > high-level consumers have. So multiple simple consumers can independently
> > consume from the same partition(s).
> >
> > Guozhang
> >
> >
> > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com> wrote:
> >
> > > Yeah, I noticed that, i'm curious how balancing happens if
> SimpleConsumer
> > > is used. I.e. i can provide a partition to read from if i use
> > > SimpleConsumer, but what if someone else already attached to that
> > > partition, what would happen? Also what would happen if one
> > SimpleConsumer
> > > attached to all partitions? Noone would be able to join?
> > >
> > >
> > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <neha.narkhede@gmail.com
> > > >wrote:
> > >
> > > > There are 2 types of consumer clients in Kafka -
> > > ZookeeperConsumerConnector
> > > > and SimpleConsumer. Only the former has the re balancing logic.
> > > >
> > > > Thanks,
> > > > Neha
> > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
> > > >
> > > > > But it looks like some clients don't implement it?
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
So essentially you can't do "queue" pattern, unless you somehow implement
locking on the client?


On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wa...@gmail.com> wrote:

> SimpleConsumer do not have any concept of group management, only the
> high-level consumers have. So multiple simple consumers can independently
> consume from the same partition(s).
>
> Guozhang
>
>
> On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com> wrote:
>
> > Yeah, I noticed that, i'm curious how balancing happens if SimpleConsumer
> > is used. I.e. i can provide a partition to read from if i use
> > SimpleConsumer, but what if someone else already attached to that
> > partition, what would happen? Also what would happen if one
> SimpleConsumer
> > attached to all partitions? Noone would be able to join?
> >
> >
> > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <neha.narkhede@gmail.com
> > >wrote:
> >
> > > There are 2 types of consumer clients in Kafka -
> > ZookeeperConsumerConnector
> > > and SimpleConsumer. Only the former has the re balancing logic.
> > >
> > > Thanks,
> > > Neha
> > > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
> > >
> > > > But it looks like some clients don't implement it?
> > > >
> > >
> >
>
>
>
> --
> -- Guozhang
>

Re: as i understand rebalance happens on client side

Posted by Guozhang Wang <wa...@gmail.com>.
SimpleConsumer do not have any concept of group management, only the
high-level consumers have. So multiple simple consumers can independently
consume from the same partition(s).

Guozhang


On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <ka...@gmail.com> wrote:

> Yeah, I noticed that, i'm curious how balancing happens if SimpleConsumer
> is used. I.e. i can provide a partition to read from if i use
> SimpleConsumer, but what if someone else already attached to that
> partition, what would happen? Also what would happen if one SimpleConsumer
> attached to all partitions? Noone would be able to join?
>
>
> On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > There are 2 types of consumer clients in Kafka -
> ZookeeperConsumerConnector
> > and SimpleConsumer. Only the former has the re balancing logic.
> >
> > Thanks,
> > Neha
> > On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
> >
> > > But it looks like some clients don't implement it?
> > >
> >
>



-- 
-- Guozhang

Re: as i understand rebalance happens on client side

Posted by Kane Kane <ka...@gmail.com>.
Yeah, I noticed that, i'm curious how balancing happens if SimpleConsumer
is used. I.e. i can provide a partition to read from if i use
SimpleConsumer, but what if someone else already attached to that
partition, what would happen? Also what would happen if one SimpleConsumer
attached to all partitions? Noone would be able to join?


On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede <ne...@gmail.com>wrote:

> There are 2 types of consumer clients in Kafka - ZookeeperConsumerConnector
> and SimpleConsumer. Only the former has the re balancing logic.
>
> Thanks,
> Neha
> On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:
>
> > But it looks like some clients don't implement it?
> >
>

Re: as i understand rebalance happens on client side

Posted by Neha Narkhede <ne...@gmail.com>.
There are 2 types of consumer clients in Kafka - ZookeeperConsumerConnector
and SimpleConsumer. Only the former has the re balancing logic.

Thanks,
Neha
On Oct 1, 2013 6:30 AM, "Kane Kane" <ka...@gmail.com> wrote:

> But it looks like some clients don't implement it?
>