You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Christian Carollo <cc...@gmail.com> on 2012/04/27 22:38:37 UTC
Zookeeper Usage and Operations
Is it possible to get a broad overview of what Zookeeper provides kafka? Why it is used with Kafka?
Also in the Wiki > Operations, there is a Zookeeper section but it really just says that it needs to be filled in.
Any chance we can get those operational details added?
https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
Thanks,
Christian
Re: Zookeeper Usage and Operations
Posted by Christian Carollo <cc...@gmail.com>.
Is there any update as to when the below documentation might be completed?
Thanks
Christian
On Apr 27, 2012, at 1:55 PM, Neha Narkhede <ne...@gmail.com> wrote:
>>> Any chance we can get those operational details added?
>
> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
>
> Certainly. I said I will do this, but then forgot about it. I will surely
> get this in by next week.
>
> Regarding Kafka's zookeeper dependency, I guess its not very clear from our
> design doc on what Kafka features you lose by not running zookeeper.
>
> If you choose to not run zookeeper, you will not be able to use the
> auto-load balancing on the consumer side. Also, the producer side
> zookeeper-based load balancing feature will be unusable.
>
> Let me see if I can update our design doc to make this clearer.
>
> Thanks,
> Neha
>
> On Fri, Apr 27, 2012 at 1:47 PM, Christian Carollo <cc...@gmail.com>wrote:
>
>> Thanks, Ed. I had seen that link. I am really looking for a clear outline/
>> set of rules as to when and why to use zookeeper and when and why not to.
>>
>> Then if using it makes sense, it seems like there are some value known
>> details about how to keep zookeeper happy that would be great to have
>> documented.
>>
>> Christian
>>
>>
>> On Apr 27, 2012, at 1:41 PM, Edward Smith <es...@stardotstar.org> wrote:
>>
>>> Christian,
>>>
>>> I'm new to Kafka, too. The page linked below describes how ZK is
>>> typically used with Kafka, although it is my impression that you don't
>>> have to use ZK if you don't want to. Not using it is also described
>>> briefly in the design.
>>>
>>> http://incubator.apache.org/kafka/design.html
>>>
>>> Ed
>>>
>>> On Fri, Apr 27, 2012 at 4:38 PM, Christian Carollo <cc...@gmail.com>
>> wrote:
>>>> Is it possible to get a broad overview of what Zookeeper provides
>> kafka? Why it is used with Kafka?
>>>>
>>>> Also in the Wiki > Operations, there is a Zookeeper section but it
>> really just says that it needs to be filled in.
>>>> Any chance we can get those operational details added?
>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
>>>>
>>>> Thanks,
>>>> Christian
>>
Re: Zookeeper Usage and Operations
Posted by Edward Smith <es...@stardotstar.org>.
Neha,
From my understanding, you would also lose the offset tracking in
the standard consumers.
Ed
On Fri, Apr 27, 2012 at 4:55 PM, Neha Narkhede <ne...@gmail.com> wrote:
>>> Any chance we can get those operational details added?
>
> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
>
> Certainly. I said I will do this, but then forgot about it. I will surely
> get this in by next week.
>
> Regarding Kafka's zookeeper dependency, I guess its not very clear from our
> design doc on what Kafka features you lose by not running zookeeper.
>
> If you choose to not run zookeeper, you will not be able to use the
> auto-load balancing on the consumer side. Also, the producer side
> zookeeper-based load balancing feature will be unusable.
>
> Let me see if I can update our design doc to make this clearer.
>
> Thanks,
> Neha
>
> On Fri, Apr 27, 2012 at 1:47 PM, Christian Carollo <cc...@gmail.com>wrote:
>
>> Thanks, Ed. I had seen that link. I am really looking for a clear outline/
>> set of rules as to when and why to use zookeeper and when and why not to.
>>
>> Then if using it makes sense, it seems like there are some value known
>> details about how to keep zookeeper happy that would be great to have
>> documented.
>>
>> Christian
>>
>>
>> On Apr 27, 2012, at 1:41 PM, Edward Smith <es...@stardotstar.org> wrote:
>>
>> > Christian,
>> >
>> > I'm new to Kafka, too. The page linked below describes how ZK is
>> > typically used with Kafka, although it is my impression that you don't
>> > have to use ZK if you don't want to. Not using it is also described
>> > briefly in the design.
>> >
>> > http://incubator.apache.org/kafka/design.html
>> >
>> > Ed
>> >
>> > On Fri, Apr 27, 2012 at 4:38 PM, Christian Carollo <cc...@gmail.com>
>> wrote:
>> >> Is it possible to get a broad overview of what Zookeeper provides
>> kafka? Why it is used with Kafka?
>> >>
>> >> Also in the Wiki > Operations, there is a Zookeeper section but it
>> really just says that it needs to be filled in.
>> >> Any chance we can get those operational details added?
>> >>
>> >>
>> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
>> >>
>> >> Thanks,
>> >> Christian
>>
Re: Zookeeper Usage and Operations
Posted by Neha Narkhede <ne...@gmail.com>.
>> Any chance we can get those operational details added?
https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
Certainly. I said I will do this, but then forgot about it. I will surely
get this in by next week.
Regarding Kafka's zookeeper dependency, I guess its not very clear from our
design doc on what Kafka features you lose by not running zookeeper.
If you choose to not run zookeeper, you will not be able to use the
auto-load balancing on the consumer side. Also, the producer side
zookeeper-based load balancing feature will be unusable.
Let me see if I can update our design doc to make this clearer.
Thanks,
Neha
On Fri, Apr 27, 2012 at 1:47 PM, Christian Carollo <cc...@gmail.com>wrote:
> Thanks, Ed. I had seen that link. I am really looking for a clear outline/
> set of rules as to when and why to use zookeeper and when and why not to.
>
> Then if using it makes sense, it seems like there are some value known
> details about how to keep zookeeper happy that would be great to have
> documented.
>
> Christian
>
>
> On Apr 27, 2012, at 1:41 PM, Edward Smith <es...@stardotstar.org> wrote:
>
> > Christian,
> >
> > I'm new to Kafka, too. The page linked below describes how ZK is
> > typically used with Kafka, although it is my impression that you don't
> > have to use ZK if you don't want to. Not using it is also described
> > briefly in the design.
> >
> > http://incubator.apache.org/kafka/design.html
> >
> > Ed
> >
> > On Fri, Apr 27, 2012 at 4:38 PM, Christian Carollo <cc...@gmail.com>
> wrote:
> >> Is it possible to get a broad overview of what Zookeeper provides
> kafka? Why it is used with Kafka?
> >>
> >> Also in the Wiki > Operations, there is a Zookeeper section but it
> really just says that it needs to be filled in.
> >> Any chance we can get those operational details added?
> >>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
> >>
> >> Thanks,
> >> Christian
>
Re: Zookeeper Usage and Operations
Posted by Christian Carollo <cc...@gmail.com>.
Thanks, Ed. I had seen that link. I am really looking for a clear outline/ set of rules as to when and why to use zookeeper and when and why not to.
Then if using it makes sense, it seems like there are some value known details about how to keep zookeeper happy that would be great to have documented.
Christian
On Apr 27, 2012, at 1:41 PM, Edward Smith <es...@stardotstar.org> wrote:
> Christian,
>
> I'm new to Kafka, too. The page linked below describes how ZK is
> typically used with Kafka, although it is my impression that you don't
> have to use ZK if you don't want to. Not using it is also described
> briefly in the design.
>
> http://incubator.apache.org/kafka/design.html
>
> Ed
>
> On Fri, Apr 27, 2012 at 4:38 PM, Christian Carollo <cc...@gmail.com> wrote:
>> Is it possible to get a broad overview of what Zookeeper provides kafka? Why it is used with Kafka?
>>
>> Also in the Wiki > Operations, there is a Zookeeper section but it really just says that it needs to be filled in.
>> Any chance we can get those operational details added?
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
>>
>> Thanks,
>> Christian
Re: Zookeeper Usage and Operations
Posted by Edward Smith <es...@stardotstar.org>.
Christian,
I'm new to Kafka, too. The page linked below describes how ZK is
typically used with Kafka, although it is my impression that you don't
have to use ZK if you don't want to. Not using it is also described
briefly in the design.
http://incubator.apache.org/kafka/design.html
Ed
On Fri, Apr 27, 2012 at 4:38 PM, Christian Carollo <cc...@gmail.com> wrote:
> Is it possible to get a broad overview of what Zookeeper provides kafka? Why it is used with Kafka?
>
> Also in the Wiki > Operations, there is a Zookeeper section but it really just says that it needs to be filled in.
> Any chance we can get those operational details added?
>
> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Zookeeper
>
> Thanks,
> Christian