New to Apache Kafka

[Joe San <co...@gmail.com>]

Hi Apache Kafka,

I'm evaluating Apache Kafka for one of the projects that I'm into. I have
used ActiveMQ in the past which makes using Kafka pretty straightforward.
One thing that I do not understand is the need for Zookeeper?

I understand what Zookeeper is, but I fail to understand what purpose it
solves with Apache Kafka. Is it needed for the Apache Kafka cluster
management? Could you please elaborate a bit on this?

Regards,
Joe

Re: New to Apache Kafka

[Li Tao <ah...@gmail.com>]

As a cluster, each none in the cluster should know each other to function
properly. For a Cassandra cluster(I don't know ActiveMQ's mechanism), as an
example, it has its own
protocal to communicate with each other to know their condition.

For Kafka, each node stays independantly, they use zookeeper to know the
condition of the whole cluster. Each node communicates with zookeeper.

On Wed, Sep 16, 2015 at 7:24 AM, Ashish Shenoy <as...@instartlogic.com>
wrote:

> Zookeeper is a distributed coordination service. Kafka uses Zookeeper for
> various things like leader election, storing consumer-partition offsets
> etc.
>
> More information on each service is available at
> http://kafka.apache.org/documentation.html and
> https://zookeeper.apache.org/
> I highly recommend reading them for more information.
>
> TLDR:
> http://stackoverflow.com/questions/23751708/kafka-is-zookeeper-a-must
>
> Thanks,
> Ashish
>
> On Tue, Sep 15, 2015 at 1:45 PM, Joe San <co...@gmail.com> wrote:
>
> > Hi Apache Kafka,
> >
> > I'm evaluating Apache Kafka for one of the projects that I'm into. I have
> > used ActiveMQ in the past which makes using Kafka pretty straightforward.
> > One thing that I do not understand is the need for Zookeeper?
> >
> > I understand what Zookeeper is, but I fail to understand what purpose it
> > solves with Apache Kafka. Is it needed for the Apache Kafka cluster
> > management? Could you please elaborate a bit on this?
> >
> > Regards,
> > Joe
> >
>

Re: New to Apache Kafka

[Ashish Shenoy <as...@instartlogic.com>]

Zookeeper is a distributed coordination service. Kafka uses Zookeeper for
various things like leader election, storing consumer-partition offsets etc.

More information on each service is available at
http://kafka.apache.org/documentation.html and https://zookeeper.apache.org/
I highly recommend reading them for more information.

TLDR: http://stackoverflow.com/questions/23751708/kafka-is-zookeeper-a-must

Thanks,
Ashish

On Tue, Sep 15, 2015 at 1:45 PM, Joe San <co...@gmail.com> wrote:

> Hi Apache Kafka,
>
> I'm evaluating Apache Kafka for one of the projects that I'm into. I have
> used ActiveMQ in the past which makes using Kafka pretty straightforward.
> One thing that I do not understand is the need for Zookeeper?
>
> I understand what Zookeeper is, but I fail to understand what purpose it
> solves with Apache Kafka. Is it needed for the Apache Kafka cluster
> management? Could you please elaborate a bit on this?
>
> Regards,
> Joe
>