Kafka - leadership

[Tomer Shavit <mo...@gmail.com>]

Hi,

I have a question regarding kafka. if i have cluster which contains only
one broker, and i have topic which contains 2 partions. does it mean that
the same broker will be the leader for those 2 partitions ? can broker be
the leader for more than one partition ?

Thanks,
Tomer

Re: Kafka - leadership

[Michal Michalski <mi...@zalando.ie>]

Hi Tomer,

1. Yes, one broker can (will) be a leader for multiple partitions - you'll
usually have a relatively small number of brokers in your cluster, but they
will store data for many topics, with many partitions, so (assuming
auto.leader.rebalance.enable is set to true; I think it's  thedefault in
recent Kafka versions?) the leadership will be split more or less evenly
between all the brokers that given partition is replicated to.
2. No, not for a "healthy" cluster - as soon as such state is "detected"
(e.g. after broker went down), controller should fix it immediately and
"rebalance" the leadership.

Michał

On 4 October 2017 at 14:30, Tomer Shavit <mo...@gmail.com> wrote:

> Hi,
>
> I have a question regarding kafka. if i have cluster which contains only
> one broker, and i have topic which contains 2 partions. does it mean that
> the same broker will be the leader for those 2 partitions ? can broker be
> the leader for more than one partition ?
>
> Thanks,
> Tomer
>

Re: Kafka - leadership

[Jakub Scholz <ja...@scholz.cz>]

One broker can be leader for many partitions. Partitions are not only used
to scale the brokers but also to scale the clients. So in many cases it
makes perfect sense to have many partitions running from the same broker.
(Of course having only one broker in total means no HA etc., but I guess
you understand that ;-))

Under normal circumstances there should be always a leader for each
partition. But of course there might be always some "temporary" situations
when there is no leader for sometime such as when failovers happen etc.

Jakub

On Wed, Oct 4, 2017 at 4:00 PM, Tomer Shavit <mo...@gmail.com> wrote:

> and i have an additional question, can partion be without leader ? and if
> so, what does it mean ?
>
> ---------- Forwarded message ----------
> From: Tomer Shavit <mo...@gmail.com>
> Date: 2017-10-04 16:30 GMT+03:00
> Subject: Kafka - leadership
> To: ‫users@kafka.apache.org‬
>
>
> Hi,
>
> I have a question regarding kafka. if i have cluster which contains only
> one broker, and i have topic which contains 2 partions. does it mean that
> the same broker will be the leader for those 2 partitions ? can broker be
> the leader for more than one partition ?
>
> Thanks,
> Tomer
>

Fwd: Kafka - leadership

[Tomer Shavit <mo...@gmail.com>]

and i have an additional question, can partion be without leader ? and if
so, what does it mean ?

---------- Forwarded message ----------
From: Tomer Shavit <mo...@gmail.com>
Date: 2017-10-04 16:30 GMT+03:00
Subject: Kafka - leadership
To: ‫users@kafka.apache.org‬


Hi,

I have a question regarding kafka. if i have cluster which contains only
one broker, and i have topic which contains 2 partions. does it mean that
the same broker will be the leader for those 2 partitions ? can broker be
the leader for more than one partition ?

Thanks,
Tomer