Partition rebalancing when both leader and follower is down?

[Devendar Rao <de...@gmail.com>]

Hi,

I don't see partition rebalancing happening when both leader and replica
nodes are down for a given partition. The partition's leader is set to -1;
but never gets fixed. I manually assign the partition using
"kafka-preferred-replica-election.sh".  Does kafka not balance the
partitions automatically for this case?

I've this setting; I'm using kafka-0.10.2 version
and auto.leader.rebalance.enable is set to true

Error:
Leader for partition *my-topic* unavailable for fetching offset, wait for
metadata refresh

Thanks

Re: Partition rebalancing when both leader and follower is down?

[Matt Andruff <ma...@gmail.com>]

Can you clear up what you mean by "both leader and replica nodes are down
for a given partition"

If all the nodes are down for a partition you can't move the partition as
you don't have any data to copy.  (As this is a fundamental part of
re-balancing and leader election. I don't think it will work if there is No
data.  See the design doc
<https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Replication>that
explains how leader election/replication works and you'll get a feel for
why I think if you loose all you won't get a re-balance.)  Do you mean when
either a leader is down or a replica is down.(but not both?)  Re
balancing should happen if you loose some but not all of a partition.



On Thu, 8 Feb 2018 at 19:23 Devendar Rao <de...@gmail.com> wrote:

> Asking again to find out if there is a setting in KAFKA which can handle
> this case automatically.
>
> On Sat, Feb 3, 2018 at 12:45 PM, Devendar Rao <de...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I don't see partition rebalancing happening when both leader and replica
> > nodes are down for a given partition. The partition's leader is set to
> -1;
> > but never gets fixed. I manually assign the partition using
> > "kafka-preferred-replica-election.sh".  Does kafka not balance the
> > partitions automatically for this case?
> >
> > I've this setting; I'm using kafka-0.10.2 version
> > and auto.leader.rebalance.enable is set to true
> >
> > Error:
> > Leader for partition *my-topic* unavailable for fetching offset, wait for
> > metadata refresh
> >
> > Thanks
> >
>

Re: Partition rebalancing when both leader and follower is down?

[Devendar Rao <de...@gmail.com>]

Asking again to find out if there is a setting in KAFKA which can handle
this case automatically.

On Sat, Feb 3, 2018 at 12:45 PM, Devendar Rao <de...@gmail.com>
wrote:

> Hi,
>
> I don't see partition rebalancing happening when both leader and replica
> nodes are down for a given partition. The partition's leader is set to -1;
> but never gets fixed. I manually assign the partition using
> "kafka-preferred-replica-election.sh".  Does kafka not balance the
> partitions automatically for this case?
>
> I've this setting; I'm using kafka-0.10.2 version
> and auto.leader.rebalance.enable is set to true
>
> Error:
> Leader for partition *my-topic* unavailable for fetching offset, wait for
> metadata refresh
>
> Thanks
>