reassign a topic partition which has no ISR and leader set to -1

[Virendra Pratap Singh <vp...@yahoo-inc.com.INVALID>]

Ran into a situation where both the leader and replica nodes for a few partitions of a given topic went down. So now these partitions have no in-sync replicas and neither any leader (leader set to -1).
I tried to reassign these partitions to a different set of brokers using partition reassignment tool and then running the preferred replica. However I don't see the partitions getting reassigned and the leader still value still remains "-1" for these.
What do I need to get this working.
Kafka cluster is on 0.8.1.1
Regards,Virendra 

Re: reassign a topic partition which has no ISR and leader set to -1

[to...@borked.ca]

When we ran in to this problem we ended up going in to zookeeper and changing the leader to point to one of the replicas, then did a force leader election. This got the partition back online.


  Original Message  
From: Virendra Pratap Singh
Sent: Wednesday, March 4, 2015 2:00 AM
To: Gwen Shapira; users@kafka.apache.org
Reply To: users@kafka.apache.org
Subject: Re: reassign a topic partition which has no ISR and leader set to -1

Thanks Gwen for your info. I brought another kafka server up with assigned broker id of the failed leader and the reassignment went through.
@kafka developers who may be on this distribution list, is there a feature planned in 0.8.2 or can we plan one, where in if the leader and all the replica brokers are lost, the manual assignment of such a partition can be performed. As of current implementation, it looks like the reassignment will not budge till it sees at least one of the leader/replica brokers back again. 
Regards,Virendra From: Gwen Shapira <gs...@cloudera.com>
To: "users@kafka.apache.org" <us...@kafka.apache.org>; Virendra Pratap Singh <vp...@yahoo-inc.com> 
Sent: Tuesday, March 3, 2015 4:29 PM
Subject: Re: reassign a topic partition which has no ISR and leader set to -1

I hate bringing bad news, but...

You can't really reassign replicas if the leader is not available.
Since the leader is gone, the replicas have no where to replicate the data from.

Until you bring the leader back (or one of the replicas with unclean
leader election), you basically lost this partition.

Gwen



On Tue, Mar 3, 2015 at 3:57 PM, Virendra Pratap Singh
<vp...@yahoo-inc.com.invalid> wrote:
> Ran into a situation where both the leader and replica nodes for a few partitions of a given topic went down. So now these partitions have no in-sync replicas and neither any leader (leader set to -1).
> I tried to reassign these partitions to a different set of brokers using partition reassignment tool and then running the preferred replica. However I don't see the partitions getting reassigned and the leader still value still remains "-1" for these.
> What do I need to get this working.
> Kafka cluster is on 0.8.1.1
> Regards,Virendra

Re: reassign a topic partition which has no ISR and leader set to -1

[Virendra Pratap Singh <vp...@yahoo-inc.com.INVALID>]

Thanks Gwen for your info. I brought another kafka server up with assigned broker id of the failed leader and the reassignment went through.
@kafka developers who may be on this distribution list, is there a feature planned in 0.8.2 or can we plan one, where in if the leader and all the replica brokers are lost, the manual assignment of such a partition can be performed. As of current implementation, it looks like the reassignment will not budge till it sees at least one of the leader/replica brokers back again. 
Regards,Virendra      From: Gwen Shapira <gs...@cloudera.com>
 To: "users@kafka.apache.org" <us...@kafka.apache.org>; Virendra Pratap Singh <vp...@yahoo-inc.com> 
 Sent: Tuesday, March 3, 2015 4:29 PM
 Subject: Re: reassign a topic partition which has no ISR and leader set to -1
   
I hate bringing bad news, but...

You can't really reassign replicas if the leader is not available.
Since the leader is gone, the replicas have no where to replicate the data from.

Until you bring the leader back (or one of the replicas with unclean
leader election), you basically lost this partition.

Gwen



On Tue, Mar 3, 2015 at 3:57 PM, Virendra Pratap Singh
<vp...@yahoo-inc.com.invalid> wrote:
> Ran into a situation where both the leader and replica nodes for a few partitions of a given topic went down. So now these partitions have no in-sync replicas and neither any leader (leader set to -1).
> I tried to reassign these partitions to a different set of brokers using partition reassignment tool and then running the preferred replica. However I don't see the partitions getting reassigned and the leader still value still remains "-1" for these.
> What do I need to get this working.
> Kafka cluster is on 0.8.1.1
> Regards,Virendra


  

Re: reassign a topic partition which has no ISR and leader set to -1

[Gwen Shapira <gs...@cloudera.com>]

I hate bringing bad news, but...

You can't really reassign replicas if the leader is not available.
Since the leader is gone, the replicas have no where to replicate the data from.

Until you bring the leader back (or one of the replicas with unclean
leader election), you basically lost this partition.

Gwen

On Tue, Mar 3, 2015 at 3:57 PM, Virendra Pratap Singh
<vp...@yahoo-inc.com.invalid> wrote:
> Ran into a situation where both the leader and replica nodes for a few partitions of a given topic went down. So now these partitions have no in-sync replicas and neither any leader (leader set to -1).
> I tried to reassign these partitions to a different set of brokers using partition reassignment tool and then running the preferred replica. However I don't see the partitions getting reassigned and the leader still value still remains "-1" for these.
> What do I need to get this working.
> Kafka cluster is on 0.8.1.1
> Regards,Virendra