You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by TianYi Zhu <tz...@pureprofile.com> on 2014/09/03 08:09:33 UTC
Leader became -1 and no ISR for all topics/patitions
Hi all,
I'm running kafka 0.8.1.1, and encountering a weird problem.
All my partitions' leader became -1 some day, and the ISR became empty, for
example
Topic:infinity PartitionCount:12 ReplicationFactor:2 Configs:
Topic: infinity Partition: 0 Leader: -1 Replicas: 0,1 Isr:
Topic: infinity Partition: 1 Leader: -1 Replicas: 1,0 Isr:
Topic: infinity Partition: 2 Leader: -1 Replicas: 0,1 Isr:
Topic: infinity Partition: 3 Leader: -1 Replicas: 1,0 Isr:
Topic: infinity Partition: 4 Leader: -1 Replicas: 0,1 Isr:
Topic: infinity Partition: 5 Leader: -1 Replicas: 1,0 Isr:
Topic: infinity Partition: 6 Leader: -1 Replicas: 0,1 Isr:
Topic: infinity Partition: 7 Leader: -1 Replicas: 1,0 Isr:
Topic: infinity Partition: 8 Leader: -1 Replicas: 0,1 Isr:
Topic: infinity Partition: 9 Leader: -1 Replicas: 1,0 Isr:
Topic: infinity Partition: 10 Leader: -1 Replicas: 0,1 Isr:
Topic: infinity Partition: 11 Leader: -1 Replicas: 1,0 Isr:
However, my high level producers and consumers are still working.
Should I run kafka-preferred-replica-election.sh at this stage? Or should I
restart the server in the cluster one by one?
There's no errors in the log file.
--
Thanks,
TianYi ZHU
Re: Leader became -1 and no ISR for all topics/patitions
Posted by TianYi Zhu <tz...@pureprofile.com>.
No I think, as the high level producer and consumer are still working.
On Fri, Sep 5, 2014 at 9:05 AM, Gwen Shapira <gs...@cloudera.com> wrote:
> Was there any data loss?
> On Sep 4, 2014 3:52 PM, "TianYi Zhu" <tz...@pureprofile.com> wrote:
>
> > No log in the recent 7 days.
> >
> > kafka-preferred-replica-election.sh won't work in this stage, as the IRS
> is
> > null. I restarted the servers one by one then it's back to normal.
> >
> >
> > On Fri, Sep 5, 2014 at 1:11 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Any error in the controller log?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Tue, Sep 2, 2014 at 11:09 PM, TianYi Zhu <tz...@pureprofile.com>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm running kafka 0.8.1.1, and encountering a weird problem.
> > > > All my partitions' leader became -1 some day, and the ISR became
> empty,
> > > for
> > > > example
> > > >
> > > > Topic:infinity PartitionCount:12 ReplicationFactor:2
> > Configs:
> > > > Topic: infinity Partition: 0 Leader: -1 Replicas: 0,1
> > > > Isr:
> > > > Topic: infinity Partition: 1 Leader: -1 Replicas: 1,0
> > > > Isr:
> > > > Topic: infinity Partition: 2 Leader: -1 Replicas: 0,1
> > > > Isr:
> > > > Topic: infinity Partition: 3 Leader: -1 Replicas: 1,0
> > > > Isr:
> > > > Topic: infinity Partition: 4 Leader: -1 Replicas: 0,1
> > > > Isr:
> > > > Topic: infinity Partition: 5 Leader: -1 Replicas: 1,0
> > > > Isr:
> > > > Topic: infinity Partition: 6 Leader: -1 Replicas: 0,1
> > > > Isr:
> > > > Topic: infinity Partition: 7 Leader: -1 Replicas: 1,0
> > > > Isr:
> > > > Topic: infinity Partition: 8 Leader: -1 Replicas: 0,1
> > > > Isr:
> > > > Topic: infinity Partition: 9 Leader: -1 Replicas: 1,0
> > > > Isr:
> > > > Topic: infinity Partition: 10 Leader: -1 Replicas: 0,1
> > > > Isr:
> > > > Topic: infinity Partition: 11 Leader: -1 Replicas: 1,0
> > > > Isr:
> > > >
> > > > However, my high level producers and consumers are still working.
> > > > Should I run kafka-preferred-replica-election.sh at this stage? Or
> > > should I
> > > > restart the server in the cluster one by one?
> > > >
> > > > There's no errors in the log file.
> > > > --
> > > > Thanks,
> > > > TianYi ZHU
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > TianYi ZHU
> >
>
--
Thanks,
TianYi ZHU
Re: Leader became -1 and no ISR for all topics/patitions
Posted by Gwen Shapira <gs...@cloudera.com>.
Was there any data loss?
On Sep 4, 2014 3:52 PM, "TianYi Zhu" <tz...@pureprofile.com> wrote:
> No log in the recent 7 days.
>
> kafka-preferred-replica-election.sh won't work in this stage, as the IRS is
> null. I restarted the servers one by one then it's back to normal.
>
>
> On Fri, Sep 5, 2014 at 1:11 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Any error in the controller log?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Tue, Sep 2, 2014 at 11:09 PM, TianYi Zhu <tz...@pureprofile.com>
> wrote:
> >
> > > Hi all,
> > >
> > > I'm running kafka 0.8.1.1, and encountering a weird problem.
> > > All my partitions' leader became -1 some day, and the ISR became empty,
> > for
> > > example
> > >
> > > Topic:infinity PartitionCount:12 ReplicationFactor:2
> Configs:
> > > Topic: infinity Partition: 0 Leader: -1 Replicas: 0,1
> > > Isr:
> > > Topic: infinity Partition: 1 Leader: -1 Replicas: 1,0
> > > Isr:
> > > Topic: infinity Partition: 2 Leader: -1 Replicas: 0,1
> > > Isr:
> > > Topic: infinity Partition: 3 Leader: -1 Replicas: 1,0
> > > Isr:
> > > Topic: infinity Partition: 4 Leader: -1 Replicas: 0,1
> > > Isr:
> > > Topic: infinity Partition: 5 Leader: -1 Replicas: 1,0
> > > Isr:
> > > Topic: infinity Partition: 6 Leader: -1 Replicas: 0,1
> > > Isr:
> > > Topic: infinity Partition: 7 Leader: -1 Replicas: 1,0
> > > Isr:
> > > Topic: infinity Partition: 8 Leader: -1 Replicas: 0,1
> > > Isr:
> > > Topic: infinity Partition: 9 Leader: -1 Replicas: 1,0
> > > Isr:
> > > Topic: infinity Partition: 10 Leader: -1 Replicas: 0,1
> > > Isr:
> > > Topic: infinity Partition: 11 Leader: -1 Replicas: 1,0
> > > Isr:
> > >
> > > However, my high level producers and consumers are still working.
> > > Should I run kafka-preferred-replica-election.sh at this stage? Or
> > should I
> > > restart the server in the cluster one by one?
> > >
> > > There's no errors in the log file.
> > > --
> > > Thanks,
> > > TianYi ZHU
> > >
> >
>
>
>
> --
> Thanks,
> TianYi ZHU
>
Re: Leader became -1 and no ISR for all topics/patitions
Posted by TianYi Zhu <tz...@pureprofile.com>.
No log in the recent 7 days.
kafka-preferred-replica-election.sh won't work in this stage, as the IRS is
null. I restarted the servers one by one then it's back to normal.
On Fri, Sep 5, 2014 at 1:11 AM, Jun Rao <ju...@gmail.com> wrote:
> Any error in the controller log?
>
> Thanks,
>
> Jun
>
>
> On Tue, Sep 2, 2014 at 11:09 PM, TianYi Zhu <tz...@pureprofile.com> wrote:
>
> > Hi all,
> >
> > I'm running kafka 0.8.1.1, and encountering a weird problem.
> > All my partitions' leader became -1 some day, and the ISR became empty,
> for
> > example
> >
> > Topic:infinity PartitionCount:12 ReplicationFactor:2 Configs:
> > Topic: infinity Partition: 0 Leader: -1 Replicas: 0,1
> > Isr:
> > Topic: infinity Partition: 1 Leader: -1 Replicas: 1,0
> > Isr:
> > Topic: infinity Partition: 2 Leader: -1 Replicas: 0,1
> > Isr:
> > Topic: infinity Partition: 3 Leader: -1 Replicas: 1,0
> > Isr:
> > Topic: infinity Partition: 4 Leader: -1 Replicas: 0,1
> > Isr:
> > Topic: infinity Partition: 5 Leader: -1 Replicas: 1,0
> > Isr:
> > Topic: infinity Partition: 6 Leader: -1 Replicas: 0,1
> > Isr:
> > Topic: infinity Partition: 7 Leader: -1 Replicas: 1,0
> > Isr:
> > Topic: infinity Partition: 8 Leader: -1 Replicas: 0,1
> > Isr:
> > Topic: infinity Partition: 9 Leader: -1 Replicas: 1,0
> > Isr:
> > Topic: infinity Partition: 10 Leader: -1 Replicas: 0,1
> > Isr:
> > Topic: infinity Partition: 11 Leader: -1 Replicas: 1,0
> > Isr:
> >
> > However, my high level producers and consumers are still working.
> > Should I run kafka-preferred-replica-election.sh at this stage? Or
> should I
> > restart the server in the cluster one by one?
> >
> > There's no errors in the log file.
> > --
> > Thanks,
> > TianYi ZHU
> >
>
--
Thanks,
TianYi ZHU
Re: Leader became -1 and no ISR for all topics/patitions
Posted by Jun Rao <ju...@gmail.com>.
Any error in the controller log?
Thanks,
Jun
On Tue, Sep 2, 2014 at 11:09 PM, TianYi Zhu <tz...@pureprofile.com> wrote:
> Hi all,
>
> I'm running kafka 0.8.1.1, and encountering a weird problem.
> All my partitions' leader became -1 some day, and the ISR became empty, for
> example
>
> Topic:infinity PartitionCount:12 ReplicationFactor:2 Configs:
> Topic: infinity Partition: 0 Leader: -1 Replicas: 0,1
> Isr:
> Topic: infinity Partition: 1 Leader: -1 Replicas: 1,0
> Isr:
> Topic: infinity Partition: 2 Leader: -1 Replicas: 0,1
> Isr:
> Topic: infinity Partition: 3 Leader: -1 Replicas: 1,0
> Isr:
> Topic: infinity Partition: 4 Leader: -1 Replicas: 0,1
> Isr:
> Topic: infinity Partition: 5 Leader: -1 Replicas: 1,0
> Isr:
> Topic: infinity Partition: 6 Leader: -1 Replicas: 0,1
> Isr:
> Topic: infinity Partition: 7 Leader: -1 Replicas: 1,0
> Isr:
> Topic: infinity Partition: 8 Leader: -1 Replicas: 0,1
> Isr:
> Topic: infinity Partition: 9 Leader: -1 Replicas: 1,0
> Isr:
> Topic: infinity Partition: 10 Leader: -1 Replicas: 0,1
> Isr:
> Topic: infinity Partition: 11 Leader: -1 Replicas: 1,0
> Isr:
>
> However, my high level producers and consumers are still working.
> Should I run kafka-preferred-replica-election.sh at this stage? Or should I
> restart the server in the cluster one by one?
>
> There's no errors in the log file.
> --
> Thanks,
> TianYi ZHU
>