You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by sunil kalva <ka...@gmail.com> on 2015/11/23 09:29:06 UTC
replication factor increased automatically !
Hi
We have a 10 node kafka cluster, and we have created a topic with
replication factor "3" . It was running fine for 10 days, after that the
replication factor is increased to "8" with out running any command to
increase the replication. We have verified in server logs and also
confirmed with the team that we haven't ran the command to increase. Once
the replication factor is increased to "8", topic also stopped taking
traffic.
Did someone faced similar issue, or its known issue ?
t
SunilKalva
Re: replication factor increased automatically !
Posted by sunil kalva <ka...@gmail.com>.
We found the issue, it was with our UI tool which had a bug in partition
reassignment tool. Nothing todo with kafka.
Sorry for spamming.
t
Sunil
On Mon, Nov 23, 2015 at 4:13 PM, sunil kalva <ka...@gmail.com> wrote:
> hi
> sorry missed the header in my prev mail
>
> kafka_2.11-0.8.2.1/bin/kafka-topics.sh --describe --zookeeper
> zookeeperhost:2181 --topic testkafka
> Topic:testkafka PartitionCount:10 ReplicationFactor:8 Configs:
> Topic: testkafka Partition: 0 Leader: 2 Replicas: 5,10,1,9,2,7,3,8 Isr:
> 9,8,2
> Topic: testkafka Partition: 1 Leader: 10 Replicas: 10,1,6,9,2,3,8,4 Isr:
> 10,6,9,2,3,8,4
> Topic: testkafka Partition: 2 Leader: 4 Replicas: 5,10,1,9,2,7,3,4 Isr:
> 10,4
> Topic: testkafka Partition: 3 Leader: 5 Replicas: 5,10,1,6,2,3,8,4 Isr:
> 2,5
> Topic: testkafka Partition: 4 Leader: 2 Replicas: 5,1,6,9,2,7,3,4 Isr: 3,2
> Topic: testkafka Partition: 5 Leader: 7 Replicas: 5,10,6,2,7,3,8,4 Isr:
> 7,3,4
> Topic: testkafka Partition: 6 Leader: 8 Replicas: 5,1,6,9,7,3,8,4 Isr: 8,4
> Topic: testkafka Partition: 7 Leader: 9 Replicas: 5,10,6,9,2,7,8,4 Isr:
> 9,6
> Topic: testkafka Partition: 8 Leader: 10 Replicas: 5,10,1,6,9,7,3,8 Isr:
> 10,7
> Topic: testkafka Partition: 9 Leader: 7 Replicas: 10,1,6,9,2,7,8,4 Isr:
> 8,7
>
> t
> SunilKalva
>
> On Mon, Nov 23, 2015 at 3:58 PM, Prabhjot Bharaj <pr...@gmail.com>
> wrote:
>
>> Hi Sunil,
>>
>> Please paste the full output, including the first line of the output of
>> kafka-topics.sh.
>>
>> e.g.
>>
>> root@x.x.x.x:~# kafka-topics.sh --describe --zookeeper localhost:2182
>> --topic part_2_repl_3
>>
>> *Topic:part_2_repl_3 PartitionCount:2 ReplicationFactor:3 Configs:*
>>
>> Topic: part_2_repl_3 Partition: 0 Leader: 4 Replicas: 4,2,6 Isr: 4,2,6
>>
>> Topic: part_2_repl_3 Partition: 1 Leader: 6 Replicas: 6,4,7 Isr: 7,4,6
>>
>> ----------------
>> The line in bold (i.e. the first line in the output of the
>> kafka-topics.sh)
>> gives on a topic level - #partitions, #replicationfactor etc
>> Please paste the full output
>>
>> Regards,
>> Prabhjot
>>
>> On Mon, Nov 23, 2015 at 2:38 PM, sunil kalva <ka...@gmail.com>
>> wrote:
>>
>> > Topic: testkafka Partition: 0 Leader: 5 Replicas:
>> 5,10,1,9,2,7,3,8
>> > Isr:
>> > 5,10,1,9,7,3,8
>> > Topic: testkafka Partition: 1 Leader: 10 Replicas: 10,1,6,9,2,3,8,4 Isr:
>> > 10,6,9,3,8,4
>> > Topic: testkafka Partition: 2 Leader: 4 Replicas: 5,10,1,9,2,7,3,4 Isr:
>> > 10,4
>> > Topic: testkafka Partition: 3 Leader: 5 Replicas: 5,10,1,6,2,3,8,4 Isr:
>> 5,1
>> > Topic: testkafka Partition: 4 Leader: 5 Replicas: 5,1,6,9,2,7,3,4 Isr:
>> > 5,1,6,9,7,3,4
>> > Topic: testkafka Partition: 5 Leader: 7 Replicas: 5,10,6,2,7,3,8,4 Isr:
>> > 7,3,4
>> > Topic: testkafka Partition: 6 Leader: 8 Replicas: 5,1,6,9,7,3,8,4 Isr:
>> 8,4
>> > Topic: testkafka Partition: 7 Leader: 9 Replicas: 5,10,6,9,2,7,8,4 Isr:
>> 9,6
>> > Topic: testkafka Partition: 8 Leader: 10 Replicas: 5,10,1,6,9,7,3,8 Isr:
>> > 10,7
>> > Topic: testkafka Partition: 9 Leader: 7 Replicas: 10,1,6,9,2,7,8,4 Isr:
>> 8,7
>> >
>> >
>> >
>> > On Mon, Nov 23, 2015 at 2:23 PM, Raju Bairishetti <ra...@apache.org>
>> wrote:
>> >
>> > > Hello Sunil,
>> > > Are you able to see 8 replicas(i.e. in ISR) for all the
>> partitions?
>> > >
>> > > Can you paste the output of *--describe *topic command?
>> > > kafka-topics --zookeeper <zk-string> *-describe* <topic name>
>> > >
>> > > On Mon, Nov 23, 2015 at 4:29 PM, sunil kalva <ka...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi
>> > > > We have a 10 node kafka cluster, and we have created a topic with
>> > > > replication factor "3" . It was running fine for 10 days, after that
>> > the
>> > > > replication factor is increased to "8" with out running any command
>> to
>> > > > increase the replication. We have verified in server logs and also
>> > > > confirmed with the team that we haven't ran the command to increase.
>> > Once
>> > > > the replication factor is increased to "8", topic also stopped
>> taking
>> > > > traffic.
>> > > > Did someone faced similar issue, or its known issue ?
>> > > >
>> > > > t
>> > > > SunilKalva
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> ---------------------------------------------------------
>> "There are only 10 types of people in the world: Those who understand
>> binary, and those who don't"
>>
>
>
Re: replication factor increased automatically !
Posted by sunil kalva <ka...@gmail.com>.
hi
sorry missed the header in my prev mail
kafka_2.11-0.8.2.1/bin/kafka-topics.sh --describe --zookeeper
zookeeperhost:2181 --topic testkafka
Topic:testkafka PartitionCount:10 ReplicationFactor:8 Configs:
Topic: testkafka Partition: 0 Leader: 2 Replicas: 5,10,1,9,2,7,3,8 Isr:
9,8,2
Topic: testkafka Partition: 1 Leader: 10 Replicas: 10,1,6,9,2,3,8,4 Isr:
10,6,9,2,3,8,4
Topic: testkafka Partition: 2 Leader: 4 Replicas: 5,10,1,9,2,7,3,4 Isr: 10,4
Topic: testkafka Partition: 3 Leader: 5 Replicas: 5,10,1,6,2,3,8,4 Isr: 2,5
Topic: testkafka Partition: 4 Leader: 2 Replicas: 5,1,6,9,2,7,3,4 Isr: 3,2
Topic: testkafka Partition: 5 Leader: 7 Replicas: 5,10,6,2,7,3,8,4 Isr:
7,3,4
Topic: testkafka Partition: 6 Leader: 8 Replicas: 5,1,6,9,7,3,8,4 Isr: 8,4
Topic: testkafka Partition: 7 Leader: 9 Replicas: 5,10,6,9,2,7,8,4 Isr: 9,6
Topic: testkafka Partition: 8 Leader: 10 Replicas: 5,10,1,6,9,7,3,8 Isr:
10,7
Topic: testkafka Partition: 9 Leader: 7 Replicas: 10,1,6,9,2,7,8,4 Isr: 8,7
t
SunilKalva
On Mon, Nov 23, 2015 at 3:58 PM, Prabhjot Bharaj <pr...@gmail.com>
wrote:
> Hi Sunil,
>
> Please paste the full output, including the first line of the output of
> kafka-topics.sh.
>
> e.g.
>
> root@x.x.x.x:~# kafka-topics.sh --describe --zookeeper localhost:2182
> --topic part_2_repl_3
>
> *Topic:part_2_repl_3 PartitionCount:2 ReplicationFactor:3 Configs:*
>
> Topic: part_2_repl_3 Partition: 0 Leader: 4 Replicas: 4,2,6 Isr: 4,2,6
>
> Topic: part_2_repl_3 Partition: 1 Leader: 6 Replicas: 6,4,7 Isr: 7,4,6
>
> ----------------
> The line in bold (i.e. the first line in the output of the kafka-topics.sh)
> gives on a topic level - #partitions, #replicationfactor etc
> Please paste the full output
>
> Regards,
> Prabhjot
>
> On Mon, Nov 23, 2015 at 2:38 PM, sunil kalva <ka...@gmail.com>
> wrote:
>
> > Topic: testkafka Partition: 0 Leader: 5 Replicas:
> 5,10,1,9,2,7,3,8
> > Isr:
> > 5,10,1,9,7,3,8
> > Topic: testkafka Partition: 1 Leader: 10 Replicas: 10,1,6,9,2,3,8,4 Isr:
> > 10,6,9,3,8,4
> > Topic: testkafka Partition: 2 Leader: 4 Replicas: 5,10,1,9,2,7,3,4 Isr:
> > 10,4
> > Topic: testkafka Partition: 3 Leader: 5 Replicas: 5,10,1,6,2,3,8,4 Isr:
> 5,1
> > Topic: testkafka Partition: 4 Leader: 5 Replicas: 5,1,6,9,2,7,3,4 Isr:
> > 5,1,6,9,7,3,4
> > Topic: testkafka Partition: 5 Leader: 7 Replicas: 5,10,6,2,7,3,8,4 Isr:
> > 7,3,4
> > Topic: testkafka Partition: 6 Leader: 8 Replicas: 5,1,6,9,7,3,8,4 Isr:
> 8,4
> > Topic: testkafka Partition: 7 Leader: 9 Replicas: 5,10,6,9,2,7,8,4 Isr:
> 9,6
> > Topic: testkafka Partition: 8 Leader: 10 Replicas: 5,10,1,6,9,7,3,8 Isr:
> > 10,7
> > Topic: testkafka Partition: 9 Leader: 7 Replicas: 10,1,6,9,2,7,8,4 Isr:
> 8,7
> >
> >
> >
> > On Mon, Nov 23, 2015 at 2:23 PM, Raju Bairishetti <ra...@apache.org>
> wrote:
> >
> > > Hello Sunil,
> > > Are you able to see 8 replicas(i.e. in ISR) for all the partitions?
> > >
> > > Can you paste the output of *--describe *topic command?
> > > kafka-topics --zookeeper <zk-string> *-describe* <topic name>
> > >
> > > On Mon, Nov 23, 2015 at 4:29 PM, sunil kalva <ka...@gmail.com>
> > > wrote:
> > >
> > > > Hi
> > > > We have a 10 node kafka cluster, and we have created a topic with
> > > > replication factor "3" . It was running fine for 10 days, after that
> > the
> > > > replication factor is increased to "8" with out running any command
> to
> > > > increase the replication. We have verified in server logs and also
> > > > confirmed with the team that we haven't ran the command to increase.
> > Once
> > > > the replication factor is increased to "8", topic also stopped taking
> > > > traffic.
> > > > Did someone faced similar issue, or its known issue ?
> > > >
> > > > t
> > > > SunilKalva
> > > >
> > >
> >
>
>
>
> --
> ---------------------------------------------------------
> "There are only 10 types of people in the world: Those who understand
> binary, and those who don't"
>
Re: replication factor increased automatically !
Posted by Prabhjot Bharaj <pr...@gmail.com>.
Hi Sunil,
Please paste the full output, including the first line of the output of
kafka-topics.sh.
e.g.
root@x.x.x.x:~# kafka-topics.sh --describe --zookeeper localhost:2182
--topic part_2_repl_3
*Topic:part_2_repl_3 PartitionCount:2 ReplicationFactor:3 Configs:*
Topic: part_2_repl_3 Partition: 0 Leader: 4 Replicas: 4,2,6 Isr: 4,2,6
Topic: part_2_repl_3 Partition: 1 Leader: 6 Replicas: 6,4,7 Isr: 7,4,6
----------------
The line in bold (i.e. the first line in the output of the kafka-topics.sh)
gives on a topic level - #partitions, #replicationfactor etc
Please paste the full output
Regards,
Prabhjot
On Mon, Nov 23, 2015 at 2:38 PM, sunil kalva <ka...@gmail.com> wrote:
> Topic: testkafka Partition: 0 Leader: 5 Replicas: 5,10,1,9,2,7,3,8
> Isr:
> 5,10,1,9,7,3,8
> Topic: testkafka Partition: 1 Leader: 10 Replicas: 10,1,6,9,2,3,8,4 Isr:
> 10,6,9,3,8,4
> Topic: testkafka Partition: 2 Leader: 4 Replicas: 5,10,1,9,2,7,3,4 Isr:
> 10,4
> Topic: testkafka Partition: 3 Leader: 5 Replicas: 5,10,1,6,2,3,8,4 Isr: 5,1
> Topic: testkafka Partition: 4 Leader: 5 Replicas: 5,1,6,9,2,7,3,4 Isr:
> 5,1,6,9,7,3,4
> Topic: testkafka Partition: 5 Leader: 7 Replicas: 5,10,6,2,7,3,8,4 Isr:
> 7,3,4
> Topic: testkafka Partition: 6 Leader: 8 Replicas: 5,1,6,9,7,3,8,4 Isr: 8,4
> Topic: testkafka Partition: 7 Leader: 9 Replicas: 5,10,6,9,2,7,8,4 Isr: 9,6
> Topic: testkafka Partition: 8 Leader: 10 Replicas: 5,10,1,6,9,7,3,8 Isr:
> 10,7
> Topic: testkafka Partition: 9 Leader: 7 Replicas: 10,1,6,9,2,7,8,4 Isr: 8,7
>
>
>
> On Mon, Nov 23, 2015 at 2:23 PM, Raju Bairishetti <ra...@apache.org> wrote:
>
> > Hello Sunil,
> > Are you able to see 8 replicas(i.e. in ISR) for all the partitions?
> >
> > Can you paste the output of *--describe *topic command?
> > kafka-topics --zookeeper <zk-string> *-describe* <topic name>
> >
> > On Mon, Nov 23, 2015 at 4:29 PM, sunil kalva <ka...@gmail.com>
> > wrote:
> >
> > > Hi
> > > We have a 10 node kafka cluster, and we have created a topic with
> > > replication factor "3" . It was running fine for 10 days, after that
> the
> > > replication factor is increased to "8" with out running any command to
> > > increase the replication. We have verified in server logs and also
> > > confirmed with the team that we haven't ran the command to increase.
> Once
> > > the replication factor is increased to "8", topic also stopped taking
> > > traffic.
> > > Did someone faced similar issue, or its known issue ?
> > >
> > > t
> > > SunilKalva
> > >
> >
>
--
---------------------------------------------------------
"There are only 10 types of people in the world: Those who understand
binary, and those who don't"
Re: replication factor increased automatically !
Posted by sunil kalva <ka...@gmail.com>.
Topic: testkafka Partition: 0 Leader: 5 Replicas: 5,10,1,9,2,7,3,8 Isr:
5,10,1,9,7,3,8
Topic: testkafka Partition: 1 Leader: 10 Replicas: 10,1,6,9,2,3,8,4 Isr:
10,6,9,3,8,4
Topic: testkafka Partition: 2 Leader: 4 Replicas: 5,10,1,9,2,7,3,4 Isr: 10,4
Topic: testkafka Partition: 3 Leader: 5 Replicas: 5,10,1,6,2,3,8,4 Isr: 5,1
Topic: testkafka Partition: 4 Leader: 5 Replicas: 5,1,6,9,2,7,3,4 Isr:
5,1,6,9,7,3,4
Topic: testkafka Partition: 5 Leader: 7 Replicas: 5,10,6,2,7,3,8,4 Isr:
7,3,4
Topic: testkafka Partition: 6 Leader: 8 Replicas: 5,1,6,9,7,3,8,4 Isr: 8,4
Topic: testkafka Partition: 7 Leader: 9 Replicas: 5,10,6,9,2,7,8,4 Isr: 9,6
Topic: testkafka Partition: 8 Leader: 10 Replicas: 5,10,1,6,9,7,3,8 Isr:
10,7
Topic: testkafka Partition: 9 Leader: 7 Replicas: 10,1,6,9,2,7,8,4 Isr: 8,7
On Mon, Nov 23, 2015 at 2:23 PM, Raju Bairishetti <ra...@apache.org> wrote:
> Hello Sunil,
> Are you able to see 8 replicas(i.e. in ISR) for all the partitions?
>
> Can you paste the output of *--describe *topic command?
> kafka-topics --zookeeper <zk-string> *-describe* <topic name>
>
> On Mon, Nov 23, 2015 at 4:29 PM, sunil kalva <ka...@gmail.com>
> wrote:
>
> > Hi
> > We have a 10 node kafka cluster, and we have created a topic with
> > replication factor "3" . It was running fine for 10 days, after that the
> > replication factor is increased to "8" with out running any command to
> > increase the replication. We have verified in server logs and also
> > confirmed with the team that we haven't ran the command to increase. Once
> > the replication factor is increased to "8", topic also stopped taking
> > traffic.
> > Did someone faced similar issue, or its known issue ?
> >
> > t
> > SunilKalva
> >
>
Re: replication factor increased automatically !
Posted by Raju Bairishetti <ra...@apache.org>.
Hello Sunil,
Are you able to see 8 replicas(i.e. in ISR) for all the partitions?
Can you paste the output of *--describe *topic command?
kafka-topics --zookeeper <zk-string> *-describe* <topic name>
On Mon, Nov 23, 2015 at 4:29 PM, sunil kalva <ka...@gmail.com> wrote:
> Hi
> We have a 10 node kafka cluster, and we have created a topic with
> replication factor "3" . It was running fine for 10 days, after that the
> replication factor is increased to "8" with out running any command to
> increase the replication. We have verified in server logs and also
> confirmed with the team that we haven't ran the command to increase. Once
> the replication factor is increased to "8", topic also stopped taking
> traffic.
> Did someone faced similar issue, or its known issue ?
>
> t
> SunilKalva
>