You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sa Li <sa...@gmail.com> on 2014/12/22 23:46:14 UTC
leader and isr were not set when create the topic
Hi, All
I created a topic with 3 replications and 6 partitions, but when I check
this topic, seems there is no leader and isr were set for this topic, see
bin/kafka-topics.sh --create --zookeeper 10.100.98.100:2181
--replication-factor 3 --partitions 6 --topic perf_producer_p6_test
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Created topic "perf_producer_p6_test".
root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
10.100.98.100:2181 --topic perf_producer_p6_test
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Topic:perf_producer_p6_test PartitionCount:6
ReplicationFactor:3 Configs:
Topic: perf_producer_p6_test Partition: 0 Leader: none
Replicas: 100,101,102 Isr:
Topic: perf_producer_p6_test Partition: 1 Leader: none
Replicas: 101,102,100 Isr:
Topic: perf_producer_p6_test Partition: 2 Leader: none
Replicas: 102,100,101 Isr:
Topic: perf_producer_p6_test Partition: 3 Leader: none
Replicas: 100,102,101 Isr:
Topic: perf_producer_p6_test Partition: 4 Leader: none
Replicas: 101,100,102 Isr:
Topic: perf_producer_p6_test Partition: 5 Leader: none
Replicas: 102,101,100 Isr:
Is there a way to specifically set leader and isr in command line, it is
strange when I create the topic with 5 partitions, it has leader and isr:
root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
10.100.98.100:2181 --topic perf_producer_p5_test
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Topic:perf_producer_p5_test PartitionCount:5
ReplicationFactor:3 Configs:
Topic: perf_producer_p5_test Partition: 0 Leader: 102
Replicas: 102,100,101 Isr: 102,100,101
Topic: perf_producer_p5_test Partition: 1 Leader: 102
Replicas: 100,101,102 Isr: 102,101
Topic: perf_producer_p5_test Partition: 2 Leader: 101
Replicas: 101,102,100 Isr: 101,102,100
Topic: perf_producer_p5_test Partition: 3 Leader: 102
Replicas: 102,101,100 Isr: 102,101,100
Topic: perf_producer_p5_test Partition: 4 Leader: 102
Replicas: 100,102,101 Isr: 102,101
Any ideas?
thanks
--
Alec Li
Re: leader and isr were not set when create the topic
Posted by Neha Narkhede <ne...@confluent.io>.
There is possibly some error in your broker logs. Can you check if you see
any and send it around?
On Mon, Dec 22, 2014 at 2:46 PM, Sa Li <sa...@gmail.com> wrote:
> Hi, All
>
> I created a topic with 3 replications and 6 partitions, but when I check
> this topic, seems there is no leader and isr were set for this topic, see
>
> bin/kafka-topics.sh --create --zookeeper 10.100.98.100:2181
> --replication-factor 3 --partitions 6 --topic perf_producer_p6_test
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Created topic "perf_producer_p6_test".
>
> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
> 10.100.98.100:2181 --topic perf_producer_p6_test
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Topic:perf_producer_p6_test PartitionCount:6
> ReplicationFactor:3 Configs:
> Topic: perf_producer_p6_test Partition: 0 Leader: none
> Replicas: 100,101,102 Isr:
> Topic: perf_producer_p6_test Partition: 1 Leader: none
> Replicas: 101,102,100 Isr:
> Topic: perf_producer_p6_test Partition: 2 Leader: none
> Replicas: 102,100,101 Isr:
> Topic: perf_producer_p6_test Partition: 3 Leader: none
> Replicas: 100,102,101 Isr:
> Topic: perf_producer_p6_test Partition: 4 Leader: none
> Replicas: 101,100,102 Isr:
> Topic: perf_producer_p6_test Partition: 5 Leader: none
> Replicas: 102,101,100 Isr:
>
> Is there a way to specifically set leader and isr in command line, it is
> strange when I create the topic with 5 partitions, it has leader and isr:
> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
> 10.100.98.100:2181 --topic perf_producer_p5_test
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Topic:perf_producer_p5_test PartitionCount:5
> ReplicationFactor:3 Configs:
> Topic: perf_producer_p5_test Partition: 0 Leader: 102
> Replicas: 102,100,101 Isr: 102,100,101
> Topic: perf_producer_p5_test Partition: 1 Leader: 102
> Replicas: 100,101,102 Isr: 102,101
> Topic: perf_producer_p5_test Partition: 2 Leader: 101
> Replicas: 101,102,100 Isr: 101,102,100
> Topic: perf_producer_p5_test Partition: 3 Leader: 102
> Replicas: 102,101,100 Isr: 102,101,100
> Topic: perf_producer_p5_test Partition: 4 Leader: 102
> Replicas: 100,102,101 Isr: 102,101
>
>
> Any ideas?
>
> thanks
>
> --
>
> Alec Li
>
--
Thanks,
Neha
Re: leader and isr were not set when create the topic
Posted by Neha Narkhede <ne...@confluent.io>.
It seems that somehow the follower lost its highwatermark checkpoint file.
Can you share the steps to reproduce this along with the Kafka versions you
are using?
On Mon, Dec 22, 2014 at 4:17 PM, Sa Li <sa...@gmail.com> wrote:
> I have three nodes: 100, 101, and 102
>
> When I restart all of them, seems now everything is ok, but I would like to
> paste the error messages I got from server.log from each node, see if you
> can help to understand what is the problem.
>
> on node 100
> [2014-12-23 00:04:39,401] ERROR [KafkaApi-100] Error when processing fetch
> request for partition [perf_producer_p8_test,7] offset 125000 from follower
> with correlation id 0 (kafka.server.KafkaApis)
> kafka.common.OffsetOutOfRangeException: Request for offset 125000 but we
> only have log segments in the range 0 to 0.
> at kafka.log.Log.read(Log.scala:380)
> at
>
> kafka.server.KafkaApis.kafka$server$KafkaApis$$readMessageSet(KafkaApis.scala:530)
> at
>
> kafka.server.KafkaApis$$anonfun$kafka$server$KafkaApis$$readMessageSets$1.apply(KafkaApis.scala:476)
>
> at
>
> kafka.server.KafkaApis$$anonfun$kafka$server$KafkaApis$$readMessageSets$1.apply(KafkaApis.scala:471)
>
> at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> at
>
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at
> scala.collection.immutable.Map$Map3.foreach(Map.scala:154)
> at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
> at
> scala.collection.AbstractTraversable.map(Traversable.scala:105)
> ..
> ..
>
>
> in Node 101 and 102
> [2014-12-23 00:04:39,440] ERROR [ReplicaFetcherThread-0-100], Current
> offset 1 25000 for partition [perf_producer_p8_test,1] out of range; reset
> offset to 0 (kafka.server.ReplicaFetcherThread)
> [2014-12-23 00:04:39,442] INFO Truncating log perf_producer_p8_test-7 to
> offset 0. (kafka.log.Log)
> [2014-12-23 00:04:39,452] WARN [ReplicaFetcherThread-0-100], Replica 102
> for partition [perf_producer_p8_test,7] reset its fetch offset to current
> leader 100's latest offset 0 (kafka.server.ReplicaFetcherThread)
>
>
>
>
>
>
> On Mon, Dec 22, 2014 at 3:55 PM, Sa Li <sa...@gmail.com> wrote:
> >
> > Hello, Neha
> >
> > This is the error from server.log
> >
> > [2014-12-22 23:53:25,663] WARN [KafkaApi-100] Fetch request with
> > correlation id 1227732 from client ReplicaFetcherThread-0-100 on
> partition
> > [perf_producer_p8_test,1] failed due to Leader not local for partition
> > [perf_producer_p8_test,1] on broker 100 (kafka.server.KafkaApis)
> >
> >
> > On Mon, Dec 22, 2014 at 3:50 PM, Sa Li <sa...@gmail.com> wrote:
> >>
> >> I restart the kafka server, it is the same thing, sometime nothing
> listed
> >> on ISR, leader, I checked the state-change log
> >>
> >> [2014-12-22 23:46:38,164] TRACE Broker 100 cached leader info
> >>
> (LeaderAndIsrInfo:(Leader:101,ISR:101,102,100,LeaderEpoch:0,ControllerEpoch:4),ReplicationFactor:3),AllReplicas:101,102,100)
> >> for partition [perf_producer_p8_test,1] in response to UpdateMetadata
> >> request sent by controller 101 epoch 4 with correlation id 138
> >> (state.change.logger)
> >>
> >>
> >>
> >> On Mon, Dec 22, 2014 at 2:46 PM, Sa Li <sa...@gmail.com> wrote:
> >>
> >>> Hi, All
> >>>
> >>> I created a topic with 3 replications and 6 partitions, but when I
> check
> >>> this topic, seems there is no leader and isr were set for this topic,
> see
> >>>
> >>> bin/kafka-topics.sh --create --zookeeper 10.100.98.100:2181
> >>> --replication-factor 3 --partitions 6 --topic perf_producer_p6_test
> >>> SLF4J: Class path contains multiple SLF4J bindings.
> >>> SLF4J: Found binding in
> >>>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >>> SLF4J: Found binding in
> >>>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> >>> explanation.
> >>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >>> Created topic "perf_producer_p6_test".
> >>>
> >>> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
> >>> 10.100.98.100:2181 --topic perf_producer_p6_test
> >>> SLF4J: Class path contains multiple SLF4J bindings.
> >>> SLF4J: Found binding in
> >>>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >>> SLF4J: Found binding in
> >>>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> >>> explanation.
> >>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >>> Topic:perf_producer_p6_test PartitionCount:6
> >>> ReplicationFactor:3 Configs:
> >>> Topic: perf_producer_p6_test Partition: 0 Leader: none
> >>> Replicas: 100,101,102 Isr:
> >>> Topic: perf_producer_p6_test Partition: 1 Leader: none
> >>> Replicas: 101,102,100 Isr:
> >>> Topic: perf_producer_p6_test Partition: 2 Leader: none
> >>> Replicas: 102,100,101 Isr:
> >>> Topic: perf_producer_p6_test Partition: 3 Leader: none
> >>> Replicas: 100,102,101 Isr:
> >>> Topic: perf_producer_p6_test Partition: 4 Leader: none
> >>> Replicas: 101,100,102 Isr:
> >>> Topic: perf_producer_p6_test Partition: 5 Leader: none
> >>> Replicas: 102,101,100 Isr:
> >>>
> >>> Is there a way to specifically set leader and isr in command line, it
> is
> >>> strange when I create the topic with 5 partitions, it has leader and
> isr:
> >>> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
> >>> 10.100.98.100:2181 --topic perf_producer_p5_test
> >>> SLF4J: Class path contains multiple SLF4J bindings.
> >>> SLF4J: Found binding in
> >>>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >>> SLF4J: Found binding in
> >>>
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> >>> explanation.
> >>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >>> Topic:perf_producer_p5_test PartitionCount:5
> >>> ReplicationFactor:3 Configs:
> >>> Topic: perf_producer_p5_test Partition: 0 Leader: 102
> >>> Replicas: 102,100,101 Isr: 102,100,101
> >>> Topic: perf_producer_p5_test Partition: 1 Leader: 102
> >>> Replicas: 100,101,102 Isr: 102,101
> >>> Topic: perf_producer_p5_test Partition: 2 Leader: 101
> >>> Replicas: 101,102,100 Isr: 101,102,100
> >>> Topic: perf_producer_p5_test Partition: 3 Leader: 102
> >>> Replicas: 102,101,100 Isr: 102,101,100
> >>> Topic: perf_producer_p5_test Partition: 4 Leader: 102
> >>> Replicas: 100,102,101 Isr: 102,101
> >>>
> >>>
> >>> Any ideas?
> >>>
> >>> thanks
> >>>
> >>> --
> >>>
> >>> Alec Li
> >>>
> >>
> >>
> >> --
> >>
> >> Alec Li
> >>
> >
> >
> > --
> >
> > Alec Li
> >
>
>
> --
>
> Alec Li
>
--
Thanks,
Neha
Re: leader and isr were not set when create the topic
Posted by Sa Li <sa...@gmail.com>.
I have three nodes: 100, 101, and 102
When I restart all of them, seems now everything is ok, but I would like to
paste the error messages I got from server.log from each node, see if you
can help to understand what is the problem.
on node 100
[2014-12-23 00:04:39,401] ERROR [KafkaApi-100] Error when processing fetch
request for partition [perf_producer_p8_test,7] offset 125000 from follower
with correlation id 0 (kafka.server.KafkaApis)
kafka.common.OffsetOutOfRangeException: Request for offset 125000 but we
only have log segments in the range 0 to 0.
at kafka.log.Log.read(Log.scala:380)
at
kafka.server.KafkaApis.kafka$server$KafkaApis$$readMessageSet(KafkaApis.scala:530)
at
kafka.server.KafkaApis$$anonfun$kafka$server$KafkaApis$$readMessageSets$1.apply(KafkaApis.scala:476)
at
kafka.server.KafkaApis$$anonfun$kafka$server$KafkaApis$$readMessageSets$1.apply(KafkaApis.scala:471)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.immutable.Map$Map3.foreach(Map.scala:154)
at
scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at
scala.collection.AbstractTraversable.map(Traversable.scala:105)
..
..
in Node 101 and 102
[2014-12-23 00:04:39,440] ERROR [ReplicaFetcherThread-0-100], Current
offset 1 25000 for partition [perf_producer_p8_test,1] out of range; reset
offset to 0 (kafka.server.ReplicaFetcherThread)
[2014-12-23 00:04:39,442] INFO Truncating log perf_producer_p8_test-7 to
offset 0. (kafka.log.Log)
[2014-12-23 00:04:39,452] WARN [ReplicaFetcherThread-0-100], Replica 102
for partition [perf_producer_p8_test,7] reset its fetch offset to current
leader 100's latest offset 0 (kafka.server.ReplicaFetcherThread)
On Mon, Dec 22, 2014 at 3:55 PM, Sa Li <sa...@gmail.com> wrote:
>
> Hello, Neha
>
> This is the error from server.log
>
> [2014-12-22 23:53:25,663] WARN [KafkaApi-100] Fetch request with
> correlation id 1227732 from client ReplicaFetcherThread-0-100 on partition
> [perf_producer_p8_test,1] failed due to Leader not local for partition
> [perf_producer_p8_test,1] on broker 100 (kafka.server.KafkaApis)
>
>
> On Mon, Dec 22, 2014 at 3:50 PM, Sa Li <sa...@gmail.com> wrote:
>>
>> I restart the kafka server, it is the same thing, sometime nothing listed
>> on ISR, leader, I checked the state-change log
>>
>> [2014-12-22 23:46:38,164] TRACE Broker 100 cached leader info
>> (LeaderAndIsrInfo:(Leader:101,ISR:101,102,100,LeaderEpoch:0,ControllerEpoch:4),ReplicationFactor:3),AllReplicas:101,102,100)
>> for partition [perf_producer_p8_test,1] in response to UpdateMetadata
>> request sent by controller 101 epoch 4 with correlation id 138
>> (state.change.logger)
>>
>>
>>
>> On Mon, Dec 22, 2014 at 2:46 PM, Sa Li <sa...@gmail.com> wrote:
>>
>>> Hi, All
>>>
>>> I created a topic with 3 replications and 6 partitions, but when I check
>>> this topic, seems there is no leader and isr were set for this topic, see
>>>
>>> bin/kafka-topics.sh --create --zookeeper 10.100.98.100:2181
>>> --replication-factor 3 --partitions 6 --topic perf_producer_p6_test
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in
>>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in
>>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> Created topic "perf_producer_p6_test".
>>>
>>> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
>>> 10.100.98.100:2181 --topic perf_producer_p6_test
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in
>>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in
>>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> Topic:perf_producer_p6_test PartitionCount:6
>>> ReplicationFactor:3 Configs:
>>> Topic: perf_producer_p6_test Partition: 0 Leader: none
>>> Replicas: 100,101,102 Isr:
>>> Topic: perf_producer_p6_test Partition: 1 Leader: none
>>> Replicas: 101,102,100 Isr:
>>> Topic: perf_producer_p6_test Partition: 2 Leader: none
>>> Replicas: 102,100,101 Isr:
>>> Topic: perf_producer_p6_test Partition: 3 Leader: none
>>> Replicas: 100,102,101 Isr:
>>> Topic: perf_producer_p6_test Partition: 4 Leader: none
>>> Replicas: 101,100,102 Isr:
>>> Topic: perf_producer_p6_test Partition: 5 Leader: none
>>> Replicas: 102,101,100 Isr:
>>>
>>> Is there a way to specifically set leader and isr in command line, it is
>>> strange when I create the topic with 5 partitions, it has leader and isr:
>>> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
>>> 10.100.98.100:2181 --topic perf_producer_p5_test
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in
>>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in
>>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> Topic:perf_producer_p5_test PartitionCount:5
>>> ReplicationFactor:3 Configs:
>>> Topic: perf_producer_p5_test Partition: 0 Leader: 102
>>> Replicas: 102,100,101 Isr: 102,100,101
>>> Topic: perf_producer_p5_test Partition: 1 Leader: 102
>>> Replicas: 100,101,102 Isr: 102,101
>>> Topic: perf_producer_p5_test Partition: 2 Leader: 101
>>> Replicas: 101,102,100 Isr: 101,102,100
>>> Topic: perf_producer_p5_test Partition: 3 Leader: 102
>>> Replicas: 102,101,100 Isr: 102,101,100
>>> Topic: perf_producer_p5_test Partition: 4 Leader: 102
>>> Replicas: 100,102,101 Isr: 102,101
>>>
>>>
>>> Any ideas?
>>>
>>> thanks
>>>
>>> --
>>>
>>> Alec Li
>>>
>>
>>
>> --
>>
>> Alec Li
>>
>
>
> --
>
> Alec Li
>
--
Alec Li
Re: leader and isr were not set when create the topic
Posted by Sa Li <sa...@gmail.com>.
Hello, Neha
This is the error from server.log
[2014-12-22 23:53:25,663] WARN [KafkaApi-100] Fetch request with
correlation id 1227732 from client ReplicaFetcherThread-0-100 on partition
[perf_producer_p8_test,1] failed due to Leader not local for partition
[perf_producer_p8_test,1] on broker 100 (kafka.server.KafkaApis)
On Mon, Dec 22, 2014 at 3:50 PM, Sa Li <sa...@gmail.com> wrote:
>
> I restart the kafka server, it is the same thing, sometime nothing listed
> on ISR, leader, I checked the state-change log
>
> [2014-12-22 23:46:38,164] TRACE Broker 100 cached leader info
> (LeaderAndIsrInfo:(Leader:101,ISR:101,102,100,LeaderEpoch:0,ControllerEpoch:4),ReplicationFactor:3),AllReplicas:101,102,100)
> for partition [perf_producer_p8_test,1] in response to UpdateMetadata
> request sent by controller 101 epoch 4 with correlation id 138
> (state.change.logger)
>
>
>
> On Mon, Dec 22, 2014 at 2:46 PM, Sa Li <sa...@gmail.com> wrote:
>
>> Hi, All
>>
>> I created a topic with 3 replications and 6 partitions, but when I check
>> this topic, seems there is no leader and isr were set for this topic, see
>>
>> bin/kafka-topics.sh --create --zookeeper 10.100.98.100:2181
>> --replication-factor 3 --partitions 6 --topic perf_producer_p6_test
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> Created topic "perf_producer_p6_test".
>>
>> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
>> 10.100.98.100:2181 --topic perf_producer_p6_test
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> Topic:perf_producer_p6_test PartitionCount:6
>> ReplicationFactor:3 Configs:
>> Topic: perf_producer_p6_test Partition: 0 Leader: none
>> Replicas: 100,101,102 Isr:
>> Topic: perf_producer_p6_test Partition: 1 Leader: none
>> Replicas: 101,102,100 Isr:
>> Topic: perf_producer_p6_test Partition: 2 Leader: none
>> Replicas: 102,100,101 Isr:
>> Topic: perf_producer_p6_test Partition: 3 Leader: none
>> Replicas: 100,102,101 Isr:
>> Topic: perf_producer_p6_test Partition: 4 Leader: none
>> Replicas: 101,100,102 Isr:
>> Topic: perf_producer_p6_test Partition: 5 Leader: none
>> Replicas: 102,101,100 Isr:
>>
>> Is there a way to specifically set leader and isr in command line, it is
>> strange when I create the topic with 5 partitions, it has leader and isr:
>> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
>> 10.100.98.100:2181 --topic perf_producer_p5_test
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> Topic:perf_producer_p5_test PartitionCount:5
>> ReplicationFactor:3 Configs:
>> Topic: perf_producer_p5_test Partition: 0 Leader: 102
>> Replicas: 102,100,101 Isr: 102,100,101
>> Topic: perf_producer_p5_test Partition: 1 Leader: 102
>> Replicas: 100,101,102 Isr: 102,101
>> Topic: perf_producer_p5_test Partition: 2 Leader: 101
>> Replicas: 101,102,100 Isr: 101,102,100
>> Topic: perf_producer_p5_test Partition: 3 Leader: 102
>> Replicas: 102,101,100 Isr: 102,101,100
>> Topic: perf_producer_p5_test Partition: 4 Leader: 102
>> Replicas: 100,102,101 Isr: 102,101
>>
>>
>> Any ideas?
>>
>> thanks
>>
>> --
>>
>> Alec Li
>>
>
>
> --
>
> Alec Li
>
--
Alec Li
Re: leader and isr were not set when create the topic
Posted by Sa Li <sa...@gmail.com>.
I restart the kafka server, it is the same thing, sometime nothing listed
on ISR, leader, I checked the state-change log
[2014-12-22 23:46:38,164] TRACE Broker 100 cached leader info
(LeaderAndIsrInfo:(Leader:101,ISR:101,102,100,LeaderEpoch:0,ControllerEpoch:4),ReplicationFactor:3),AllReplicas:101,102,100)
for partition [perf_producer_p8_test,1] in response to UpdateMetadata
request sent by controller 101 epoch 4 with correlation id 138
(state.change.logger)
On Mon, Dec 22, 2014 at 2:46 PM, Sa Li <sa...@gmail.com> wrote:
>
> Hi, All
>
> I created a topic with 3 replications and 6 partitions, but when I check
> this topic, seems there is no leader and isr were set for this topic, see
>
> bin/kafka-topics.sh --create --zookeeper 10.100.98.100:2181
> --replication-factor 3 --partitions 6 --topic perf_producer_p6_test
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Created topic "perf_producer_p6_test".
>
> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
> 10.100.98.100:2181 --topic perf_producer_p6_test
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Topic:perf_producer_p6_test PartitionCount:6
> ReplicationFactor:3 Configs:
> Topic: perf_producer_p6_test Partition: 0 Leader: none
> Replicas: 100,101,102 Isr:
> Topic: perf_producer_p6_test Partition: 1 Leader: none
> Replicas: 101,102,100 Isr:
> Topic: perf_producer_p6_test Partition: 2 Leader: none
> Replicas: 102,100,101 Isr:
> Topic: perf_producer_p6_test Partition: 3 Leader: none
> Replicas: 100,102,101 Isr:
> Topic: perf_producer_p6_test Partition: 4 Leader: none
> Replicas: 101,100,102 Isr:
> Topic: perf_producer_p6_test Partition: 5 Leader: none
> Replicas: 102,101,100 Isr:
>
> Is there a way to specifically set leader and isr in command line, it is
> strange when I create the topic with 5 partitions, it has leader and isr:
> root@precise64:/etc/kafka# bin/kafka-topics.sh --describe --zookeeper
> 10.100.98.100:2181 --topic perf_producer_p5_test
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/etc/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Topic:perf_producer_p5_test PartitionCount:5
> ReplicationFactor:3 Configs:
> Topic: perf_producer_p5_test Partition: 0 Leader: 102
> Replicas: 102,100,101 Isr: 102,100,101
> Topic: perf_producer_p5_test Partition: 1 Leader: 102
> Replicas: 100,101,102 Isr: 102,101
> Topic: perf_producer_p5_test Partition: 2 Leader: 101
> Replicas: 101,102,100 Isr: 101,102,100
> Topic: perf_producer_p5_test Partition: 3 Leader: 102
> Replicas: 102,101,100 Isr: 102,101,100
> Topic: perf_producer_p5_test Partition: 4 Leader: 102
> Replicas: 100,102,101 Isr: 102,101
>
>
> Any ideas?
>
> thanks
>
> --
>
> Alec Li
>
--
Alec Li