You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Zack Payton <zp...@gmail.com> on 2014/06/25 03:56:38 UTC
kafka.common.LeaderNotAvailableException
Hi all,
I have 3 zookeeper servers and 2 Kafka servers.
Running Kafka version 0.8.1.1.
Running zookeeper 3.3.5-cdh3u6.
>From the Kafka servers I can access the zookeeper servers on 2181.
>From one of the Kafka servers I can create a topic no problem:
[root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topics.sh --zookeeper
zookeeper1:2181,zookeeper2:2181,zookeeper3 --create --topic test1
--replication-factor 2 --partitions 2
Created topic "test1".
But, when I go to write to it I get the following:
[root@kafka1 kafka-0.8.1.1-src]# bin/kafka-console-producer.sh
--broker-list kafka1.queue.customer.sea1.ops.riotgames.com:9092,
kafka2.queue.customer.sea1.ops.riotgames.com:9092 --topic test1
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
TEST1
[2014-06-24 18:49:15,753] WARN Error while fetching metadata
[{TopicMetadata for topic test1 ->
No partition metadata for topic test1 due to
kafka.common.LeaderNotAvailableException}] for topic [test1]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:15,813] WARN Error while fetching metadata partition
0 leader:
none replicas: isr: isUnderReplicated: false for topic partition [test1,0]:
[class kafka.common.LeaderNotAvailableException]
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:15,814] WARN Error while fetching metadata partition
1 leader:
none replicas: isr: isUnderReplicated: false for topic partition [test1,1]:
[class kafka.common.LeaderNotAvailableException]
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:15,821] WARN Failed to collate messages by
topic,partition due to: No leader for any partition in topic test1
(kafka.producer.async.DefaultEventHandler)
[2014-06-24 18:49:15,933] WARN Error while fetching metadata
[{TopicMetadata for topic test1 ->
No partition metadata for topic test1 due to
kafka.common.LeaderNotAvailableException}] for topic [test1]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:15,933] WARN Failed to collate messages by
topic,partition due to: No leader for any partition in topic test1
(kafka.producer.async.DefaultEventHandler)
[2014-06-24 18:49:16,041] WARN Error while fetching metadata
[{TopicMetadata for topic test1 ->
No partition metadata for topic test1 due to
kafka.common.LeaderNotAvailableException}] for topic [test1]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:16,042] WARN Failed to collate messages by
topic,partition due to: No leader for any partition in topic test1
(kafka.producer.async.DefaultEventHandler)
[2014-06-24 18:49:16,148] WARN Error while fetching metadata partition
0 leader:
none replicas: isr: isUnderReplicated: false for topic partition [test1,0]:
[class kafka.common.LeaderNotAvailableException]
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:16,149] WARN Error while fetching metadata partition
1 leader:
none replicas: isr: isUnderReplicated: false for topic partition [test1,1]:
[class kafka.common.LeaderNotAvailableException]
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:16,149] WARN Failed to collate messages by
topic,partition due to: No leader for any partition in topic test1
(kafka.producer.async.DefaultEventHandler)
[2014-06-24 18:49:16,257] WARN Error while fetching metadata
[{TopicMetadata for topic test1 ->
No partition metadata for topic test1 due to
kafka.common.LeaderNotAvailableException}] for topic [test1]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2014-06-24 18:49:16,260] ERROR Failed to send requests for topics test1
with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2014-06-24 18:49:16,262] ERROR Error in handling batch of 1 events
(kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3
tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at
kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
at scala.collection.immutable.Stream.foreach(Stream.scala:254)
at
kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
Documentation on this issue is sparse. Any ideas?
Thanks,
Z
Re: kafka.common.LeaderNotAvailableException
Posted by Zack <zp...@gmail.com>.
This worked. Thank you.
> On Jun 25, 2014, at 7:04 AM, Joe Stein <jo...@stealth.ly> wrote:
>
> Zach, that error will occur if different brokers are advertising themselves
> in such a way where they are resolving to the same IP address. The
> advertised hostname is the hostname that will be given out to producers,
> consumers, and other brokers to connect to from the fetch meta data request.
>
> Check what you have set as the advertised hostname value in
> server.properties for both of your brokers... If that is not set then
> something is wrong with your server/network configurations and DNS is
> resolving each broker's hostname to the same IP as a result of each box
> having to default to java.net.InetAddress.getCanonicalHostName() without
> that value set.
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
>> On Tue, Jun 24, 2014 at 11:24 PM, Zack Payton <zp...@gmail.com> wrote:
>>
>> server.log has a lot of these errors:
>> [2014-06-24 20:07:16,124] ERROR [KafkaApi-6] error when handling request
>> Name: FetchRequest; Version: 0; CorrelationId: 81138; ClientId:
>> ReplicaFetche
>> rThread-0-5; ReplicaId: 6; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo:
>> [test1,0] -> PartitionFetchInfo(0,1048576) (kafka.server.KafkaApis)
>> *kafka.common.KafkaException: Shouldn't set logEndOffset for replica 6
>> partition [test1,0] since it's local*
>> at kafka.cluster.Replica.logEndOffset_$eq(Replica.scala:46)
>> at
>>
>> kafka.cluster.Partition.updateLeaderHWAndMaybeExpandIsr(Partition.scala:236)
>> at
>>
>> kafka.server.ReplicaManager.recordFollowerPosition(ReplicaManager.scala:432)
>> at
>>
>> kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:460)
>> at
>>
>> kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:458)
>> at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
>> at
>> kafka.server.KafkaApis.maybeUpdatePartitionHw(KafkaApis.scala:458)
>> at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:424)
>> at kafka.server.KafkaApis.handle(KafkaApis.scala:186)
>> at
>> kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42)
>> at java.lang.Thread.run(Thread.java:701)
>>
>>
>>
>>> On Tue, Jun 24, 2014 at 8:10 PM, Joe Stein <jo...@stealth.ly> wrote:
>>>
>>> Are there any errors in the broker's logs?
>>>
>>> /*******************************************
>>> Joe Stein
>>> Founder, Principal Consultant
>>> Big Data Open Source Security LLC
>>> http://www.stealth.ly
>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>> ********************************************/
>>>
>>>
>>>> On Tue, Jun 24, 2014 at 9:56 PM, Zack Payton <zp...@gmail.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I have 3 zookeeper servers and 2 Kafka servers.
>>>> Running Kafka version 0.8.1.1.
>>>> Running zookeeper 3.3.5-cdh3u6.
>>>> From the Kafka servers I can access the zookeeper servers on 2181.
>>>> From one of the Kafka servers I can create a topic no problem:
>>>> [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topics.sh --zookeeper
>>>> zookeeper1:2181,zookeeper2:2181,zookeeper3 --create --topic test1
>>>> --replication-factor 2 --partitions 2
>>>> Created topic "test1".
>>>>
>>>> But, when I go to write to it I get the following:
>>>> [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-console-producer.sh
>>>> --broker-list kafka1.queue.customer.sea1.ops.riotgames.com:9092,
>>>> kafka2.queue.customer.sea1.ops.riotgames.com:9092 --topic test1
>>>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>>>> SLF4J: Defaulting to no-operation (NOP) logger implementation
>>>> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
>>> further
>>>> details.
>>>> TEST1
>>>> [2014-06-24 18:49:15,753] WARN Error while fetching metadata
>>>> [{TopicMetadata for topic test1 ->
>>>> No partition metadata for topic test1 due to
>>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
>>>> kafka.common.LeaderNotAvailableException
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:15,813] WARN Error while fetching metadata partition
>>>> 0 leader:
>>>> none replicas: isr: isUnderReplicated: false for topic partition
>>> [test1,0]:
>>>> [class kafka.common.LeaderNotAvailableException]
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:15,814] WARN Error while fetching metadata partition
>>>> 1 leader:
>>>> none replicas: isr: isUnderReplicated: false for topic partition
>>> [test1,1]:
>>>> [class kafka.common.LeaderNotAvailableException]
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:15,821] WARN Failed to collate messages by
>>>> topic,partition due to: No leader for any partition in topic test1
>>>> (kafka.producer.async.DefaultEventHandler)
>>>> [2014-06-24 18:49:15,933] WARN Error while fetching metadata
>>>> [{TopicMetadata for topic test1 ->
>>>> No partition metadata for topic test1 due to
>>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
>>>> kafka.common.LeaderNotAvailableException
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:15,933] WARN Failed to collate messages by
>>>> topic,partition due to: No leader for any partition in topic test1
>>>> (kafka.producer.async.DefaultEventHandler)
>>>> [2014-06-24 18:49:16,041] WARN Error while fetching metadata
>>>> [{TopicMetadata for topic test1 ->
>>>> No partition metadata for topic test1 due to
>>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
>>>> kafka.common.LeaderNotAvailableException
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:16,042] WARN Failed to collate messages by
>>>> topic,partition due to: No leader for any partition in topic test1
>>>> (kafka.producer.async.DefaultEventHandler)
>>>> [2014-06-24 18:49:16,148] WARN Error while fetching metadata partition
>>>> 0 leader:
>>>> none replicas: isr: isUnderReplicated: false for topic partition
>>> [test1,0]:
>>>> [class kafka.common.LeaderNotAvailableException]
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:16,149] WARN Error while fetching metadata partition
>>>> 1 leader:
>>>> none replicas: isr: isUnderReplicated: false for topic partition
>>> [test1,1]:
>>>> [class kafka.common.LeaderNotAvailableException]
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:16,149] WARN Failed to collate messages by
>>>> topic,partition due to: No leader for any partition in topic test1
>>>> (kafka.producer.async.DefaultEventHandler)
>>>> [2014-06-24 18:49:16,257] WARN Error while fetching metadata
>>>> [{TopicMetadata for topic test1 ->
>>>> No partition metadata for topic test1 due to
>>>> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
>>>> kafka.common.LeaderNotAvailableException
>>>> (kafka.producer.BrokerPartitionInfo)
>>>> [2014-06-24 18:49:16,260] ERROR Failed to send requests for topics
>> test1
>>>> with correlation ids in [0,8]
>> (kafka.producer.async.DefaultEventHandler)
>>>> [2014-06-24 18:49:16,262] ERROR Error in handling batch of 1 events
>>>> (kafka.producer.async.ProducerSendThread)
>>>> kafka.common.FailedToSendMessageException: Failed to send messages
>> after
>>> 3
>>>> tries.
>>>> at
>> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
>>>> at
>> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
>>>> at
>> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
>>>> at
>> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
>>>> at scala.collection.immutable.Stream.foreach(Stream.scala:254)
>>>> at
>> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
>>>> at
>>> kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
>>>>
>>>>
>>>>
>>>> Documentation on this issue is sparse. Any ideas?
>>>>
>>>> Thanks,
>>>> Z
>>
Re: kafka.common.LeaderNotAvailableException
Posted by Joe Stein <jo...@stealth.ly>.
Zach, that error will occur if different brokers are advertising themselves
in such a way where they are resolving to the same IP address. The
advertised hostname is the hostname that will be given out to producers,
consumers, and other brokers to connect to from the fetch meta data request.
Check what you have set as the advertised hostname value in
server.properties for both of your brokers... If that is not set then
something is wrong with your server/network configurations and DNS is
resolving each broker's hostname to the same IP as a result of each box
having to default to java.net.InetAddress.getCanonicalHostName() without
that value set.
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Tue, Jun 24, 2014 at 11:24 PM, Zack Payton <zp...@gmail.com> wrote:
> server.log has a lot of these errors:
> [2014-06-24 20:07:16,124] ERROR [KafkaApi-6] error when handling request
> Name: FetchRequest; Version: 0; CorrelationId: 81138; ClientId:
> ReplicaFetche
> rThread-0-5; ReplicaId: 6; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo:
> [test1,0] -> PartitionFetchInfo(0,1048576) (kafka.server.KafkaApis)
> *kafka.common.KafkaException: Shouldn't set logEndOffset for replica 6
> partition [test1,0] since it's local*
> at kafka.cluster.Replica.logEndOffset_$eq(Replica.scala:46)
> at
>
> kafka.cluster.Partition.updateLeaderHWAndMaybeExpandIsr(Partition.scala:236)
> at
>
> kafka.server.ReplicaManager.recordFollowerPosition(ReplicaManager.scala:432)
> at
>
> kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:460)
> at
>
> kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:458)
> at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
> at
> kafka.server.KafkaApis.maybeUpdatePartitionHw(KafkaApis.scala:458)
> at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:424)
> at kafka.server.KafkaApis.handle(KafkaApis.scala:186)
> at
> kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42)
> at java.lang.Thread.run(Thread.java:701)
>
>
>
> On Tue, Jun 24, 2014 at 8:10 PM, Joe Stein <jo...@stealth.ly> wrote:
>
> > Are there any errors in the broker's logs?
> >
> > /*******************************************
> > Joe Stein
> > Founder, Principal Consultant
> > Big Data Open Source Security LLC
> > http://www.stealth.ly
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
> >
> > On Tue, Jun 24, 2014 at 9:56 PM, Zack Payton <zp...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > I have 3 zookeeper servers and 2 Kafka servers.
> > > Running Kafka version 0.8.1.1.
> > > Running zookeeper 3.3.5-cdh3u6.
> > > From the Kafka servers I can access the zookeeper servers on 2181.
> > > From one of the Kafka servers I can create a topic no problem:
> > > [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topics.sh --zookeeper
> > > zookeeper1:2181,zookeeper2:2181,zookeeper3 --create --topic test1
> > > --replication-factor 2 --partitions 2
> > > Created topic "test1".
> > >
> > > But, when I go to write to it I get the following:
> > > [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-console-producer.sh
> > > --broker-list kafka1.queue.customer.sea1.ops.riotgames.com:9092,
> > > kafka2.queue.customer.sea1.ops.riotgames.com:9092 --topic test1
> > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> > further
> > > details.
> > > TEST1
> > > [2014-06-24 18:49:15,753] WARN Error while fetching metadata
> > > [{TopicMetadata for topic test1 ->
> > > No partition metadata for topic test1 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > > kafka.common.LeaderNotAvailableException
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:15,813] WARN Error while fetching metadata partition
> > > 0 leader:
> > > none replicas: isr: isUnderReplicated: false for topic partition
> > [test1,0]:
> > > [class kafka.common.LeaderNotAvailableException]
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:15,814] WARN Error while fetching metadata partition
> > > 1 leader:
> > > none replicas: isr: isUnderReplicated: false for topic partition
> > [test1,1]:
> > > [class kafka.common.LeaderNotAvailableException]
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:15,821] WARN Failed to collate messages by
> > > topic,partition due to: No leader for any partition in topic test1
> > > (kafka.producer.async.DefaultEventHandler)
> > > [2014-06-24 18:49:15,933] WARN Error while fetching metadata
> > > [{TopicMetadata for topic test1 ->
> > > No partition metadata for topic test1 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > > kafka.common.LeaderNotAvailableException
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:15,933] WARN Failed to collate messages by
> > > topic,partition due to: No leader for any partition in topic test1
> > > (kafka.producer.async.DefaultEventHandler)
> > > [2014-06-24 18:49:16,041] WARN Error while fetching metadata
> > > [{TopicMetadata for topic test1 ->
> > > No partition metadata for topic test1 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > > kafka.common.LeaderNotAvailableException
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:16,042] WARN Failed to collate messages by
> > > topic,partition due to: No leader for any partition in topic test1
> > > (kafka.producer.async.DefaultEventHandler)
> > > [2014-06-24 18:49:16,148] WARN Error while fetching metadata partition
> > > 0 leader:
> > > none replicas: isr: isUnderReplicated: false for topic partition
> > [test1,0]:
> > > [class kafka.common.LeaderNotAvailableException]
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:16,149] WARN Error while fetching metadata partition
> > > 1 leader:
> > > none replicas: isr: isUnderReplicated: false for topic partition
> > [test1,1]:
> > > [class kafka.common.LeaderNotAvailableException]
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:16,149] WARN Failed to collate messages by
> > > topic,partition due to: No leader for any partition in topic test1
> > > (kafka.producer.async.DefaultEventHandler)
> > > [2014-06-24 18:49:16,257] WARN Error while fetching metadata
> > > [{TopicMetadata for topic test1 ->
> > > No partition metadata for topic test1 due to
> > > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > > kafka.common.LeaderNotAvailableException
> > > (kafka.producer.BrokerPartitionInfo)
> > > [2014-06-24 18:49:16,260] ERROR Failed to send requests for topics
> test1
> > > with correlation ids in [0,8]
> (kafka.producer.async.DefaultEventHandler)
> > > [2014-06-24 18:49:16,262] ERROR Error in handling batch of 1 events
> > > (kafka.producer.async.ProducerSendThread)
> > > kafka.common.FailedToSendMessageException: Failed to send messages
> after
> > 3
> > > tries.
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> > > at
> > >
> > >
> >
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
> > > at
> > >
> > >
> >
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
> > > at
> > >
> > >
> >
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
> > > at scala.collection.immutable.Stream.foreach(Stream.scala:254)
> > > at
> > >
> > >
> >
> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
> > > at
> > kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
> > >
> > >
> > >
> > > Documentation on this issue is sparse. Any ideas?
> > >
> > > Thanks,
> > > Z
> > >
> >
>
Re: kafka.common.LeaderNotAvailableException
Posted by Zack Payton <zp...@gmail.com>.
server.log has a lot of these errors:
[2014-06-24 20:07:16,124] ERROR [KafkaApi-6] error when handling request
Name: FetchRequest; Version: 0; CorrelationId: 81138; ClientId:
ReplicaFetche
rThread-0-5; ReplicaId: 6; MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo:
[test1,0] -> PartitionFetchInfo(0,1048576) (kafka.server.KafkaApis)
*kafka.common.KafkaException: Shouldn't set logEndOffset for replica 6
partition [test1,0] since it's local*
at kafka.cluster.Replica.logEndOffset_$eq(Replica.scala:46)
at
kafka.cluster.Partition.updateLeaderHWAndMaybeExpandIsr(Partition.scala:236)
at
kafka.server.ReplicaManager.recordFollowerPosition(ReplicaManager.scala:432)
at
kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:460)
at
kafka.server.KafkaApis$$anonfun$maybeUpdatePartitionHw$2.apply(KafkaApis.scala:458)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
at
kafka.server.KafkaApis.maybeUpdatePartitionHw(KafkaApis.scala:458)
at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:424)
at kafka.server.KafkaApis.handle(KafkaApis.scala:186)
at
kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42)
at java.lang.Thread.run(Thread.java:701)
On Tue, Jun 24, 2014 at 8:10 PM, Joe Stein <jo...@stealth.ly> wrote:
> Are there any errors in the broker's logs?
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
> On Tue, Jun 24, 2014 at 9:56 PM, Zack Payton <zp...@gmail.com> wrote:
>
> > Hi all,
> >
> > I have 3 zookeeper servers and 2 Kafka servers.
> > Running Kafka version 0.8.1.1.
> > Running zookeeper 3.3.5-cdh3u6.
> > From the Kafka servers I can access the zookeeper servers on 2181.
> > From one of the Kafka servers I can create a topic no problem:
> > [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topics.sh --zookeeper
> > zookeeper1:2181,zookeeper2:2181,zookeeper3 --create --topic test1
> > --replication-factor 2 --partitions 2
> > Created topic "test1".
> >
> > But, when I go to write to it I get the following:
> > [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-console-producer.sh
> > --broker-list kafka1.queue.customer.sea1.ops.riotgames.com:9092,
> > kafka2.queue.customer.sea1.ops.riotgames.com:9092 --topic test1
> > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > SLF4J: Defaulting to no-operation (NOP) logger implementation
> > SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> further
> > details.
> > TEST1
> > [2014-06-24 18:49:15,753] WARN Error while fetching metadata
> > [{TopicMetadata for topic test1 ->
> > No partition metadata for topic test1 due to
> > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:15,813] WARN Error while fetching metadata partition
> > 0 leader:
> > none replicas: isr: isUnderReplicated: false for topic partition
> [test1,0]:
> > [class kafka.common.LeaderNotAvailableException]
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:15,814] WARN Error while fetching metadata partition
> > 1 leader:
> > none replicas: isr: isUnderReplicated: false for topic partition
> [test1,1]:
> > [class kafka.common.LeaderNotAvailableException]
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:15,821] WARN Failed to collate messages by
> > topic,partition due to: No leader for any partition in topic test1
> > (kafka.producer.async.DefaultEventHandler)
> > [2014-06-24 18:49:15,933] WARN Error while fetching metadata
> > [{TopicMetadata for topic test1 ->
> > No partition metadata for topic test1 due to
> > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:15,933] WARN Failed to collate messages by
> > topic,partition due to: No leader for any partition in topic test1
> > (kafka.producer.async.DefaultEventHandler)
> > [2014-06-24 18:49:16,041] WARN Error while fetching metadata
> > [{TopicMetadata for topic test1 ->
> > No partition metadata for topic test1 due to
> > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:16,042] WARN Failed to collate messages by
> > topic,partition due to: No leader for any partition in topic test1
> > (kafka.producer.async.DefaultEventHandler)
> > [2014-06-24 18:49:16,148] WARN Error while fetching metadata partition
> > 0 leader:
> > none replicas: isr: isUnderReplicated: false for topic partition
> [test1,0]:
> > [class kafka.common.LeaderNotAvailableException]
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:16,149] WARN Error while fetching metadata partition
> > 1 leader:
> > none replicas: isr: isUnderReplicated: false for topic partition
> [test1,1]:
> > [class kafka.common.LeaderNotAvailableException]
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:16,149] WARN Failed to collate messages by
> > topic,partition due to: No leader for any partition in topic test1
> > (kafka.producer.async.DefaultEventHandler)
> > [2014-06-24 18:49:16,257] WARN Error while fetching metadata
> > [{TopicMetadata for topic test1 ->
> > No partition metadata for topic test1 due to
> > kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2014-06-24 18:49:16,260] ERROR Failed to send requests for topics test1
> > with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
> > [2014-06-24 18:49:16,262] ERROR Error in handling batch of 1 events
> > (kafka.producer.async.ProducerSendThread)
> > kafka.common.FailedToSendMessageException: Failed to send messages after
> 3
> > tries.
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
> > at scala.collection.immutable.Stream.foreach(Stream.scala:254)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
> > at
> kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
> >
> >
> >
> > Documentation on this issue is sparse. Any ideas?
> >
> > Thanks,
> > Z
> >
>
Re: kafka.common.LeaderNotAvailableException
Posted by Joe Stein <jo...@stealth.ly>.
Are there any errors in the broker's logs?
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Tue, Jun 24, 2014 at 9:56 PM, Zack Payton <zp...@gmail.com> wrote:
> Hi all,
>
> I have 3 zookeeper servers and 2 Kafka servers.
> Running Kafka version 0.8.1.1.
> Running zookeeper 3.3.5-cdh3u6.
> From the Kafka servers I can access the zookeeper servers on 2181.
> From one of the Kafka servers I can create a topic no problem:
> [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topics.sh --zookeeper
> zookeeper1:2181,zookeeper2:2181,zookeeper3 --create --topic test1
> --replication-factor 2 --partitions 2
> Created topic "test1".
>
> But, when I go to write to it I get the following:
> [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-console-producer.sh
> --broker-list kafka1.queue.customer.sea1.ops.riotgames.com:9092,
> kafka2.queue.customer.sea1.ops.riotgames.com:9092 --topic test1
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> TEST1
> [2014-06-24 18:49:15,753] WARN Error while fetching metadata
> [{TopicMetadata for topic test1 ->
> No partition metadata for topic test1 due to
> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:15,813] WARN Error while fetching metadata partition
> 0 leader:
> none replicas: isr: isUnderReplicated: false for topic partition [test1,0]:
> [class kafka.common.LeaderNotAvailableException]
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:15,814] WARN Error while fetching metadata partition
> 1 leader:
> none replicas: isr: isUnderReplicated: false for topic partition [test1,1]:
> [class kafka.common.LeaderNotAvailableException]
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:15,821] WARN Failed to collate messages by
> topic,partition due to: No leader for any partition in topic test1
> (kafka.producer.async.DefaultEventHandler)
> [2014-06-24 18:49:15,933] WARN Error while fetching metadata
> [{TopicMetadata for topic test1 ->
> No partition metadata for topic test1 due to
> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:15,933] WARN Failed to collate messages by
> topic,partition due to: No leader for any partition in topic test1
> (kafka.producer.async.DefaultEventHandler)
> [2014-06-24 18:49:16,041] WARN Error while fetching metadata
> [{TopicMetadata for topic test1 ->
> No partition metadata for topic test1 due to
> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:16,042] WARN Failed to collate messages by
> topic,partition due to: No leader for any partition in topic test1
> (kafka.producer.async.DefaultEventHandler)
> [2014-06-24 18:49:16,148] WARN Error while fetching metadata partition
> 0 leader:
> none replicas: isr: isUnderReplicated: false for topic partition [test1,0]:
> [class kafka.common.LeaderNotAvailableException]
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:16,149] WARN Error while fetching metadata partition
> 1 leader:
> none replicas: isr: isUnderReplicated: false for topic partition [test1,1]:
> [class kafka.common.LeaderNotAvailableException]
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:16,149] WARN Failed to collate messages by
> topic,partition due to: No leader for any partition in topic test1
> (kafka.producer.async.DefaultEventHandler)
> [2014-06-24 18:49:16,257] WARN Error while fetching metadata
> [{TopicMetadata for topic test1 ->
> No partition metadata for topic test1 due to
> kafka.common.LeaderNotAvailableException}] for topic [test1]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2014-06-24 18:49:16,260] ERROR Failed to send requests for topics test1
> with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
> [2014-06-24 18:49:16,262] ERROR Error in handling batch of 1 events
> (kafka.producer.async.ProducerSendThread)
> kafka.common.FailedToSendMessageException: Failed to send messages after 3
> tries.
> at
>
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> at
>
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
> at
>
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
> at
>
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
> at scala.collection.immutable.Stream.foreach(Stream.scala:254)
> at
>
> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
> at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
>
>
>
> Documentation on this issue is sparse. Any ideas?
>
> Thanks,
> Z
>