You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Alex Melville <am...@g.hmc.edu> on 2015/02/08 01:38:51 UTC
Console Producer Throwing LeaderNotAvailableException Despite
Existing Leader for Partition
Howdy all,
I recently upgraded to Kafka 0.8.2.0 and am trying to verify that
everything still works as expected. I spin up two brokers, one zk instance,
and then create a topic using
kafka-topics.sh --create --zookeeper ad-0104:2181 --topic deleteme
--partitions 2 --replication-factor 1
Then I run --describe to check if the partitions have leaders. I get
kafka-topics.sh --describe --zookeeper ad-0104:2181 --topic deleteme
Topic:deleteme PartitionCount:2 ReplicationFactor:1 Configs:
Topic: deleteme Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: deleteme Partition: 1 Leader: 1 Replicas: 1 Isr: 1
Finally, I run the console producer
kafka-console-producer.sh --broker-list ad-0102:9092 --topic deleteme
I get the following warning
[2015-02-08 00:36:24,244] WARN Property topic is not valid
(kafka.utils.VerifiableProperties)
and then it waits for console input. When I try to send a message I get the
following list of error messages
[2015-02-08 00:37:04,735] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:04,751] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:04,752] ERROR Failed to collate messages by topic,
partition due to: Failed to fetch topic metadata for topic: deleteme
(kafka.producer.async.DefaultEventHandler)
[2015-02-08 00:37:04,859] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:04,863] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:04,863] ERROR Failed to collate messages by topic,
partition due to: Failed to fetch topic metadata for topic: deleteme
(kafka.producer.async.DefaultEventHandler)
[2015-02-08 00:37:04,968] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:04,974] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:04,974] ERROR Failed to collate messages by topic,
partition due to: Failed to fetch topic metadata for topic: deleteme
(kafka.producer.async.DefaultEventHandler)
[2015-02-08 00:37:05,079] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:05,084] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:05,084] ERROR Failed to collate messages by topic,
partition due to: Failed to fetch topic metadata for topic: deleteme
(kafka.producer.async.DefaultEventHandler)
[2015-02-08 00:37:05,189] WARN Error while fetching metadata
[{TopicMetadata for topic deleteme ->
No partition metadata for topic deleteme due to
kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-02-08 00:37:05,191] ERROR Failed to send requests for topics deleteme
with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2015-02-08 00:37:05,192] ERROR Error in handling batch of 1 events
(kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3
tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at
kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:526)
at
kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Clearly there's a leader, but it's throwing a LeaderNotAvailableException.
Any help here is appreciated.
Cheers,
Alex Melville
Re: Console Producer Throwing LeaderNotAvailableException Despite
Existing Leader for Partition
Posted by Alex Melville <am...@g.hmc.edu>.
Thanks Tao,
That fixed the problem. Console producer now correctly pushes to the topic
and console consumer can read the topic data.
-Alex
On Sun, Feb 8, 2015 at 1:47 AM, tao xiao <xi...@gmail.com> wrote:
> Alex,
>
> I got similar error before due to incorrect network binding of my laptop's
> wireless interface. You can try with setting advertised.host.name=kafka's
> server hostname in the server.properties and run it again.
>
> On Sun, Feb 8, 2015 at 8:38 AM, Alex Melville <am...@g.hmc.edu> wrote:
>
> > Howdy all,
> >
> > I recently upgraded to Kafka 0.8.2.0 and am trying to verify that
> > everything still works as expected. I spin up two brokers, one zk
> instance,
> > and then create a topic using
> >
> > kafka-topics.sh --create --zookeeper ad-0104:2181 --topic deleteme
> > --partitions 2 --replication-factor 1
> >
> > Then I run --describe to check if the partitions have leaders. I get
> >
> >
> > kafka-topics.sh --describe --zookeeper ad-0104:2181 --topic deleteme
> >
> > Topic:deleteme PartitionCount:2 ReplicationFactor:1 Configs:
> > Topic: deleteme Partition: 0 Leader: 0 Replicas: 0 Isr: 0
> > Topic: deleteme Partition: 1 Leader: 1 Replicas: 1 Isr: 1
> >
> >
> > Finally, I run the console producer
> >
> > kafka-console-producer.sh --broker-list ad-0102:9092 --topic deleteme
> >
> > I get the following warning
> >
> > [2015-02-08 00:36:24,244] WARN Property topic is not valid
> > (kafka.utils.VerifiableProperties)
> >
> > and then it waits for console input. When I try to send a message I get
> the
> > following list of error messages
> >
> > [2015-02-08 00:37:04,735] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:04,751] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:04,752] ERROR Failed to collate messages by topic,
> > partition due to: Failed to fetch topic metadata for topic: deleteme
> > (kafka.producer.async.DefaultEventHandler)
> > [2015-02-08 00:37:04,859] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:04,863] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:04,863] ERROR Failed to collate messages by topic,
> > partition due to: Failed to fetch topic metadata for topic: deleteme
> > (kafka.producer.async.DefaultEventHandler)
> > [2015-02-08 00:37:04,968] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:04,974] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:04,974] ERROR Failed to collate messages by topic,
> > partition due to: Failed to fetch topic metadata for topic: deleteme
> > (kafka.producer.async.DefaultEventHandler)
> > [2015-02-08 00:37:05,079] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:05,084] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:05,084] ERROR Failed to collate messages by topic,
> > partition due to: Failed to fetch topic metadata for topic: deleteme
> > (kafka.producer.async.DefaultEventHandler)
> > [2015-02-08 00:37:05,189] WARN Error while fetching metadata
> > [{TopicMetadata for topic deleteme ->
> > No partition metadata for topic deleteme due to
> > kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> > kafka.common.LeaderNotAvailableException
> > (kafka.producer.BrokerPartitionInfo)
> > [2015-02-08 00:37:05,191] ERROR Failed to send requests for topics
> deleteme
> > with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
> > [2015-02-08 00:37:05,192] ERROR Error in handling batch of 1 events
> > (kafka.producer.async.ProducerSendThread)
> > kafka.common.FailedToSendMessageException: Failed to send messages after
> 3
> > tries.
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
> > at scala.collection.immutable.Stream.foreach(Stream.scala:526)
> > at
> >
> >
> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
> > at
> kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
> >
> >
> >
> >
> >
> >
> > Clearly there's a leader, but it's throwing a
> LeaderNotAvailableException.
> > Any help here is appreciated.
> >
> >
> > Cheers,
> >
> > Alex Melville
> >
>
>
>
> --
> Regards,
> Tao
>
Re: Console Producer Throwing LeaderNotAvailableException Despite
Existing Leader for Partition
Posted by tao xiao <xi...@gmail.com>.
Alex,
I got similar error before due to incorrect network binding of my laptop's
wireless interface. You can try with setting advertised.host.name=kafka's
server hostname in the server.properties and run it again.
On Sun, Feb 8, 2015 at 8:38 AM, Alex Melville <am...@g.hmc.edu> wrote:
> Howdy all,
>
> I recently upgraded to Kafka 0.8.2.0 and am trying to verify that
> everything still works as expected. I spin up two brokers, one zk instance,
> and then create a topic using
>
> kafka-topics.sh --create --zookeeper ad-0104:2181 --topic deleteme
> --partitions 2 --replication-factor 1
>
> Then I run --describe to check if the partitions have leaders. I get
>
>
> kafka-topics.sh --describe --zookeeper ad-0104:2181 --topic deleteme
>
> Topic:deleteme PartitionCount:2 ReplicationFactor:1 Configs:
> Topic: deleteme Partition: 0 Leader: 0 Replicas: 0 Isr: 0
> Topic: deleteme Partition: 1 Leader: 1 Replicas: 1 Isr: 1
>
>
> Finally, I run the console producer
>
> kafka-console-producer.sh --broker-list ad-0102:9092 --topic deleteme
>
> I get the following warning
>
> [2015-02-08 00:36:24,244] WARN Property topic is not valid
> (kafka.utils.VerifiableProperties)
>
> and then it waits for console input. When I try to send a message I get the
> following list of error messages
>
> [2015-02-08 00:37:04,735] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:04,751] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:04,752] ERROR Failed to collate messages by topic,
> partition due to: Failed to fetch topic metadata for topic: deleteme
> (kafka.producer.async.DefaultEventHandler)
> [2015-02-08 00:37:04,859] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:04,863] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:04,863] ERROR Failed to collate messages by topic,
> partition due to: Failed to fetch topic metadata for topic: deleteme
> (kafka.producer.async.DefaultEventHandler)
> [2015-02-08 00:37:04,968] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:04,974] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:04,974] ERROR Failed to collate messages by topic,
> partition due to: Failed to fetch topic metadata for topic: deleteme
> (kafka.producer.async.DefaultEventHandler)
> [2015-02-08 00:37:05,079] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:05,084] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:05,084] ERROR Failed to collate messages by topic,
> partition due to: Failed to fetch topic metadata for topic: deleteme
> (kafka.producer.async.DefaultEventHandler)
> [2015-02-08 00:37:05,189] WARN Error while fetching metadata
> [{TopicMetadata for topic deleteme ->
> No partition metadata for topic deleteme due to
> kafka.common.LeaderNotAvailableException}] for topic [deleteme]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-02-08 00:37:05,191] ERROR Failed to send requests for topics deleteme
> with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
> [2015-02-08 00:37:05,192] ERROR Error in handling batch of 1 events
> (kafka.producer.async.ProducerSendThread)
> kafka.common.FailedToSendMessageException: Failed to send messages after 3
> tries.
> at
>
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> at
>
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
> at
>
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
> at
>
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
> at scala.collection.immutable.Stream.foreach(Stream.scala:526)
> at
>
> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
> at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
>
>
>
>
>
>
> Clearly there's a leader, but it's throwing a LeaderNotAvailableException.
> Any help here is appreciated.
>
>
> Cheers,
>
> Alex Melville
>
--
Regards,
Tao