You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Vi...@Dell.com on 2015/01/22 10:33:56 UTC
Help: Kafka LeaderNotAvailableException
Hi,
Let me overview on the issue that I am facing on producing message in Kafka:
I have horthonworks HDP-2.1 installed, along that we have Kafka on other node.
* On kafka node:
Start Zookeepeer
Start Kafka Broker service
Send message/producer
Consume message - Works (Note: here we start Zookeeper locally on kafka01 node)
* Issue side:
>Now in HDP-2.1 we have Zookeeper inbuild & we have Zookeeper service running on master node
>I go to Kafka server & Started Kafka Broker
(In config\server.properties file I have added zookeeper.connect with maasternode:2181)
Then I start producer & Send message... after that we got error like kafka.common.LeaderNotAvailableException
[2015-01-17 05:54:09,465] WARN Error while fetching metadata [{TopicMetadata for topic fred ->
No partition metadata for topic fred due to kafka.common.LeaderNotAvailableException}] for topic [fred]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-01-17 05:54:09,659] WARN Error while fetching metadata [{TopicMetadata for topic fred ->
No partition metadata for topic fred due to kafka.common.LeaderNotAvailableException}] for topic [fred]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-01-17 05:54:09,659] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: fred (kafka.producer.async.DefaultEventHandler)
[2015-01-17 05:54:09,802] WARN Error while fetching metadata [{TopicMetadata for topic fred ->
No partition metadata for topic fred due to kafka.common.LeaderNotAvailableException}] for topic [fred]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-01-17 05:54:09,820] ERROR Failed to send requests for topics fred with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2015-01-17 05:54:09,822] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
Can someone suggest what is going wrong.!!!!
Thanks.
-Vishal
Regards,
Vishal
Software Dev Staff Engineer
Dell | Bangalore
Ext : 79268
Re: Help: Kafka LeaderNotAvailableException
Posted by Gwen Shapira <gs...@cloudera.com>.
It sounds like you have two zookeepers, one for HDP and one for Kafka.
Did you move Kafka from one zookeeper to another?
Perhaps Kafka finds the topics (logs) on disk, but they do not exist
in ZK because you are using a different zookeeper now.
Gwen
On Thu, Jan 22, 2015 at 6:38 PM, Jun Rao <ju...@confluent.io> wrote:
> Any error in the controller and the broker log?
>
> Thanks,
>
> Jun
>
> On Thu, Jan 22, 2015 at 1:33 AM, <Vi...@dell.com> wrote:
>
>> Hi,
>> Let me overview on the issue that I am facing on producing message in
>> Kafka:
>> I have horthonworks HDP-2.1 installed, along that we have Kafka on other
>> node.
>>
>> * On kafka node:
>> Start Zookeepeer
>> Start Kafka Broker service
>> Send message/producer
>> Consume message - Works (Note: here we start Zookeeper locally on kafka01
>> node)
>>
>> * Issue side:
>> >Now in HDP-2.1 we have Zookeeper inbuild & we have Zookeeper service
>> running on master node
>> >I go to Kafka server & Started Kafka Broker
>> (In config\server.properties file I have added zookeeper.connect with
>> maasternode:2181)
>> Then I start producer & Send message... after that we got error like
>> kafka.common.LeaderNotAvailableException
>>
>> [2015-01-17 05:54:09,465] WARN Error while fetching metadata
>> [{TopicMetadata for topic fred ->
>> No partition metadata for topic fred due to
>> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
>> kafka.common.LeaderNotAvailableException
>> (kafka.producer.BrokerPartitionInfo)
>> [2015-01-17 05:54:09,659] WARN Error while fetching metadata
>> [{TopicMetadata for topic fred ->
>> No partition metadata for topic fred due to
>> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
>> kafka.common.LeaderNotAvailableException
>> (kafka.producer.BrokerPartitionInfo)
>> [2015-01-17 05:54:09,659] ERROR Failed to collate messages by topic,
>> partition due to: Failed to fetch topic metadata for topic: fred
>> (kafka.producer.async.DefaultEventHandler)
>> [2015-01-17 05:54:09,802] WARN Error while fetching metadata
>> [{TopicMetadata for topic fred ->
>> No partition metadata for topic fred due to
>> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
>> kafka.common.LeaderNotAvailableException
>> (kafka.producer.BrokerPartitionInfo)
>> [2015-01-17 05:54:09,820] ERROR Failed to send requests for topics fred
>> with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
>> [2015-01-17 05:54:09,822] ERROR Error in handling batch of 1 events
>> (kafka.producer.async.ProducerSendThread)
>> kafka.common.FailedToSendMessageException: Failed to send messages after 3
>> tries.
>> at
>> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
>> at
>> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
>> at
>> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
>>
>> Can someone suggest what is going wrong.!!!!
>> Thanks.
>>
>>
>>
>>
>>
>> -Vishal
>>
>>
>> Regards,
>> Vishal
>> Software Dev Staff Engineer
>> Dell | Bangalore
>> Ext : 79268
>>
>>
Re: Help: Kafka LeaderNotAvailableException
Posted by Jun Rao <ju...@confluent.io>.
Any error in the controller and the broker log?
Thanks,
Jun
On Thu, Jan 22, 2015 at 1:33 AM, <Vi...@dell.com> wrote:
> Hi,
> Let me overview on the issue that I am facing on producing message in
> Kafka:
> I have horthonworks HDP-2.1 installed, along that we have Kafka on other
> node.
>
> * On kafka node:
> Start Zookeepeer
> Start Kafka Broker service
> Send message/producer
> Consume message - Works (Note: here we start Zookeeper locally on kafka01
> node)
>
> * Issue side:
> >Now in HDP-2.1 we have Zookeeper inbuild & we have Zookeeper service
> running on master node
> >I go to Kafka server & Started Kafka Broker
> (In config\server.properties file I have added zookeeper.connect with
> maasternode:2181)
> Then I start producer & Send message... after that we got error like
> kafka.common.LeaderNotAvailableException
>
> [2015-01-17 05:54:09,465] WARN Error while fetching metadata
> [{TopicMetadata for topic fred ->
> No partition metadata for topic fred due to
> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-01-17 05:54:09,659] WARN Error while fetching metadata
> [{TopicMetadata for topic fred ->
> No partition metadata for topic fred due to
> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-01-17 05:54:09,659] ERROR Failed to collate messages by topic,
> partition due to: Failed to fetch topic metadata for topic: fred
> (kafka.producer.async.DefaultEventHandler)
> [2015-01-17 05:54:09,802] WARN Error while fetching metadata
> [{TopicMetadata for topic fred ->
> No partition metadata for topic fred due to
> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-01-17 05:54:09,820] ERROR Failed to send requests for topics fred
> with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
> [2015-01-17 05:54:09,822] ERROR Error in handling batch of 1 events
> (kafka.producer.async.ProducerSendThread)
> kafka.common.FailedToSendMessageException: Failed to send messages after 3
> tries.
> at
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> at
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
> at
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
>
> Can someone suggest what is going wrong.!!!!
> Thanks.
>
>
>
>
>
> -Vishal
>
>
> Regards,
> Vishal
> Software Dev Staff Engineer
> Dell | Bangalore
> Ext : 79268
>
>
Re: Help: Kafka LeaderNotAvailableException
Posted by Joe Stein <jo...@stealth.ly>.
Vishal,
Does this error happen every time you are sending? Or just the first time?
Joe Stein
On Thu, Jan 22, 2015 at 4:33 AM, <Vi...@dell.com> wrote:
> Hi,
> Let me overview on the issue that I am facing on producing message in
> Kafka:
> I have horthonworks HDP-2.1 installed, along that we have Kafka on other
> node.
>
> * On kafka node:
> Start Zookeepeer
> Start Kafka Broker service
> Send message/producer
> Consume message - Works (Note: here we start Zookeeper locally on kafka01
> node)
>
> * Issue side:
> >Now in HDP-2.1 we have Zookeeper inbuild & we have Zookeeper service
> running on master node
> >I go to Kafka server & Started Kafka Broker
> (In config\server.properties file I have added zookeeper.connect with
> maasternode:2181)
> Then I start producer & Send message... after that we got error like
> kafka.common.LeaderNotAvailableException
>
> [2015-01-17 05:54:09,465] WARN Error while fetching metadata
> [{TopicMetadata for topic fred ->
> No partition metadata for topic fred due to
> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-01-17 05:54:09,659] WARN Error while fetching metadata
> [{TopicMetadata for topic fred ->
> No partition metadata for topic fred due to
> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-01-17 05:54:09,659] ERROR Failed to collate messages by topic,
> partition due to: Failed to fetch topic metadata for topic: fred
> (kafka.producer.async.DefaultEventHandler)
> [2015-01-17 05:54:09,802] WARN Error while fetching metadata
> [{TopicMetadata for topic fred ->
> No partition metadata for topic fred due to
> kafka.common.LeaderNotAvailableException}] for topic [fred]: class
> kafka.common.LeaderNotAvailableException
> (kafka.producer.BrokerPartitionInfo)
> [2015-01-17 05:54:09,820] ERROR Failed to send requests for topics fred
> with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
> [2015-01-17 05:54:09,822] ERROR Error in handling batch of 1 events
> (kafka.producer.async.ProducerSendThread)
> kafka.common.FailedToSendMessageException: Failed to send messages after 3
> tries.
> at
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
> at
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
> at
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
>
> Can someone suggest what is going wrong.!!!!
> Thanks.
>
>
>
>
>
> -Vishal
>
>
> Regards,
> Vishal
> Software Dev Staff Engineer
> Dell | Bangalore
> Ext : 79268
>
>