You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Igor Kravzov <ig...@gmail.com> on 2016/05/26 21:17:22 UTC
GetKafka exception
Why I am getting exception bellow
2016-05-26 17:13:36,776 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.consumer.ConsumerFetcherManager
[ConsumerFetcherManager-1464297163222] Added fetcher for partitions
ArrayBuffer()
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.utils.VerifiableProperties Verifying properties
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.utils.VerifiableProperties Property client.id is overridden to
NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.utils.VerifiableProperties Property metadata.broker.list is
overridden to 10.1.132.113:6667
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.utils.VerifiableProperties Property request.timeout.ms is overridden
to 30000
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.client.ClientUtils$ Fetching metadata from broker
id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s)
Set(Test1)
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for producing
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
2016-05-26 17:13:36,981 WARN
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256
for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667]
failed
java.nio.channels.ClosedChannelException: null
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
~[kafka_2.10-0.8.2.2.jar:na]
at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
[kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
[kafka_2.10-0.8.2.2.jar:na]
at
kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
[kafka_2.10-0.8.2.2.jar:na]
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
[kafka_2.10-0.8.2.2.jar:na]
2016-05-26 17:13:36,981 INFO
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
2016-05-26 17:13:36,981 WARN
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
k.c.ConsumerFetcherManager$LeaderFinderThread
[d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread],
Failed to find leader for Set([Test1,0])
kafka.common.KafkaException: fetching topic metadata for topics
[Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)]
failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
~[kafka_2.10-0.8.2.2.jar:na]
at
kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
[kafka_2.10-0.8.2.2.jar:na]
Caused by: java.nio.channels.ClosedChannelException: null
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
~[kafka_2.10-0.8.2.2.jar:na]
at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
~[kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
~[kafka_2.10-0.8.2.2.jar:na]
... 3 common frames omitted
Re: GetKafka exception
Posted by Igor Kravzov <ig...@gmail.com>.
Hi ANdrew,
here is output for the first command
Topic:Test1 PartitionCount:1 ReplicationFactor:1 Configs:
Topic: Test1 Partition: 0 Leader: 1001 Replicas: 1001
Isr: 1001
Here is for second. I think i read the message.
{metadata.broker.list=xxxnode04.xxxxxxxxx.net:6667, request.timeout.ms=30000,
client.id=console-consumer-85156, security.protocol=PLAINTEXT}
▒urlXhttp://
www.jpost.com/Middle-East/Analysis-What-Exactly-is-Happening-Saudi-Arabia-454018pageTypenews
addDate▒▒V▒T postDate▒|.▒Tposttext
ghdhjdshd
PuTTY
^CProcessed a total of 1 messages
On Thu, May 26, 2016 at 8:08 PM, Andrew Psaltis <ps...@gmail.com>
wrote:
> Hi Igor,
> It looks from the stack trace that the consumer is having trouble finding
> the leader for topic "Test1". Can you execute on one of your brokers,
> perhaps the one with ip address 10.1.132.113 the following command:
>
> $KAFKA_HOME/bin/kafka-topics.sh --describe --zookeeper <urls> --topic Test1
>
> and let me know what the output is.
>
> Also another test that you can do to take NiFi out of the picture is to
> use the console consumer. You can execute that with the following command,
> again from one of your Kafka brokers.
>
> $KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper <urls> --topic Test1
>
> If you do not have a process writing any data into kafka, then you can add
> the following parameter to the above command "--from-beginning"
>
>
> Looking forward to hearing back from you.
>
> Thanks,
> Andrew
>
>
> On Thu, May 26, 2016 at 5:44 PM, Igor Kravzov <ig...@gmail.com>
> wrote:
>
>> Thanks Oleg. I'll try. If you find more data please let me know..
>>
>> On Thu, May 26, 2016 at 5:39 PM, Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com> wrote:
>>
>>> Igor, you may want to go they Kafka mailing list. I've seen this several
>>> times, and all I know it has to do with broker
>>> I'll look in more details once back on line
>>>
>>> Oleg
>>>
>>> Sent from my iPhone
>>>
>>> On May 26, 2016, at 17:20, Igor Kravzov <ig...@gmail.com> wrote:
>>>
>>> Why I am getting exception bellow
>>>
>>>
>>> 2016-05-26 17:13:36,776 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.consumer.ConsumerFetcherManager
>>> [ConsumerFetcherManager-1464297163222] Added fetcher for partitions
>>> ArrayBuffer()
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Verifying properties
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Property client.id is overridden to
>>> NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Property metadata.broker.list is
>>> overridden to 10.1.132.113:6667
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Property request.timeout.ms is
>>> overridden to 30000
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.client.ClientUtils$ Fetching metadata from broker
>>> id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s)
>>> Set(Test1)
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for producing
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>>> 2016-05-26 17:13:36,981 WARN
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256
>>> for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667]
>>> failed
>>> java.nio.channels.ClosedChannelException: null
>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>>> 2016-05-26 17:13:36,981 WARN
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> k.c.ConsumerFetcherManager$LeaderFinderThread
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread],
>>> Failed to find leader for Set([Test1,0])
>>> kafka.common.KafkaException: fetching topic metadata for topics
>>> [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)]
>>> failed
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> Caused by: java.nio.channels.ClosedChannelException: null
>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> ... 3 common frames omitted
>>>
>>>
>>
>
>
> --
> Thanks,
> Andrew
>
> Subscribe to my book: Streaming Data <http://manning.com/psaltis>
> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
>
Re: GetKafka exception
Posted by Joe Witt <jo...@gmail.com>.
Not bothering. This is good info we should document....
On May 27, 2016 11:47 AM, "Igor Kravzov" <ig...@gmail.com> wrote:
> Guys, I found the problem. It was my fault.
> Kafka broker configured as fully qualified domain name.
> What happens is when you call zookeper it returns FQDN but my hosts file
> was not updated with IPs and node names.
>
> Sorry guys for bothering you.
>
> On Thu, May 26, 2016 at 8:08 PM Andrew Psaltis <ps...@gmail.com>
> wrote:
>
>> Hi Igor,
>> It looks from the stack trace that the consumer is having trouble finding
>> the leader for topic "Test1". Can you execute on one of your brokers,
>> perhaps the one with ip address 10.1.132.113 the following command:
>>
>> $KAFKA_HOME/bin/kafka-topics.sh --describe --zookeeper <urls> --topic
>> Test1
>>
>> and let me know what the output is.
>>
>> Also another test that you can do to take NiFi out of the picture is to
>> use the console consumer. You can execute that with the following command,
>> again from one of your Kafka brokers.
>>
>> $KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper <urls> --topic Test1
>>
>> If you do not have a process writing any data into kafka, then you can
>> add the following parameter to the above command "--from-beginning"
>>
>>
>> Looking forward to hearing back from you.
>>
>> Thanks,
>> Andrew
>>
>>
>> On Thu, May 26, 2016 at 5:44 PM, Igor Kravzov <ig...@gmail.com>
>> wrote:
>>
>>> Thanks Oleg. I'll try. If you find more data please let me know..
>>>
>>> On Thu, May 26, 2016 at 5:39 PM, Oleg Zhurakousky <
>>> ozhurakousky@hortonworks.com> wrote:
>>>
>>>> Igor, you may want to go they Kafka mailing list. I've seen this
>>>> several times, and all I know it has to do with broker
>>>> I'll look in more details once back on line
>>>>
>>>> Oleg
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On May 26, 2016, at 17:20, Igor Kravzov <ig...@gmail.com> wrote:
>>>>
>>>> Why I am getting exception bellow
>>>>
>>>>
>>>> 2016-05-26 17:13:36,776 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.consumer.ConsumerFetcherManager
>>>> [ConsumerFetcherManager-1464297163222] Added fetcher for partitions
>>>> ArrayBuffer()
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.utils.VerifiableProperties Verifying properties
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.utils.VerifiableProperties Property client.id is overridden to
>>>> NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.utils.VerifiableProperties Property metadata.broker.list is
>>>> overridden to 10.1.132.113:6667
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.utils.VerifiableProperties Property request.timeout.ms is
>>>> overridden to 30000
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.client.ClientUtils$ Fetching metadata from broker
>>>> id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s)
>>>> Set(Test1)
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for
>>>> producing
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>>>> 2016-05-26 17:13:36,981 WARN
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256
>>>> for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667]
>>>> failed
>>>> java.nio.channels.ClosedChannelException: null
>>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at
>>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>>>> [kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>>>> [kafka_2.10-0.8.2.2.jar:na]
>>>> at
>>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>>>> [kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>>>> [kafka_2.10-0.8.2.2.jar:na]
>>>> 2016-05-26 17:13:36,981 INFO
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>>>> 2016-05-26 17:13:36,981 WARN
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>>> k.c.ConsumerFetcherManager$LeaderFinderThread
>>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread],
>>>> Failed to find leader for Set([Test1,0])
>>>> kafka.common.KafkaException: fetching topic metadata for topics
>>>> [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)]
>>>> failed
>>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at
>>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>>>> [kafka_2.10-0.8.2.2.jar:na]
>>>> Caused by: java.nio.channels.ClosedChannelException: null
>>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at
>>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>>> ... 3 common frames omitted
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Andrew
>>
>> Subscribe to my book: Streaming Data <http://manning.com/psaltis>
>> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
>> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
>>
>
Re: GetKafka exception
Posted by Igor Kravzov <ig...@gmail.com>.
Guys, I found the problem. It was my fault.
Kafka broker configured as fully qualified domain name.
What happens is when you call zookeper it returns FQDN but my hosts file
was not updated with IPs and node names.
Sorry guys for bothering you.
On Thu, May 26, 2016 at 8:08 PM Andrew Psaltis <ps...@gmail.com>
wrote:
> Hi Igor,
> It looks from the stack trace that the consumer is having trouble finding
> the leader for topic "Test1". Can you execute on one of your brokers,
> perhaps the one with ip address 10.1.132.113 the following command:
>
> $KAFKA_HOME/bin/kafka-topics.sh --describe --zookeeper <urls> --topic Test1
>
> and let me know what the output is.
>
> Also another test that you can do to take NiFi out of the picture is to
> use the console consumer. You can execute that with the following command,
> again from one of your Kafka brokers.
>
> $KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper <urls> --topic Test1
>
> If you do not have a process writing any data into kafka, then you can add
> the following parameter to the above command "--from-beginning"
>
>
> Looking forward to hearing back from you.
>
> Thanks,
> Andrew
>
>
> On Thu, May 26, 2016 at 5:44 PM, Igor Kravzov <ig...@gmail.com>
> wrote:
>
>> Thanks Oleg. I'll try. If you find more data please let me know..
>>
>> On Thu, May 26, 2016 at 5:39 PM, Oleg Zhurakousky <
>> ozhurakousky@hortonworks.com> wrote:
>>
>>> Igor, you may want to go they Kafka mailing list. I've seen this several
>>> times, and all I know it has to do with broker
>>> I'll look in more details once back on line
>>>
>>> Oleg
>>>
>>> Sent from my iPhone
>>>
>>> On May 26, 2016, at 17:20, Igor Kravzov <ig...@gmail.com> wrote:
>>>
>>> Why I am getting exception bellow
>>>
>>>
>>> 2016-05-26 17:13:36,776 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.consumer.ConsumerFetcherManager
>>> [ConsumerFetcherManager-1464297163222] Added fetcher for partitions
>>> ArrayBuffer()
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Verifying properties
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Property client.id is overridden to
>>> NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Property metadata.broker.list is
>>> overridden to 10.1.132.113:6667
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.utils.VerifiableProperties Property request.timeout.ms is
>>> overridden to 30000
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.client.ClientUtils$ Fetching metadata from broker
>>> id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s)
>>> Set(Test1)
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for producing
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>>> 2016-05-26 17:13:36,981 WARN
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256
>>> for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667]
>>> failed
>>> java.nio.channels.ClosedChannelException: null
>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> 2016-05-26 17:13:36,981 INFO
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>>> 2016-05-26 17:13:36,981 WARN
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>>> k.c.ConsumerFetcherManager$LeaderFinderThread
>>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread],
>>> Failed to find leader for Set([Test1,0])
>>> kafka.common.KafkaException: fetching topic metadata for topics
>>> [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)]
>>> failed
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>>> [kafka_2.10-0.8.2.2.jar:na]
>>> Caused by: java.nio.channels.ClosedChannelException: null
>>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at
>>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>>> ~[kafka_2.10-0.8.2.2.jar:na]
>>> ... 3 common frames omitted
>>>
>>>
>>
>
>
> --
> Thanks,
> Andrew
>
> Subscribe to my book: Streaming Data <http://manning.com/psaltis>
> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
>
Re: GetKafka exception
Posted by Andrew Psaltis <ps...@gmail.com>.
Hi Igor,
It looks from the stack trace that the consumer is having trouble finding
the leader for topic "Test1". Can you execute on one of your brokers,
perhaps the one with ip address 10.1.132.113 the following command:
$KAFKA_HOME/bin/kafka-topics.sh --describe --zookeeper <urls> --topic Test1
and let me know what the output is.
Also another test that you can do to take NiFi out of the picture is to use
the console consumer. You can execute that with the following command,
again from one of your Kafka brokers.
$KAFKA_HOME/bin/kafka-console-consumer.sh --zookeeper <urls> --topic Test1
If you do not have a process writing any data into kafka, then you can add
the following parameter to the above command "--from-beginning"
Looking forward to hearing back from you.
Thanks,
Andrew
On Thu, May 26, 2016 at 5:44 PM, Igor Kravzov <ig...@gmail.com>
wrote:
> Thanks Oleg. I'll try. If you find more data please let me know..
>
> On Thu, May 26, 2016 at 5:39 PM, Oleg Zhurakousky <
> ozhurakousky@hortonworks.com> wrote:
>
>> Igor, you may want to go they Kafka mailing list. I've seen this several
>> times, and all I know it has to do with broker
>> I'll look in more details once back on line
>>
>> Oleg
>>
>> Sent from my iPhone
>>
>> On May 26, 2016, at 17:20, Igor Kravzov <ig...@gmail.com> wrote:
>>
>> Why I am getting exception bellow
>>
>>
>> 2016-05-26 17:13:36,776 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.consumer.ConsumerFetcherManager
>> [ConsumerFetcherManager-1464297163222] Added fetcher for partitions
>> ArrayBuffer()
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.utils.VerifiableProperties Verifying properties
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.utils.VerifiableProperties Property client.id is overridden to
>> NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.utils.VerifiableProperties Property metadata.broker.list is
>> overridden to 10.1.132.113:6667
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.utils.VerifiableProperties Property request.timeout.ms is
>> overridden to 30000
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.client.ClientUtils$ Fetching metadata from broker
>> id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s)
>> Set(Test1)
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for producing
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>> 2016-05-26 17:13:36,981 WARN
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256
>> for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667]
>> failed
>> java.nio.channels.ClosedChannelException: null
>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at
>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>> [kafka_2.10-0.8.2.2.jar:na]
>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>> [kafka_2.10-0.8.2.2.jar:na]
>> at
>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>> [kafka_2.10-0.8.2.2.jar:na]
>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>> [kafka_2.10-0.8.2.2.jar:na]
>> 2016-05-26 17:13:36,981 INFO
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
>> 2016-05-26 17:13:36,981 WARN
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
>> k.c.ConsumerFetcherManager$LeaderFinderThread
>> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread],
>> Failed to find leader for Set([Test1,0])
>> kafka.common.KafkaException: fetching topic metadata for topics
>> [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)]
>> failed
>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at
>> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
>> [kafka_2.10-0.8.2.2.jar:na]
>> Caused by: java.nio.channels.ClosedChannelException: null
>> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at
>> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
>> ~[kafka_2.10-0.8.2.2.jar:na]
>> ... 3 common frames omitted
>>
>>
>
--
Thanks,
Andrew
Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
Re: GetKafka exception
Posted by Igor Kravzov <ig...@gmail.com>.
Thanks Oleg. I'll try. If you find more data please let me know..
On Thu, May 26, 2016 at 5:39 PM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com> wrote:
> Igor, you may want to go they Kafka mailing list. I've seen this several
> times, and all I know it has to do with broker
> I'll look in more details once back on line
>
> Oleg
>
> Sent from my iPhone
>
> On May 26, 2016, at 17:20, Igor Kravzov <ig...@gmail.com> wrote:
>
> Why I am getting exception bellow
>
>
> 2016-05-26 17:13:36,776 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.consumer.ConsumerFetcherManager
> [ConsumerFetcherManager-1464297163222] Added fetcher for partitions
> ArrayBuffer()
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.utils.VerifiableProperties Verifying properties
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.utils.VerifiableProperties Property client.id is overridden to
> NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.utils.VerifiableProperties Property metadata.broker.list is
> overridden to 10.1.132.113:6667
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.utils.VerifiableProperties Property request.timeout.ms is
> overridden to 30000
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.client.ClientUtils$ Fetching metadata from broker
> id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s)
> Set(Test1)
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.producer.SyncProducer Connected to 10.1.132.113:6667 for producing
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
> 2016-05-26 17:13:36,981 WARN
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256
> for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667]
> failed
> java.nio.channels.ClosedChannelException: null
> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at
> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
> [kafka_2.10-0.8.2.2.jar:na]
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
> [kafka_2.10-0.8.2.2.jar:na]
> at
> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
> [kafka_2.10-0.8.2.2.jar:na]
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [kafka_2.10-0.8.2.2.jar:na]
> 2016-05-26 17:13:36,981 INFO
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667
> 2016-05-26 17:13:36,981 WARN
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread]
> k.c.ConsumerFetcherManager$LeaderFinderThread
> [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread],
> Failed to find leader for Set([Test1,0])
> kafka.common.KafkaException: fetching topic metadata for topics
> [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)]
> failed
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at
> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
> [kafka_2.10-0.8.2.2.jar:na]
> Caused by: java.nio.channels.ClosedChannelException: null
> at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at
> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
> ~[kafka_2.10-0.8.2.2.jar:na]
> at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
> ~[kafka_2.10-0.8.2.2.jar:na]
> ... 3 common frames omitted
>
>
Re: GetKafka exception
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Igor, you may want to go they Kafka mailing list. I've seen this several times, and all I know it has to do with broker
I'll look in more details once back on line
Oleg
Sent from my iPhone
On May 26, 2016, at 17:20, Igor Kravzov <ig...@gmail.com>> wrote:
Why I am getting exception bellow
2016-05-26 17:13:36,776 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.consumer.ConsumerFetcherManager [ConsumerFetcherManager-1464297163222] Added fetcher for partitions ArrayBuffer()
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.utils.VerifiableProperties Verifying properties
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.utils.VerifiableProperties Property client.id<http://client.id> is overridden to NiFi-d217bceb-d58b-4031-9143-26d8ddd15e40
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.utils.VerifiableProperties Property metadata.broker.list is overridden to 10.1.132.113:6667<http://10.1.132.113:6667>
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.utils.VerifiableProperties Property request.timeout.ms<http://request.timeout.ms> is overridden to 30000
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.client.ClientUtils$ Fetching metadata from broker id:1001,host:10.1.132.113,port:6667 with correlation id 256 for 1 topic(s) Set(Test1)
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.producer.SyncProducer Connected to 10.1.132.113:6667<http://10.1.132.113:6667> for producing
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667<http://10.1.132.113:6667>
2016-05-26 17:13:36,981 WARN [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.client.ClientUtils$ Fetching topic metadata with correlation id 256 for topics [Set(Test1)] from broker [id:1001,host:10.1.132.113,port:6667] failed
java.nio.channels.ClosedChannelException: null
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.send(SyncProducer.scala:113) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) [kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) [kafka_2.10-0.8.2.2.jar:na]
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) [kafka_2.10-0.8.2.2.jar:na]
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) [kafka_2.10-0.8.2.2.jar:na]
2016-05-26 17:13:36,981 INFO [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] kafka.producer.SyncProducer Disconnecting from 10.1.132.113:6667<http://10.1.132.113:6667>
2016-05-26 17:13:36,981 WARN [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread] k.c.ConsumerFetcherManager$LeaderFinderThread [d217bceb-d58b-4031-9143-26d8ddd15e40_BPIW01a-1464297163195-b78e95a5-leader-finder-thread], Failed to find leader for Set([Test1,0])
kafka.common.KafkaException: fetching topic metadata for topics [Set(Test1)] from broker [ArrayBuffer(id:1001,host:10.1.132.113,port:6667)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) [kafka_2.10-0.8.2.2.jar:na]
Caused by: java.nio.channels.ClosedChannelException: null
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.producer.SyncProducer.send(SyncProducer.scala:113) ~[kafka_2.10-0.8.2.2.jar:na]
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58) ~[kafka_2.10-0.8.2.2.jar:na]
... 3 common frames omitted