You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Hema malini <nh...@gmail.com> on 2019/10/21 05:35:57 UTC
Re: Kafka error in metron
Hi ,
Thanks . We fixed the issue. It is due to FQDN not properly set. Once again
thanks a lot for your support.
Thanks and Regards,
Hema
On Thu, 5 Sep, 2019, 11:26 PM Hema malini, <nh...@gmail.com> wrote:
> Thanks Michael.can you please let me know what config should I check in
> metron.
>
> On Thu, 5 Sep, 2019, 9:33 PM Michael Miklavcic, <
> michael.miklavcic@gmail.com> wrote:
>
>> You may need to also check your Metron configs for the readers/writers
>> via Ambari. Since it's failing in the bulk writer, I suspect there may
>> still be a problem with the config there.
>>
>> On Thu, Sep 5, 2019 at 7:24 AM Hema malini <nh...@gmail.com>
>> wrote:
>>
>>> That issue got resolved after changing the property listeners to
>>> PLAINTEXT://:9092.
>>>
>>> I pushed the logsto bro and syslog.i could see messages only in bro
>>> topology but with the error as below.can someone pls help
>>>
>>> 2019-09-05 18:43:31.622 o.a.s.d.executor Thread-12-parserBolt-executor[5 5] [ERROR]
>>> java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
>>> at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:730) ~[stormjar.jar:?]
>>> at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:483) ~[stormjar.jar:?]
>>> at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:430) ~[stormjar.jar:?]
>>> at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:353) ~[stormjar.jar:?]
>>> at org.apache.metron.writer.kafka.KafkaWriter.write(KafkaWriter.java:257) ~[stormjar.jar:?]
>>> at org.apache.metron.writer.BulkWriterComponent.flush(BulkWriterComponent.java:123) [stormjar.jar:?]
>>> at org.apache.metron.writer.BulkWriterComponent.applyShouldFlush(BulkWriterComponent.java:179) [stormjar.jar:?]
>>> at org.apache.metron.writer.BulkWriterComponent.write(BulkWriterComponent.java:99) [stormjar.jar:?]
>>> at org.apache.metron.parsers.bolt.WriterHandler.write(WriterHandler.java:90) [stormjar.jar:?]
>>> at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:269) [stormjar.jar:?]
>>> at org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
>>> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
>>> Caused by: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
>>>
>>>
>>>
>>> On Wed, 4 Sep, 2019, 3:15 PM Hema malini, <nh...@gmail.com>
>>> wrote:
>>>
>>>> 9092 for bootstrap server . Node1 is the example hostname. Kafka port
>>>> is 6667. Can you please let me know what shouldbe configured for listeners
>>>> for three node.
>>>> Thanks and regards,
>>>> Hema
>>>> On Wed, 4 Sep, 2019, 1:45 PM Simon Elliston Ball, <
>>>> simon@simonellistonball.com> wrote:
>>>>
>>>>> The default port for Kafka in an HDP install is 6667, not 9092. Also
>>>>> node1 is the full dev Kafka. You will need to provide a correct
>>>>> bootstrap-server setting for your brokers.
>>>>>
>>>>> Simon
>>>>>
>>>>> On Wed, 4 Sep 2019 at 09:12, Hema malini <nh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I installed using Hdp and managing Kafka using ambari.
>>>>>>
>>>>>> I gave the command from the node as
>>>>>> bin/kafka-console-consumer.sh --bootstrap-server node1:9092 --topic
>>>>>> bro --from-beginning
>>>>>>
>>>>>> I am getting warning as
>>>>>> Connection to node -1 could not be established. Broker may not be
>>>>>> available. (Org.apache.kafka.clients.NetworkClient)
>>>>>>
>>>>>> Error as below
>>>>>> kafka.common.NoReplicaOnlineException: No replica in ISR for
>>>>>> partition __consumer_offsets-2 is alive. Live brokers are: [Set(1002)], ISR
>>>>>> brokers are: [1001]
>>>>>> at
>>>>>> kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:65)
>>>>>> at
>>>>>> kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:303)
>>>>>> at
>>>>>> kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:163)
>>>>>> at
>>>>>> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:84)
>>>>>> at
>>>>>> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:81)
>>>>>> at
>>>>>> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
>>>>>> at scala.collection.mutable.HashMap$$anonfun$foreach$1
>>>>>> .apply(HashMap.scala:130)
>>>>>> at scala.collection.mutable.HashMap$$anonfun$foreach$1
>>>>>> .apply(HashMap.scala:130)
>>>>>> at
>>>>>> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)
>>>>>> at
>>>>>> scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
>>>>>> at scala.collection.mutable.HashMap.foreach(HashMap.scala:130)
>>>>>> at
>>>>>> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
>>>>>> at
>>>>>> kafka.controller.PartitionStateMachine.triggerOnlinePartitionStateChange(PartitionStateMachine.scala:81)
>>>>>> at
>>>>>> kafka.controller.KafkaController.onBrokerStartup(KafkaController.scala:402)
>>>>>> at
>>>>>> kafka.controller.KafkaController$BrokerChange.process(KafkaController.scala:1226)
>>>>>> at
>>>>>> kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply$mcV$sp(ControllerEventManager.scala:53)
>>>>>> at
>>>>>> kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53)
>>>>>> at
>>>>>> kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53)
>>>>>> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31)
>>>>>> at
>>>>>> kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:52)
>>>>>> at
>>>>>> kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64)
>>>>>> For multinode,(3 node) , what should I configure in ambari console
>>>>>> Kafka config for
>>>>>> 1.listeners ( do I need to give ip of three nodes)
>>>>>>
>>>>>> Asofnow for three nodes, individually changed the property in each
>>>>>> node and started Kafka from console using
>>>>>> bin/kafka-server-start.sh
>>>>>> How can I manage from ambari console.
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Hema
>>>>>>
>>>>>>
>>>>>> On Tue, 3 Sep, 2019, 11:05 PM James Sirota, <js...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> + 1 to what Mike said. Also, if you could attach any kafka logs
>>>>>>> that contain any error messages that would be helpful
>>>>>>>
>>>>>>>
>>>>>>> 03.09.2019, 08:42, "Michael Miklavcic" <michael.miklavcic@gmail.com
>>>>>>> >:
>>>>>>>
>>>>>>> Hi Hema,
>>>>>>>
>>>>>>> A couple Q's for you to help narrow this down:
>>>>>>>
>>>>>>> 1. How did you got about installing Kafka and the rest of your
>>>>>>> Hadoop cluster? Is it an HDP installation managed by Ambari?
>>>>>>> 2. Please copy/paste the exact commands you're running to
>>>>>>> produce/consume messages to Kafka
>>>>>>> 3. Full stack trace of any errors you encounter.
>>>>>>>
>>>>>>> If you're using Ambari, this should be fully managed for you. It
>>>>>>> looks like you may have installed Kafka manually? e.g.
>>>>>>> https://kafka.apache.org/quickstart#quickstart_multibroker
>>>>>>>
>>>>>>> Best,
>>>>>>> Mike
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Sep 3, 2019 at 8:39 AM Hema malini <nh...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> I am able to send messages when I configure listeners properties to
>>>>>>> single node in Kafka ( for each node,changed the listener property to that
>>>>>>> host name) and then restarted Kafka from command prompt. How can I manage
>>>>>>> the same using ambari.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Hema
>>>>>>>
>>>>>>> On Tue, 3 Sep, 2019, 7:04 PM Hema malini, <nh...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Also, I am able to create topic and see the topics being created.
>>>>>>> Facing issues while consuming the messages.
>>>>>>>
>>>>>>> On Tue, 3 Sep, 2019, 7:00 PM Hema malini, <nh...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have installed Metron 0.7.2 in three node cluster set up. When
>>>>>>> running Kafka consumer in the command prompt getting error as "connection
>>>>>>> to node -1 could not be established.broker may not be available". What
>>>>>>> should I configure listeners properties in server.properties file.what are
>>>>>>> all the other properties needs to be changed. please help to fix the Kafka
>>>>>>> issue.
>>>>>>>
>>>>>>> Thanks and regards,
>>>>>>> Hema
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -------------------
>>>>>>> Thank you,
>>>>>>>
>>>>>>> James Sirota
>>>>>>> PMC- Apache Metron
>>>>>>> jsirota AT apache DOT org
>>>>>>>
>>>>>>> --
>>>>> --
>>>>> simon elliston ball
>>>>> @sireb
>>>>>
>>>>