Re: Issues in sending syslog

[Hema malini <nh...@gmail.com>]

Hi,

Finally able to fix the issue. It is due to FQDN.

Thanks and regards,
Hema

On Mon, 9 Sep, 2019, 9:37 PM Hema malini, <nh...@gmail.com> wrote:

> Nothing is written to error indexing topic on es. I could seethe same
> timeout error in storm.can you please help which property I need to change
> for this.
>
> Error
>
>
> 2019-09-05 18:43:31.622 o.a.s.d.executor Thread-12-parserBolt-executor[5
> 5] [ERROR] java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.TimeoutException: Failed to update
> metadata after 60000 ms. at org.apache.kafka.clients.
> producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:730)
> ~[stormjar.jar:?] at org.apache.kafka.clients.
> producer.KafkaProducer.doSend(KafkaProducer.java:483) ~[stormjar.jar:?]
> at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:430)
> ~[stormjar.jar:?] at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:353)
> ~[stormjar.jar:?] at org.apache.metron.writer.kafka.KafkaWriter.write(KafkaWriter.java:257)
> ~[stormjar.jar:?] at org.apache.metron.writer.BulkWriterComponent.flush(BulkWriterComponent.java:123)
> [stormjar.jar:?] at org.apache.metron.writer.BulkWriterComponent.
> applyShouldFlush(BulkWriterComponent.java:179) [stormjar.jar:?] at
> org.apache.metron.writer.BulkWriterComponent.write(BulkWriterComponent.java:99)
> [stormjar.jar:?] at org.apache.metron.parsers.bolt.WriterHandler.write(WriterHandler.java:90)
> [stormjar.jar:?] at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:269)
> [stormjar.jar:?] at org.apache.storm.daemon.executor$fn__10195$tuple_
> action_fn__10197.invoke(executor.clj:735) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
> at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at
> org.apache.storm.daemon.executor$fn__10195$fn__10208$
> fn__10263.invoke(executor.clj:855) [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1]
> at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> [storm-core-1.1.0.2.6.5.1175-1.jar:1.1.0.2.6.5.1175-1] at
> clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at
> java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by:
> org.apache.kafka.common.
>
> On Thu, 5 Sep, 2019, 9:24 PM Michael Miklavcic, <
> michael.miklavcic@gmail.com> wrote:
>
>> Also look in the error indexing topic if Solr or ES. And there should
>> also be data written to HDFS, even in case of error. hdfs dfs -ls
>> /apps/metron/indexing
>>
>> On Wed, Sep 4, 2019 at 10:17 AM Hema malini <nh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I am using Metron 0.7.2. I have setup of three node cluster in centos 7.
>>> I am sending syslog and bro logs to corresponding Kafka topic but it is not
>>> reaching the bro or syslog storm topology. I could see the messages in the
>>> consumer of kafka. But itis neither reaching the enrichment or bro nor
>>> syslog3164. I couldn't find any error logs in storm(worker -artifacts not
>>> updated with any logs) nor no error messages in Kafka. Can someone please
>>> help to debug this issue. I could see only syslog5674 topic created. I
>>> created manually the other syslog3164 Kafka topic. I was able to send logs
>>> to syslog3164 and I could seethe messages in Kafka consumer. But nothing
>>> has reached storm.can someone please help.
>>>
>>> Thanks and regards,
>>> Hema
>>>
>>