You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "wangxiaozhou (Jira)" <ji...@apache.org> on 2019/12/05 07:26:01 UTC
[jira] [Created] (METRON-2333) My squid case cannot be written into
HDFS and ES
wangxiaozhou created METRON-2333:
------------------------------------
Summary: My squid case cannot be written into HDFS and ES
Key: METRON-2333
URL: https://issues.apache.org/jira/browse/METRON-2333
Project: Metron
Issue Type: Bug
Affects Versions: 0.7.1
Reporter: wangxiaozhou
Attachments: Enrichment01.png, Enrichment02.png, Enrichments03.png, Enrichments04.png, Enrichments05.png, Index Settings.png, Indexing01.png, Indexing02.png, Indexing03.png, Parsers.png
My metron has been installed, but I found that it can't write data to es and HDFS, and I can't see the visual interface in kibana. Then, I tried to run the case of squid, but the same problem still occurred. I posted the configuration of my metron and the error information I saw in storm's random access ﹣ indexing topology below. It was me Is there a problem with the configuration of? Why can't I write data?
{code:java}
2019-12-05 09:12:59.493 o.a.s.d.executor Thread-15-indexingErrorBolt-executor[4 4] [ERROR]
java.lang.Exception: WARNING: Default and (likely) unoptimized writer config used for kafka writer and sensor error
at org.apache.metron.writer.bolt.BulkMessageWriterBolt.execute(BulkMessageWriterBolt.java:299) [stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__10193$tuple_action_fn__10195.invoke(executor.clj:730) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:462) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.daemon.executor$fn__10193$fn__10206$fn__10259.invoke(executor.clj:849) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
2019-12-05 09:12:59.493 o.a.m.e.b.ElasticsearchBulkDocumentWriter Thread-7-indexingBolt-executor[3 3] [ERROR] Failed to submit bulk request; all documents failed
java.net.ConnectException: 拒绝连接
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_172]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_172]
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171) ~[stormjar.jar:?]
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145) ~[stormjar.jar:?]
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348) ~[stormjar.jar:?]
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192) ~[stormjar.jar:?]
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[stormjar.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
2019-12-05 09:12:59.500 o.a.s.d.executor Thread-7-indexingBolt-executor[3 3] [ERROR]
java.net.ConnectException: 拒绝连接
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_172]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_172]
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171) ~[stormjar.jar:?]
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145) ~[stormjar.jar:?]
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348) ~[stormjar.jar:?]
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192) ~[stormjar.jar:?]
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[stormjar.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
2019-12-05 09:12:59.501 o.a.s.d.executor Thread-15-indexingErrorBolt-executor[4 4] [ERROR]
java.lang.Exception: WARNING: Default and (likely) unoptimized writer config used for kafka writer and sensor error
at org.apache.metron.writer.bolt.BulkMessageWriterBolt.execute(BulkMessageWriterBolt.java:299) [stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__10193$tuple_action_fn__10195.invoke(executor.clj:730) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:462) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.1.0.2.6.3.118-1.jar:1.1.0.2.6.3.118-1]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)