You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "Anand Subramanian (JIRA)" <ji...@apache.org> on 2016/08/18 08:05:20 UTC
[jira] [Commented] (METRON-293) indexingBolt errors out for bro
logs having IPV6 address
[ https://issues.apache.org/jira/browse/METRON-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426064#comment-15426064 ]
Anand Subramanian commented on METRON-293:
------------------------------------------
I am noticing the same error when the IPs are FQDNs. Here's one of the error messages:
{code}
2016-08-18 06:44:55.655 o.a.m.w.BulkWriterComponent [ERROR] Failing 5 tuples
java.lang.Exception: failure in bulk execution:
[3]: index [bro_index_2016.08.18.06], type [bro_doc], id [AVacZTyCgcU1KRlw-1Rp], message [MapperParsingException[failed to parse [answers]]; nested: IllegalArgumentException[failed to parse ip [www.nissan-corp.com.edgesuite.net], not a valid ip address];]
at org.apache.metron.elasticsearch.writer.ElasticsearchWriter.write(ElasticsearchWriter.java:173) ~[stormjar.jar:?]
at org.apache.metron.writer.BulkWriterComponent.write(BulkWriterComponent.java:108) [stormjar.jar:?]
at org.apache.metron.writer.bolt.BulkMessageWriterBolt.execute(BulkMessageWriterBolt.java:98) [stormjar.jar:?]
at backtype.storm.daemon.executor$fn__7143$tuple_action_fn__7145.invoke(executor.clj:684) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at backtype.storm.daemon.executor$mk_task_receiver$fn__7066.invoke(executor.clj:431) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at backtype.storm.disruptor$clojure_handler$reify__6642.onEvent(disruptor.clj:58) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at backtype.storm.daemon.executor$fn__7143$fn__7156$fn__7207.invoke(executor.clj:813) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at backtype.storm.util$async_loop$fn__1985.invoke(util.clj:479) [storm-core-0.10.0.2.3.6.0-3796.jar:0.10.0.2.3.6.0-3796]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
{code}
Note that the metron node running the bro parser is able to resolve the IP fine.
{code}
[root@metron-test-1 ~]# nslookup www.nissan-corp.com.edgesuite.net
Server: 10.42.1.20
Address: 10.42.1.20#53
Non-authoritative answer:
www.nissan-corp.com.edgesuite.net canonical name = a175.g.akamai.net.
Name: a175.g.akamai.net
Address: 184.25.56.157
Name: a175.g.akamai.net
Address: 184.25.56.171
{code}
> indexingBolt errors out for bro logs having IPV6 address
> --------------------------------------------------------
>
> Key: METRON-293
> URL: https://issues.apache.org/jira/browse/METRON-293
> Project: Metron
> Issue Type: Bug
> Affects Versions: 0.2.1BETA
> Reporter: Neha Sinha
> Priority: Minor
> Fix For: 0.3.0BETA
>
> Attachments: Screen Shot 2016-07-08 at 4.59.56 PM.png, bro_ipv6_address.rtf, enrichment_indexingBolt_error_stack_trace.rtf
>
>
> Hi,
> So i am injecting the following bro log that has IPV6 addresses :-
> {"http": {"ts":1467617777.886267,"uid":"CXkPNR186cdD0rqPi","id.orig_h":"2001:cdba:0:0:0:0:3257:9652","id.orig_p":49191,"id.resp_h":"2001:cdba:0:0:0:0:3257:9651","id.resp_p":80,"trans_depth":1,"method":"GET","host":"62.75.195.236","uri":"/?3a08b0be8322c244f5a1cb9c1057d941","user_agent":"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)","request_body_len":0,"response_body_len":0,"status_code":200,"status_msg":"OK","tags":[]}}
> The bro parser parses the above log all good but I happen to see error with the enrichment indexingBolt.
> Please find attached the stacktrace for enrichment indexing bolt and also the storm log captured for bro parser.
> Regards,
> Neha
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)