You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by updates on tube <ab...@gmail.com> on 2020/03/05 13:30:56 UTC
conn.log unable to parse in apeche metron
##sample log or input log
1583402931.976871 CCBAYr2KnmpaWDtxO2 xx.xx.xx.xx 65184 xx.xx.xx.xx 4200 tcp - 1.855212 503 0 SH T T 0 ScADaF 5 715 2 80 -
1583402933.241900 C6C59e3TdNbeTTBZ7j xx.xx.xx.xx 16020 xx.xx.xx.xx 34032 tcp - 0.015988 2981 0 OTH T T 0 HcADC 6 352 0 0 -
##grok pattern that i used ( https://grokconstructor.appspot.com/groklib/bro)
BRO_CONN %{NUMBER:ts}\t%{NOTSPACE:uid}\t%{IP:orig_h}\t%{INT:orig_p}\t%{IP:resp_h}\t%{INT:resp_p}\t%{WORD:proto}\t%{GREEDYDATA:service}\t%{NUMBER:duration}\t%{NUMBER:orig_bytes}\t%{NUMBER:resp_bytes}\t%{GREEDYDATA:conn_state}\t%{GREEDYDATA:local_orig}\t%{GREEDYDATA:missed_bytes}\t%{GREEDYDATA:history}\t%{GREEDYDATA:orig_pkts}\t%{GREEDYDATA:orig_ip_bytes}\t%{GREEDYDATA:resp_pkts}\t%{GREEDYDATA:resp_ip_bytes}\t%{GREEDYDATA:tunnel_parents}
##the error shown in metron-rest.log
Caused by: java.lang.IllegalStateException: Unable to parse Message: 1583402939.738024 CTGU7D24R7NL5eTGef xx.xx.xx.xx 50998 xx.xx.xx.xx 6188 tcp - - - - OTH T T 0C 0 0 0 0 -
at org.apache.metron.parsers.bro.BasicBroParser.parse(BasicBroParser.java:145) ~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at org.apache.metron.parsers.interfaces.MessageParser.parseOptional(MessageParser.java:54) ~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at org.apache.metron.parsers.interfaces.MessageParser.parseOptionalResult(MessageParser.java:67) ~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at org.apache.metron.rest.service.impl.SensorParserConfigServiceImpl.parseMessage(SensorParserConfigServiceImpl.java:155) ~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
... 94 more
Caused by: org.json.simple.parser.ParseException
at org.json.simple.parser.Yylex.yylex(Yylex.java:610) ~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.nextToken(JSONParser.java:269) ~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.parse(JSONParser.java:118) ~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.parse(JSONParser.java:81) ~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.parse(JSONParser.java:75) ~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.apache.metron.parsers.bro.JSONCleaner.clean(JSONCleaner.java:49) ~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at org.apache.metron.parsers.bro.BasicBroParser.parse(BasicBroParser.java:68) ~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at org.apache.metron.parsers.interfaces.MessageParser.parseOptional(MessageParser.java:54) ~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at org.apache.metron.parsers.interfaces.MessageParser.parseOptionalResult(MessageParser.java:67) ~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
#i need your help as always.
Re: conn.log unable to parse in apeche metron
Posted by Otto Fowler <ot...@gmail.com>.
I’m confused at what you are doing here. What parser are you using? grok or
bro?
The bro parse works on bro JSON output. Your logs don’t look like they are
output as JSON, that is why it is failing I would guess.
On March 5, 2020 at 08:30:58, updates on tube (abrahamfikire@gmail.com)
wrote:
##sample log or input log
1583402931.976871 CCBAYr2KnmpaWDtxO2 xx.xx.xx.xx 65184 xx.xx.xx.xx 4200 tcp
- 1.855212 503 0 SH T T 0 ScADaF 5 715 2 80 -
1583402933.241900 C6C59e3TdNbeTTBZ7j xx.xx.xx.xx 16020 xx.xx.xx.xx 34032
tcp - 0.015988 2981 0 OTH T T 0 HcADC 6 352 0 0 -
##grok pattern that i used ( https://grokconstructor.appspot.com/groklib/bro)
BRO_CONN
%{NUMBER:ts}\t%{NOTSPACE:uid}\t%{IP:orig_h}\t%{INT:orig_p}\t%{IP:resp_h}\t%{INT:resp_p}\t%{WORD:proto}\t%{GREEDYDATA:service}\t%{NUMBER:duration}\t%{NUMBER:orig_bytes}\t%{NUMBER:resp_bytes}\t%{GREEDYDATA:conn_state}\t%{GREEDYDATA:local_orig}\t%{GREEDYDATA:missed_bytes}\t%{GREEDYDATA:history}\t%{GREEDYDATA:orig_pkts}\t%{GREEDYDATA:orig_ip_bytes}\t%{GREEDYDATA:resp_pkts}\t%{GREEDYDATA:resp_ip_bytes}\t%{GREEDYDATA:tunnel_parents}
##the error shown in metron-rest.log
Caused by: java.lang.IllegalStateException: Unable to parse Message:
1583402939.738024 CTGU7D24R7NL5eTGef xx.xx.xx.xx 50998 xx.xx.xx.xx 6188 tcp
- - - - OTH T T 0C 0 0 0 0 -
at
org.apache.metron.parsers.bro.BasicBroParser.parse(BasicBroParser.java:145)
~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at
org.apache.metron.parsers.interfaces.MessageParser.parseOptional(MessageParser.java:54)
~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at
org.apache.metron.parsers.interfaces.MessageParser.parseOptionalResult(MessageParser.java:67)
~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at
org.apache.metron.rest.service.impl.SensorParserConfigServiceImpl.parseMessage(SensorParserConfigServiceImpl.java:155)
~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
... 94 more
Caused by: org.json.simple.parser.ParseException
at org.json.simple.parser.Yylex.yylex(Yylex.java:610)
~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.nextToken(JSONParser.java:269)
~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.parse(JSONParser.java:118)
~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.parse(JSONParser.java:81)
~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.json.simple.parser.JSONParser.parse(JSONParser.java:75)
~[metron-rest-0.7.1.1.9.1.0-6.jar:?]
at org.apache.metron.parsers.bro.JSONCleaner.clean(JSONCleaner.java:49)
~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at
org.apache.metron.parsers.bro.BasicBroParser.parse(BasicBroParser.java:68)
~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at
org.apache.metron.parsers.interfaces.MessageParser.parseOptional(MessageParser.java:54)
~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
at
org.apache.metron.parsers.interfaces.MessageParser.parseOptionalResult(MessageParser.java:67)
~[metron-parsing-storm-0.7.1.1.9.1.0-6-uber.jar:?]
#i need your help as always.