You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by updates on tube <ab...@gmail.com> on 2020/02/24 13:41:48 UTC
linux-syslog(centos 7) parsing in apache metron error
i get such error on kibana dashboard no error in storm
com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0 no viable alternative at input 'F'
at com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
at org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
at org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
at com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
at com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
at org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
at org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
at org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
at org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.antlr.v4.runtime.NoViableAltException
at org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
... 18 more
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by updates on tube <ab...@gmail.com>.
it worked!!! thank you so much..
On 2020/02/27 22:37:12, Otto Fowler <ot...@gmail.com> wrote:
> org.apache.metron.parsers.syslog.Syslog3164Parser
> is the classname.
>
> You have confused me with your description.
>
> 1st. The exception you show, the error points to you using some version of
> a syslog parser.
>
> 2nd. You only talk about using grok after that.
>
> I have tried your sample string with the above parser and it works.
>
> On February 27, 2020 at 09:19:08, updates on tube (abrahamfikire@gmail.com)
> wrote:
>
> but i can't get the parser?
>
> On 2020/02/27 12:13:35, Otto Fowler <ot...@gmail.com> wrote: br/>>
> Parsing this messages works with the Syslog31164Parser. Maybe you could
> > use that.
> > br/>> On FFebruary 27, 2020 at 02:03:50, updates on tube (
> abrahamfikire@gmail.com)
> > wrote:
> > br/>> br/>> ############## I really apriciate your quick responses..
> please tell us the
> > valid grok patterns for such kind of log ####################
> > # this is my parser configuration
> > {
> > "parserClassName": "org.apache.metron.parsers.GrokParser",
> > "sensorTopic": "linuxsyslog",
> > "parserConfig": {
> > "grokPath": "/apps/metron/patterns/linuxsyslog",
> > "patternLabel": "SYSLOGBASE2",
> > "timestampField": "timestamp"
> > },
> > br/>> ""fieldTransformations" : [
> > br/>> {{
> > br/>> ""transformation" : "STELLAR"
> > ,"output" : [ "full_hostname", "domain_without_subdomains" ]
> > ,"config" : {
> > "full_hostname" : "URL_TO_HOST(url)"
> > ,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
> > }
> > }
> > ]
> > br/>> }}
> > br/>> ## this is my grok pattern
> > (?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601})
> > (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}
> > br/>> br/>> ##this is the sample log that couse cause error br/> FFeb 16
> 08:00:23
> > myhostname NetworkManager[1686]: <info> [1581858023.4306] dhcp4 (eth0):
> > address xxx.xxx.xxx.xxx
> > Feb 16 08:00:23 myhostname dhclient[1710]: DHCPREQUEST on eth0 to
> > xxx.xxx.xxx.xxx port 67 (xid=0x170e0b99)
> > br/>> br/>> ##this is the error message found in kibana
> > Syntax error @ 1:0 no viable alternative at input 'F'
> > br/>> ## detail error found in kibana shows as follow
> > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0
> no
> > viable alternative at input 'F'
> > at
> >
> com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
>
> > br/>> at <
> >
> org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
>
> > br/>> at
> org.antlr.v4.runtime.Parser.notifyErrorListeeners(Parser.java:558)
> > at
> >
> org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
>
> > br/>> at <
> >
> org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
>
> > br/>> at <
> >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
>
> > br/>> at <
> >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
>
> > br/>> at <
> >
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
>
> > br/>> at <
> >
> com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
>
> > br/>> at java.util.ArrayList.forEach(ArrayList.java:11249)
> > at
> >
> com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
>
> > br/>> at <
> >
> org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
>
> > br/>> at <
> >
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
>
> > br/>> at
> org.apache.metron.parsers.bolt.ParserBolt.exxecute(ParserBolt.java:257)
> > at
> >
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
>
> > br/>> at <
> >
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
>
> > br/>> at <
> >
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
>
> > br/>> at <
> >
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
>
> > br/>> at <
> >
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
>
> > br/>> at <
> >
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>
> > br/>> at <
> >
> org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
>
> > br/>> at org.apache.storm.util$$async_loop$fn__1221.invoke(util.clj:484)
> > at clojure.lang.AFn.run(AFn.java:22)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: org.antlr.v4.runtime.NoViableAltException
> > at
> >
> org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
>
> > br/>> at <
> >
> org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
>
> > br/>> at <
> >
> org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
>
> > br/>> at <
> >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
>
> > br/>> ... 18 more <
> > br/> br/> <
> > br/>> On 2020/02/24 19:31:36, Michael Miklavcic <
> mmichael.miklavcic@gmail.com>
> > wrote: br/>> That's how we route errors. Looks like the syslog parser had
> > trouble with
> > > one of your syslog messages
> > > br/>> On Mon, FFeb 24, 2020, 5:41 AM updates on tube <
> > abrahamfikire@gmail.com>
> > > wrote:
> > > br/>> > i get such error on kibana dashboard no errror in storm
> > > > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @
> 1:0
> > no
> > > > viable alternative at input 'F'
> > > > at
> > > >
> >
> com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
>
> > br/>> > > at <
> > > >
> >
> org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
>
> > br/>> > > at <
> > > > org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> > > > at
> > > >
> >
> org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
>
> > br/>> > > at <
> > > >
> >
> org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
>
> > br/>> > > at <
> > > >
> >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
>
> > br/>> > > at <
> > > >
> >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
>
> > br/>> > > at <
> > > >
> >
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
>
> > br/>> > > at <
> > > >
> >
> com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
>
> > br/>> > > at java.util.ArrayList.forEach(ArrayLList.java:1249)
> > > > at
> > > >
> >
> com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
>
> > br/>> > > at <
> > > >
> >
> org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
>
> > br/>> > > at <
> > > >
> >
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
>
> > br/>> > > at <
> > > >
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> > > > at
> > > >
> >
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
>
> > br/>> > > at <
> > > >
> >
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
>
> > br/>> > > at <
> > > >
> >
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
>
> > br/>> > > at <
> > > >
> >
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
>
> > br/>> > > at <
> > > >
> >
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
>
> > br/>> > > at <
> > > >
> >
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>
> > br/>> > > at <
> > > >
> >
> org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
>
> > br/>> > > at
> org.apache.storm.util$$async_loop$fn__1221.invoke(util.clj:484)
> > > > at clojure.lang.AFn.run(AFn.java:22)
> > > > at java.lang.Thread.run(Thread.java:745)
> > > > Caused by: org.antlr.v4.runtime.NoViableAltException
> > > > at
> > > >
> >
> org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
>
> > br/>> > > at <
> > > >
> >
> org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
>
> > br/>> > > at <
> > > >
> >
> org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
>
> > br/>> > > at <
> > > >
> >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
>
> > br/>> > > ... 18 more <
> > > >
> > > >
> > > >
> > > >
> > > br/>
> > br/>> ; <
> > br/>
>
> b
>
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by Otto Fowler <ot...@gmail.com>.
org.apache.metron.parsers.syslog.Syslog3164Parser
is the classname.
You have confused me with your description.
1st. The exception you show, the error points to you using some version of
a syslog parser.
2nd. You only talk about using grok after that.
I have tried your sample string with the above parser and it works.
On February 27, 2020 at 09:19:08, updates on tube (abrahamfikire@gmail.com)
wrote:
but i can't get the parser?
On 2020/02/27 12:13:35, Otto Fowler <ot...@gmail.com> wrote: br/>>
Parsing this messages works with the Syslog31164Parser. Maybe you could
> use that.
> br/>> On FFebruary 27, 2020 at 02:03:50, updates on tube (
abrahamfikire@gmail.com)
> wrote:
> br/>> br/>> ############## I really apriciate your quick responses..
please tell us the
> valid grok patterns for such kind of log ####################
> # this is my parser configuration
> {
> "parserClassName": "org.apache.metron.parsers.GrokParser",
> "sensorTopic": "linuxsyslog",
> "parserConfig": {
> "grokPath": "/apps/metron/patterns/linuxsyslog",
> "patternLabel": "SYSLOGBASE2",
> "timestampField": "timestamp"
> },
> br/>> ""fieldTransformations" : [
> br/>> {{
> br/>> ""transformation" : "STELLAR"
> ,"output" : [ "full_hostname", "domain_without_subdomains" ]
> ,"config" : {
> "full_hostname" : "URL_TO_HOST(url)"
> ,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
> }
> }
> ]
> br/>> }}
> br/>> ## this is my grok pattern
> (?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601})
> (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}
> br/>> br/>> ##this is the sample log that couse cause error br/> FFeb 16
08:00:23
> myhostname NetworkManager[1686]: <info> [1581858023.4306] dhcp4 (eth0):
> address xxx.xxx.xxx.xxx
> Feb 16 08:00:23 myhostname dhclient[1710]: DHCPREQUEST on eth0 to
> xxx.xxx.xxx.xxx port 67 (xid=0x170e0b99)
> br/>> br/>> ##this is the error message found in kibana
> Syntax error @ 1:0 no viable alternative at input 'F'
> br/>> ## detail error found in kibana shows as follow
> com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0
no
> viable alternative at input 'F'
> at
>
com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
> br/>> at <
>
org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
> br/>> at
org.antlr.v4.runtime.Parser.notifyErrorListeeners(Parser.java:558)
> at
>
org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
> br/>> at <
>
org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
> br/>> at <
>
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
> br/>> at <
>
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> br/>> at <
>
com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
> br/>> at <
>
com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
> br/>> at java.util.ArrayList.forEach(ArrayList.java:11249)
> at
>
com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
> br/>> at <
>
org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
> br/>> at <
>
org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
> br/>> at
org.apache.metron.parsers.bolt.ParserBolt.exxecute(ParserBolt.java:257)
> at
>
org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> br/>> at <
>
org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> br/>> at <
>
org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> br/>> at <
>
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> br/>> at <
>
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> br/>> at <
>
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> br/>> at <
>
org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> br/>> at org.apache.storm.util$$async_loop$fn__1221.invoke(util.clj:484)
> at clojure.lang.AFn.run(AFn.java:22)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.antlr.v4.runtime.NoViableAltException
> at
>
org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
> br/>> at <
>
org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
> br/>> at <
>
org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
> br/>> at <
>
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
> br/>> ... 18 more <
> br/> br/> <
> br/>> On 2020/02/24 19:31:36, Michael Miklavcic <
mmichael.miklavcic@gmail.com>
> wrote: br/>> That's how we route errors. Looks like the syslog parser had
> trouble with
> > one of your syslog messages
> > br/>> On Mon, FFeb 24, 2020, 5:41 AM updates on tube <
> abrahamfikire@gmail.com>
> > wrote:
> > br/>> > i get such error on kibana dashboard no errror in storm
> > > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @
1:0
> no
> > > viable alternative at input 'F'
> > > at
> > >
>
com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
> br/>> > > at <
> > >
>
org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
> br/>> > > at <
> > > org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> > > at
> > >
>
org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
> br/>> > > at <
> > >
>
org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
> br/>> > > at <
> > >
>
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
> br/>> > > at <
> > >
>
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> br/>> > > at <
> > >
>
com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
> br/>> > > at <
> > >
>
com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
> br/>> > > at java.util.ArrayList.forEach(ArrayLList.java:1249)
> > > at
> > >
>
com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
> br/>> > > at <
> > >
>
org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
> br/>> > > at <
> > >
>
org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
> br/>> > > at <
> > >
org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> > > at
> > >
>
org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> br/>> > > at <
> > >
>
org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> br/>> > > at <
> > >
>
org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> br/>> > > at <
> > >
>
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> br/>> > > at <
> > >
>
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> br/>> > > at <
> > >
>
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> br/>> > > at <
> > >
>
org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> br/>> > > at
org.apache.storm.util$$async_loop$fn__1221.invoke(util.clj:484)
> > > at clojure.lang.AFn.run(AFn.java:22)
> > > at java.lang.Thread.run(Thread.java:745)
> > > Caused by: org.antlr.v4.runtime.NoViableAltException
> > > at
> > >
>
org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
> br/>> > > at <
> > >
>
org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
> br/>> > > at <
> > >
>
org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
> br/>> > > at <
> > >
>
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
> br/>> > > ... 18 more <
> > >
> > >
> > >
> > >
> > br/>
> br/>> ; <
> br/>
b
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by updates on tube <ab...@gmail.com>.
but i can't get the parser?
On 2020/02/27 12:13:35, Otto Fowler <ot...@gmail.com> wrote:
> Parsing this messages works with the Syslog3164Parser. Maybe you could
> use that.
>
> On February 27, 2020 at 02:03:50, updates on tube (abrahamfikire@gmail.com)
> wrote:
>
>
> ############# I really apriciate your quick responses.. please tell us the
> valid grok patterns for such kind of log ####################
> # this is my parser configuration
> {
> "parserClassName": "org.apache.metron.parsers.GrokParser",
> "sensorTopic": "linuxsyslog",
> "parserConfig": {
> "grokPath": "/apps/metron/patterns/linuxsyslog",
> "patternLabel": "SYSLOGBASE2",
> "timestampField": "timestamp"
> },
>
> "fieldTransformations" : [
>
> {
>
> "transformation" : "STELLAR"
> ,"output" : [ "full_hostname", "domain_without_subdomains" ]
> ,"config" : {
> "full_hostname" : "URL_TO_HOST(url)"
> ,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
> }
> }
> ]
>
> }
>
> # this is my grok pattern
> (?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601})
> (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}
>
>
> #this is the sample log that couse cause error br/> FFeb 16 08:00:23
> myhostname NetworkManager[1686]: <info> [1581858023.4306] dhcp4 (eth0):
> address xxx.xxx.xxx.xxx
> Feb 16 08:00:23 myhostname dhclient[1710]: DHCPREQUEST on eth0 to
> xxx.xxx.xxx.xxx port 67 (xid=0x170e0b99)
>
>
> #this is the error message found in kibana
> Syntax error @ 1:0 no viable alternative at input 'F'
>
> # detail error found in kibana shows as follow
> com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0 no
> viable alternative at input 'F'
> at
> com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
>
> at
> org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
>
> at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> at
> org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
>
> at
> org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
>
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
>
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
>
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
>
> at
> com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
>
> at java.util.ArrayList.forEach(ArrayList.java:1249)
> at
> com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
>
> at
> org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
>
> at
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
>
> at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> at
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
>
> at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
>
> at
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
>
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
>
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
>
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>
> at
> org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
>
> at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> at clojure.lang.AFn.run(AFn.java:22)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.antlr.v4.runtime.NoViableAltException
> at
> org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
>
> at
> org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
>
> at
> org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
>
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
>
> ... 18 more
> br/> br/> <
>
> On 2020/02/24 19:31:36, Michael Miklavcic <mi...@gmail.com>
> wrote: br/>> That's how we route errors. Looks like the syslog parser had
> trouble with
> > one of your syslog messages
> > br/>> On Mon, FFeb 24, 2020, 5:41 AM updates on tube <
> abrahamfikire@gmail.com>
> > wrote:
> > br/>> > i get such error on kibana dashboard no errror in storm
> > > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0
> no
> > > viable alternative at input 'F'
> > > at
> > >
> com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
>
> > > at
> > >
> org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
>
> > > at
> > > org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> > > at
> > >
> org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
>
> > > at
> > >
> org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
>
> > > at
> > >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
>
> > > at
> > >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
>
> > > at
> > >
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
>
> > > at
> > >
> com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
>
> > > at java.util.ArrayList.forEach(ArrayList.java:1249)
> > > at
> > >
> com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
>
> > > at
> > >
> org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
>
> > > at
> > >
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
>
> > > at
> > > org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> > > at
> > >
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
>
> > > at
> > >
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
>
> > > at
> > >
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
>
> > > at
> > >
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
>
> > > at
> > >
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
>
> > > at
> > >
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
>
> > > at
> > >
> org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
>
> > > at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> > > at clojure.lang.AFn.run(AFn.java:22)
> > > at java.lang.Thread.run(Thread.java:745)
> > > Caused by: org.antlr.v4.runtime.NoViableAltException
> > > at
> > >
> org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
>
> > > at
> > >
> org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
>
> > > at
> > >
> org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
>
> > > at
> > >
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
>
> > > ... 18 more
> > >
> > >
> > >
> > >
> > br/>
>
> ;
>
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by Otto Fowler <ot...@gmail.com>.
Parsing this messages works with the Syslog3164Parser. Maybe you could
use that.
On February 27, 2020 at 02:03:50, updates on tube (abrahamfikire@gmail.com)
wrote:
############# I really apriciate your quick responses.. please tell us the
valid grok patterns for such kind of log ####################
# this is my parser configuration
{
"parserClassName": "org.apache.metron.parsers.GrokParser",
"sensorTopic": "linuxsyslog",
"parserConfig": {
"grokPath": "/apps/metron/patterns/linuxsyslog",
"patternLabel": "SYSLOGBASE2",
"timestampField": "timestamp"
},
"fieldTransformations" : [
{
"transformation" : "STELLAR"
,"output" : [ "full_hostname", "domain_without_subdomains" ]
,"config" : {
"full_hostname" : "URL_TO_HOST(url)"
,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
}
}
]
}
# this is my grok pattern
(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601})
(?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}
#this is the sample log that couse cause error br/> FFeb 16 08:00:23
myhostname NetworkManager[1686]: <info> [1581858023.4306] dhcp4 (eth0):
address xxx.xxx.xxx.xxx
Feb 16 08:00:23 myhostname dhclient[1710]: DHCPREQUEST on eth0 to
xxx.xxx.xxx.xxx port 67 (xid=0x170e0b99)
#this is the error message found in kibana
Syntax error @ 1:0 no viable alternative at input 'F'
# detail error found in kibana shows as follow
com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0 no
viable alternative at input 'F'
at
com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
at
org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
at
org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
at
org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
at
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
at
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
at
com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
at
com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at
com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
at
org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
at
org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
at
org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
at
org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
at
org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
at
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
at
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
at
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
at
org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.antlr.v4.runtime.NoViableAltException
at
org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
at
org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
at
org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
at
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
... 18 more
br/> br/> <
On 2020/02/24 19:31:36, Michael Miklavcic <mi...@gmail.com>
wrote: br/>> That's how we route errors. Looks like the syslog parser had
trouble with
> one of your syslog messages
> br/>> On Mon, FFeb 24, 2020, 5:41 AM updates on tube <
abrahamfikire@gmail.com>
> wrote:
> br/>> > i get such error on kibana dashboard no errror in storm
> > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0
no
> > viable alternative at input 'F'
> > at
> >
com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
> > at
> >
org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
> > at
> > org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> > at
> >
org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
> > at
> >
org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
> > at
> >
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
> > at
> >
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> > at
> >
com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
> > at
> >
com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
> > at java.util.ArrayList.forEach(ArrayList.java:1249)
> > at
> >
com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
> > at
> >
org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
> > at
> >
org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
> > at
> > org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> > at
> >
org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> > at
> >
org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> > at
> >
org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> > at
> >
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> > at
> >
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> > at
> >
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> > at
> >
org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> > at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> > at clojure.lang.AFn.run(AFn.java:22)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: org.antlr.v4.runtime.NoViableAltException
> > at
> >
org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
> > at
> >
org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
> > at
> >
org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
> > at
> >
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
> > ... 18 more
> >
> >
> >
> >
> br/>
;
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by updates on tube <ab...@gmail.com>.
############# I really apriciate your quick responses.. please tell us the valid grok patterns for such kind of log ####################
# this is my parser configuration
{
"parserClassName": "org.apache.metron.parsers.GrokParser",
"sensorTopic": "linuxsyslog",
"parserConfig": {
"grokPath": "/apps/metron/patterns/linuxsyslog",
"patternLabel": "SYSLOGBASE2",
"timestampField": "timestamp"
},
"fieldTransformations" : [
{
"transformation" : "STELLAR"
,"output" : [ "full_hostname", "domain_without_subdomains" ]
,"config" : {
"full_hostname" : "URL_TO_HOST(url)"
,"domain_without_subdomains" : "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
}
}
]
}
# this is my grok pattern
(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}
#this is the sample log that couse cause error
Feb 16 08:00:23 myhostname NetworkManager[1686]: <info> [1581858023.4306] dhcp4 (eth0): address xxx.xxx.xxx.xxx
Feb 16 08:00:23 myhostname dhclient[1710]: DHCPREQUEST on eth0 to xxx.xxx.xxx.xxx port 67 (xid=0x170e0b99)
#this is the error message found in kibana
Syntax error @ 1:0 no viable alternative at input 'F'
# detail error found in kibana shows as follow
com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0 no viable alternative at input 'F'
at com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
at org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
at org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
at com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
at com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
at org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
at org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
at org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
at org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.antlr.v4.runtime.NoViableAltException
at org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
at com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
... 18 more
On 2020/02/24 19:31:36, Michael Miklavcic <mi...@gmail.com> wrote:
> That's how we route errors. Looks like the syslog parser had trouble with
> one of your syslog messages
>
> On Mon, Feb 24, 2020, 5:41 AM updates on tube <ab...@gmail.com>
> wrote:
>
> > i get such error on kibana dashboard no error in storm
> > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0 no
> > viable alternative at input 'F'
> > at
> > com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
> > at
> > org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
> > at
> > org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> > at
> > org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
> > at
> > org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
> > at
> > com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
> > at
> > com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> > at
> > com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
> > at
> > com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
> > at java.util.ArrayList.forEach(ArrayList.java:1249)
> > at
> > com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
> > at
> > org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
> > at
> > org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
> > at
> > org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> > at
> > org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> > at
> > org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> > at
> > org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> > at
> > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> > at
> > org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> > at
> > org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> > at
> > org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> > at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> > at clojure.lang.AFn.run(AFn.java:22)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: org.antlr.v4.runtime.NoViableAltException
> > at
> > org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
> > at
> > org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
> > at
> > org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
> > at
> > com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
> > ... 18 more
> >
> >
> >
> >
>
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by Otto Fowler <ot...@gmail.com>.
Can you provide an example of a syslog line that fails? Clean of personal
data of course.
Also what is your parser configuration?
On February 25, 2020 at 01:05:00, updates on tube (abrahamfikire@gmail.com)
wrote:
On 2020/02/24 19:31:36, Michael Miklavcic <mi...@gmail.com>
wrote: br/>> That's how we route errors. Looks like the syslog parser had
trouble with
> one of your syslog messages
> br/>> On Mon, FFeb 24, 2020, 5:41 AM updates on tube <
abrahamfikire@gmail.com>
> wrote:
> br/>> > i get such error on kibana dashboard no errror in storm
> > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0
no
> > viable alternative at input 'F'
> > at
> >
com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
> > at
> >
org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
> > at
> > org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> > at
> >
org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
> > at
> >
org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
> > at
> >
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
> > at
> >
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> > at
> >
com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
> > at
> >
com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
> > at java.util.ArrayList.forEach(ArrayList.java:1249)
> > at
> >
com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
> > at
> >
org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
> > at
> >
org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
> > at
> > org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> > at
> >
org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> > at
> >
org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> > at
> >
org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> > at
> >
org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> > at
> >
org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> > at
> >
org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> > at
> >
org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> > at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> > at clojure.lang.AFn.run(AFn.java:22)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: org.antlr.v4.runtime.NoViableAltException
> > at
> >
org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
> > at
> >
org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
> > at
> >
org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
> > at
> >
com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
> > ... 18 more
> >
> >
> >
> >
okay so my log file look like this found in /var/log/messages centos os 7
Feb 25 00:54:55 master3 dbus[1615]: [system] Successfully activated service
'org.freedesktop.nm_dispatcher'
Feb 25 00:54:55 master3 systemd: Started Network Manager Script Dispatcher
Service.
Feb 25 00:54:55 master3 nm-dispatcher: req:1 'dhcp4-change' [eth0]: new
request (5 scripts)
Feb 25 00:54:55 master3 nm-dispatcher: req:1 'dhcp4-change' [eth0]: start
running ordered scripts...
Feb 25 00:55:23 master3 su: (to root) root on none
Feb 25 00:55:23 master3 systemd: Started Session c212834 of user root.
Feb 25 00:55:28 master3 su: (to kibana) root on none
Feb 25 00:55:28 master3 systemd: Created slice User Slice of kibana.
Feb 25 00:55:28 master3 systemd: Started Session c212835 of user kibana.
Feb 25 00:55:28 master3 /etc/init.d/kibana: kibana is running
Feb 25 00:55:28 master3 systemd: Removed slice User Slice of kibana.
Feb 25 00:55:39 master3 su: (to metron) root on none
and i use parser as follow that works in http://grokdebug.herokuapp.com/
but not in metron;
(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601})
(?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}
br/>what should I do??
br/>
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by updates on tube <ab...@gmail.com>.
On 2020/02/24 19:31:36, Michael Miklavcic <mi...@gmail.com> wrote:
> That's how we route errors. Looks like the syslog parser had trouble with
> one of your syslog messages
>
> On Mon, Feb 24, 2020, 5:41 AM updates on tube <ab...@gmail.com>
> wrote:
>
> > i get such error on kibana dashboard no error in storm
> > com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0 no
> > viable alternative at input 'F'
> > at
> > com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
> > at
> > org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
> > at
> > org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> > at
> > org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
> > at
> > org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
> > at
> > com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
> > at
> > com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> > at
> > com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
> > at
> > com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
> > at java.util.ArrayList.forEach(ArrayList.java:1249)
> > at
> > com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
> > at
> > org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
> > at
> > org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
> > at
> > org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> > at
> > org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> > at
> > org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> > at
> > org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> > at
> > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> > at
> > org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> > at
> > org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> > at
> > org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> > at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> > at clojure.lang.AFn.run(AFn.java:22)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: org.antlr.v4.runtime.NoViableAltException
> > at
> > org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
> > at
> > org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
> > at
> > org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
> > at
> > com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
> > ... 18 more
> >
> >
> >
> >
okay so my log file look like this found in /var/log/messages centos os 7
Feb 25 00:54:55 master3 dbus[1615]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Feb 25 00:54:55 master3 systemd: Started Network Manager Script Dispatcher Service.
Feb 25 00:54:55 master3 nm-dispatcher: req:1 'dhcp4-change' [eth0]: new request (5 scripts)
Feb 25 00:54:55 master3 nm-dispatcher: req:1 'dhcp4-change' [eth0]: start running ordered scripts...
Feb 25 00:55:23 master3 su: (to root) root on none
Feb 25 00:55:23 master3 systemd: Started Session c212834 of user root.
Feb 25 00:55:28 master3 su: (to kibana) root on none
Feb 25 00:55:28 master3 systemd: Created slice User Slice of kibana.
Feb 25 00:55:28 master3 systemd: Started Session c212835 of user kibana.
Feb 25 00:55:28 master3 /etc/init.d/kibana: kibana is running
Feb 25 00:55:28 master3 systemd: Removed slice User Slice of kibana.
Feb 25 00:55:39 master3 su: (to metron) root on none
and i use parser as follow that works in http://grokdebug.herokuapp.com/ but not in metron;
(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}
what should I do?
Re: linux-syslog(centos 7) parsing in apache metron error
Posted by Michael Miklavcic <mi...@gmail.com>.
That's how we route errors. Looks like the syslog parser had trouble with
one of your syslog messages
On Mon, Feb 24, 2020, 5:41 AM updates on tube <ab...@gmail.com>
wrote:
> i get such error on kibana dashboard no error in storm
> com.github.palindromicity.syslog.dsl.ParseException: Syntax error @ 1:0 no
> viable alternative at input 'F'
> at
> com.github.palindromicity.syslog.dsl.DefaultErrorListener.syntaxError(DefaultErrorListener.java:33)
> at
> org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:65)
> at
> org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:558)
> at
> org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
> at
> org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:147)
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:412)
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.syslog_msg(Rfc5424Parser.java:273)
> at
> com.github.palindromicity.syslog.Rfc5424SyslogParser.parseLine(Rfc5424SyslogParser.java:66)
> at
> com.github.palindromicity.syslog.AbstractSyslogParser.lambda$parseLines$0(AbstractSyslogParser.java:144)
> at java.util.ArrayList.forEach(ArrayList.java:1249)
> at
> com.github.palindromicity.syslog.AbstractSyslogParser.parseLines(AbstractSyslogParser.java:142)
> at
> org.apache.metron.parsers.syslog.BaseSyslogParser.parseOptionalResult(BaseSyslogParser.java:116)
> at
> org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144)
> at
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257)
> at
> org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735)
> at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466)
> at
> org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40)
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472)
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451)
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> at
> org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855)
> at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484)
> at clojure.lang.AFn.run(AFn.java:22)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.antlr.v4.runtime.NoViableAltException
> at
> org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:1894)
> at
> org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:498)
> at
> org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:424)
> at
> com.github.palindromicity.syslog.dsl.generated.Rfc5424Parser.header(Rfc5424Parser.java:373)
> ... 18 more
>
>
>
>