You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@metron.apache.org by updates on tube <ab...@gmail.com> on 2019/11/25 14:12:55 UTC

ingesting syslog and asa log into metron

hey guys first I really appreciate your urgent replies on my previous posts >>
and for now, I went to ask how can I ingest Syslog and asa log into apache metron using nifi?

Re: ingesting syslog and asa log into metron

Posted by Simon Elliston Ball <si...@simonellistonball.com>.

If you find any missing patterns you should also consider contributing them
back to the open source project.

https://metron.apache.org/current-book/CONTRIBUTING.html

Simon

On Mon, 25 Nov 2019 at 15:00, Hema malini <nh...@gmail.com> wrote:

> After enabling the parsers, kindly check for the patterns missed out and
> add grok patterns based on the log messages.
>
>
>
> On Mon, 25 Nov, 2019, 7:44 PM Simon Elliston Ball, <
> simon@simonellistonball.com> wrote:
>
>> Use the nifi listen syslog processor to push Asa logs into a Kafka topic,
>> then the metron asa parser to get that into your metron flow.
>>
>> Simon
>>
>> On Mon, 25 Nov 2019 at 14:12, updates on tube <ab...@gmail.com>
>> wrote:
>>
>>> hey guys first I really appreciate your urgent replies on my previous
>>> posts >>
>>> and for now, I went to ask how can I ingest Syslog and asa log into
>>> apache metron using nifi?
>>>
>> --
>> --
>> simon elliston ball
>> @sireb
>>
> --
--
simon elliston ball
@sireb

Re: ingesting syslog and asa log into metron

Posted by Hema malini <nh...@gmail.com>.

After enabling the parsers, kindly check for the patterns missed out and
add grok patterns based on the log messages.



On Mon, 25 Nov, 2019, 7:44 PM Simon Elliston Ball, <
simon@simonellistonball.com> wrote:

> Use the nifi listen syslog processor to push Asa logs into a Kafka topic,
> then the metron asa parser to get that into your metron flow.
>
> Simon
>
> On Mon, 25 Nov 2019 at 14:12, updates on tube <ab...@gmail.com>
> wrote:
>
>> hey guys first I really appreciate your urgent replies on my previous
>> posts >>
>> and for now, I went to ask how can I ingest Syslog and asa log into
>> apache metron using nifi?
>>
> --
> --
> simon elliston ball
> @sireb
>

Re: ingesting syslog and asa log into metron

Posted by updates on tube <ab...@gmail.com>.


On 2019/12/23 06:37:19, updates on tube <ab...@gmail.com> wrote: 
> 
> 
> On 2019/11/25 14:14:38, Simon Elliston Ball <si...@simonellistonball.com> wrote: 
> > Use the nifi listen syslog processor to push Asa logs into a Kafka topic,
> > then the metron asa parser to get that into your metron flow.
> > 
> > Simon
> > 
> > On Mon, 25 Nov 2019 at 14:12, updates on tube <ab...@gmail.com>
> > wrote:
> > 
> > > hey guys first I really appreciate your urgent replies on my previous
> > > posts >>
> > > and for now, I went to ask how can I ingest Syslog and asa log into apache
> > > metron using nifi?
> > >
> > -- 
> > --
> > simon elliston ball
> > @sireb
> > 
> i was trying to stream rsyslog log data to apache metron using asa parser. the log look like down below 

2019-12-20T07:06:41-05:00 ab TESTING: Fri 20 Dec 2019 07:06:41 AM EST
 the log 2019-12-20T07:06:41-05:00 ab rsyslogd: action 'action-13-builtin:omfwd' resumed (module 'builtin:omfwd') [v8.1911.0 try https://www.rsyslog.com/e/2359 ]
2019-12-20T07:08:04-05:00 ab TESTING: Fri 20 Dec 2019 07:08:04 AM EST
2019-12-20T07:08:05-05:00 ab TESTING: Fri 20 Dec 2019 07:08:05 AM EST
2019-12-20T07:08:06-05:00 ab TESTING: Fri 20 Dec 2019 07:08:06 AM EST
2019-12-20T07:08:06-05:00 ab TESTING: Fri 20 Dec 2019 07:08:06 AM EST
2019-12-20T07:08:08-05:00 ab TESTING: Fri 20 Dec 2019 07:08:08 AM EST
2019-12-20T07:08:08-05:00 ab TESTING: Fri 20 Dec 2019 07:08:08 AM EST
2019-12-20T07:08:09-05:00 ab TESTING: Fri 20 Dec 2019 07:08:09 AM EST
2019-12-20T07:08:09-05:00 ab TESTING: Fri 20 Dec 2019 07:08:09 AM EST
2019-12-20T07:09:01-05:00 ab CRON[3174]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:09:01-05:00 ab CRON[3175]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
2019-12-20T07:09:01-05:00 ab CRON[3174]: pam_unix(cron:session): session closed for user root
2019-12-20T07:09:01-05:00 ab systemd[1]: Starting Clean php session files...
2019-12-20T07:09:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded.
2019-12-20T07:09:01-05:00 ab systemd[1]: Started Clean php session files.
2019-12-20T07:10:04-05:00 ab TESTING: Fri 20 Dec 2019 07:10:04 AM EST
2019-12-20T07:10:05-05:00 ab TESTING: Fri 20 Dec 2019 07:10:05 AM EST
2019-12-20T07:10:05-05:00 ab TESTING: Fri 20 Dec 2019 07:10:05 AM EST
2019-12-20T07:10:06-05:00 ab TESTING: Fri 20 Dec 2019 07:10:06 AM EST
2019-12-20T07:10:07-05:00 ab TESTING: Fri 20 Dec 2019 07:10:07 AM EST
2019-12-20T07:10:07-05:00 ab TESTING: Fri 20 Dec 2019 07:10:07 AM EST
2019-12-20T07:10:08-05:00 ab TESTING: Fri 20 Dec 2019 07:10:08 AM EST
2019-12-20T07:10:08-05:00 ab TESTING: Fri 20 Dec 2019 07:10:08 AM EST
2019-12-20T07:10:09-05:00 ab TESTING: Fri 20 Dec 2019 07:10:09 AM EST
2019-12-20T07:10:09-05:00 ab TESTING: Fri 20 Dec 2019 07:10:09 AM EST
2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST
2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST
2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST
2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST
2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST
2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST
2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST
2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST
2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST
2019-12-20T07:10:13-05:00 ab TESTING: Fri 20 Dec 2019 07:10:13 AM EST
2019-12-20T07:10:13-05:00 ab TESTING: Fri 20 Dec 2019 07:10:13 AM EST
2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST
2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST
2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST
2019-12-20T07:10:15-05:00 ab systemd[1]: Stopping System Logging Service...
2019-12-20T07:10:15-05:00 ab rsyslogd: [origin software="rsyslogd" swVersion="8.1911.0" x-pid="3071" x-info="https://www.rsyslog.com"] exiting on signal 15.
2019-12-20T07:10:15-05:00 ab systemd[1]: rsyslog.service: Succeeded.
2019-12-20T07:10:15-05:00 ab systemd[1]: Stopped System Logging Service.
2019-12-20T07:10:15-05:00 ab systemd[1]: Starting System Logging Service...
2019-12-20T07:10:15-05:00 ab rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd.  [v8.1911.0]
2019-12-20T07:10:15-05:00 ab rsyslogd: [origin software="rsyslogd" swVersion="8.1911.0" x-pid="3270" x-info="https://www.rsyslog.com"] start
2019-12-20T07:10:15-05:00 ab systemd[1]: Started System Logging Service.
2019-12-20T07:10:18-05:00 ab TESTING: Fri 20 Dec 2019 07:10:18 AM EST
2019-12-20T07:15:01-05:00 ab CRON[3283]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:15:01-05:00 ab CRON[3284]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2019-12-20T07:15:01-05:00 ab CRON[3283]: pam_unix(cron:session): session closed for user root
2019-12-20T07:17:01-05:00 ab CRON[3323]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:17:01-05:00 ab CRON[3324]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
2019-12-20T07:17:01-05:00 ab CRON[3323]: pam_unix(cron:session): session closed for user root
2019-12-20T07:25:01-05:00 ab CRON[3333]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:25:01-05:00 ab CRON[3334]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2019-12-20T07:25:01-05:00 ab CRON[3333]: pam_unix(cron:session): session closed for user root
2019-12-20T07:29:38-05:00 ab snapd[666]: storehelpers.go:436: cannot refresh: snap has no updates available: "barrier", "barrier-kvm", "gtk-common-themes", "notepad-plus-plus", "snapd", "wine-platform-3-stable"
2019-12-20T07:34:26-05:00 ab smartd[665]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 67 to 66
2019-12-20T07:34:26-05:00 ab smartd[665]: Device: /dev/sda [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 110 to 109
2019-12-20T07:35:01-05:00 ab CRON[3450]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:35:01-05:00 ab CRON[3451]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2019-12-20T07:35:01-05:00 ab CRON[3450]: pam_unix(cron:session): session closed for user root
2019-12-20T07:39:01-05:00 ab CRON[3460]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:39:01-05:00 ab CRON[3461]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
2019-12-20T07:39:01-05:00 ab CRON[3460]: pam_unix(cron:session): session closed for user root
2019-12-20T07:39:01-05:00 ab systemd[1]: Starting Clean php session files...
2019-12-20T07:39:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded.
2019-12-20T07:39:01-05:00 ab systemd[1]: Started Clean php session files.
2019-12-20T07:45:01-05:00 ab CRON[3525]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:45:01-05:00 ab CRON[3526]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2019-12-20T07:45:01-05:00 ab CRON[3525]: pam_unix(cron:session): session closed for user root
2019-12-20T07:55:01-05:00 ab CRON[3549]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T07:55:01-05:00 ab CRON[3550]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2019-12-20T07:55:01-05:00 ab CRON[3549]: pam_unix(cron:session): session closed for user root
2019-12-20T08:05:01-05:00 ab CRON[3575]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T08:05:01-05:00 ab CRON[3576]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2019-12-20T08:05:01-05:00 ab CRON[3575]: pam_unix(cron:session): session closed for user root
2019-12-20T08:09:01-05:00 ab CRON[3586]: pam_unix(cron:session): session opened for user root by (uid=0)
2019-12-20T08:09:01-05:00 ab CRON[3587]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
2019-12-20T08:09:01-05:00 ab CRON[3586]: pam_unix(cron:session): session closed for user root
2019-12-20T08:09:01-05:00 ab systemd[1]: Starting Clean php session files...
2019-12-20T08:09:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded.
2019-12-20T08:09:01-05:00 ab systemd[1]: Started Clean php session files







		THIS IS THE ERROR FOUND IN STORM UI parserBolt

java.lang.RuntimeException: [Metron] Message '2019-12-20T07:06:41-05:00 ab TESTING: Fri 20 Dec 2019 07:06:41 AM EST 2019-12-20T07:06:41-05:00 ab rsyslogd: action 'action-13-builtin:omfwd' resumed (module 'builtin:omfwd') [v8.1911.0 try https://www.rsyslog.com/e/2359 ] 2019-12-20T07:08:04-05:00 ab TESTING: Fri 20 Dec 2019 07:08:04 AM EST 2019-12-20T07:08:05-05:00 ab TESTING: Fri 20 Dec 2019 07:08:05 AM EST 2019-12-20T07:08:06-05:00 ab TESTING: Fri 20 Dec 2019 07:08:06 AM EST 2019-12-20T07:08:06-05:00 ab TESTING: Fri 20 Dec 2019 07:08:06 AM EST 2019-12-20T07:08:08-05:00 ab TESTING: Fri 20 Dec 2019 07:08:08 AM EST 2019-12-20T07:08:08-05:00 ab TESTING: Fri 20 Dec 2019 07:08:08 AM EST 2019-12-20T07:08:09-05:00 ab TESTING: Fri 20 Dec 2019 07:08:09 AM EST 2019-12-20T07:08:09-05:00 ab TESTING: Fri 20 Dec 2019 07:08:09 AM EST 2019-12-20T07:09:01-05:00 ab CRON[3174]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:09:01-05:00 ab CRON[3175]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi) 2019-12-20T07:09:01-05:00 ab CRON[3174]: pam_unix(cron:session): session closed for user root 2019-12-20T07:09:01-05:00 ab systemd[1]: Starting Clean php session files... 2019-12-20T07:09:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded. 2019-12-20T07:09:01-05:00 ab systemd[1]: Started Clean php session files. 2019-12-20T07:10:04-05:00 ab TESTING: Fri 20 Dec 2019 07:10:04 AM EST 2019-12-20T07:10:05-05:00 ab TESTING: Fri 20 Dec 2019 07:10:05 AM EST 2019-12-20T07:10:05-05:00 ab TESTING: Fri 20 Dec 2019 07:10:05 AM EST 2019-12-20T07:10:06-05:00 ab TESTING: Fri 20 Dec 2019 07:10:06 AM EST 2019-12-20T07:10:07-05:00 ab TESTING: Fri 20 Dec 2019 07:10:07 AM EST 2019-12-20T07:10:07-05:00 ab TESTING: Fri 20 Dec 2019 07:10:07 AM EST 2019-12-20T07:10:08-05:00 ab TESTING: Fri 20 Dec 2019 07:10:08 AM EST 2019-12-20T07:10:08-05:00 ab TESTING: Fri 20 Dec 2019 07:10:08 AM EST 2019-12-20T07:10:09-05:00 ab TESTING: Fri 20 Dec 2019 07:10:09 AM EST 2019-12-20T07:10:09-05:00 ab TESTING: Fri 20 Dec 2019 07:10:09 AM EST 2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST 2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST 2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST 2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST 2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST 2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST 2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST 2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST 2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST 2019-12-20T07:10:13-05:00 ab TESTING: Fri 20 Dec 2019 07:10:13 AM EST 2019-12-20T07:10:13-05:00 ab TESTING: Fri 20 Dec 2019 07:10:13 AM EST 2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST 2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST 2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST 2019-12-20T07:10:15-05:00 ab systemd[1]: Stopping System Logging Service... 2019-12-20T07:10:15-05:00 ab rsyslogd: [origin software="rsyslogd" swVersion="8.1911.0" x-pid="3071" x-info="https://www.rsyslog.com"] exiting on signal 15. 2019-12-20T07:10:15-05:00 ab systemd[1]: rsyslog.service: Succeeded. 2019-12-20T07:10:15-05:00 ab systemd[1]: Stopped System Logging Service. 2019-12-20T07:10:15-05:00 ab systemd[1]: Starting System Logging Service... 2019-12-20T07:10:15-05:00 ab rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd. [v8.1911.0] 2019-12-20T07:10:15-05:00 ab rsyslogd: [origin software="rsyslogd" swVersion="8.1911.0" x-pid="3270" x-info="https://www.rsyslog.com"] start 2019-12-20T07:10:15-05:00 ab systemd[1]: Started System Logging Service. 2019-12-20T07:10:18-05:00 ab TESTING: Fri 20 Dec 2019 07:10:18 AM EST 2019-12-20T07:15:01-05:00 ab CRON[3283]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:15:01-05:00 ab CRON[3284]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:15:01-05:00 ab CRON[3283]: pam_unix(cron:session): session closed for user root 2019-12-20T07:17:01-05:00 ab CRON[3323]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:17:01-05:00 ab CRON[3324]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 2019-12-20T07:17:01-05:00 ab CRON[3323]: pam_unix(cron:session): session closed for user root 2019-12-20T07:25:01-05:00 ab CRON[3333]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:25:01-05:00 ab CRON[3334]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:25:01-05:00 ab CRON[3333]: pam_unix(cron:session): session closed for user root 2019-12-20T07:29:38-05:00 ab snapd[666]: storehelpers.go:436: cannot refresh: snap has no updates available: "barrier", "barrier-kvm", "gtk-common-themes", "notepad-plus-plus", "snapd", "wine-platform-3-stable" 2019-12-20T07:34:26-05:00 ab smartd[665]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 67 to 66 2019-12-20T07:34:26-05:00 ab smartd[665]: Device: /dev/sda [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 110 to 109 2019-12-20T07:35:01-05:00 ab CRON[3450]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:35:01-05:00 ab CRON[3451]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:35:01-05:00 ab CRON[3450]: pam_unix(cron:session): session closed for user root 2019-12-20T07:39:01-05:00 ab CRON[3460]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:39:01-05:00 ab CRON[3461]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi) 2019-12-20T07:39:01-05:00 ab CRON[3460]: pam_unix(cron:session): session closed for user root 2019-12-20T07:39:01-05:00 ab systemd[1]: Starting Clean php session files... 2019-12-20T07:39:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded. 2019-12-20T07:39:01-05:00 ab systemd[1]: Started Clean php session files. 2019-12-20T07:45:01-05:00 ab CRON[3525]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:45:01-05:00 ab CRON[3526]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:45:01-05:00 ab CRON[3525]: pam_unix(cron:session): session closed for user root 2019-12-20T07:55:01-05:00 ab CRON[3549]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:55:01-05:00 ab CRON[3550]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:55:01-05:00 ab CRON[3549]: pam_unix(cron:session): session closed for user root 2019-12-20T08:05:01-05:00 ab CRON[3575]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T08:05:01-05:00 ab CRON[3576]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T08:05:01-05:00 ab CRON[3575]: pam_unix(cron:session): session closed for user root 2019-12-20T08:09:01-05:00 ab CRON[3586]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T08:09:01-05:00 ab CRON[3587]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi) 2019-12-20T08:09:01-05:00 ab CRON[3586]: pam_unix(cron:session): session closed for user root 2019-12-20T08:09:01-05:00 ab systemd[1]: Starting Clean php session files... 2019-12-20T08:09:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded. 2019-12-20T08:09:01-05:00 ab systemd[1]: Started Clean php session files. ' does not match pattern '%{CISCO_TAGGED_SYSLOG}' at org.apache.metron.parsers.asa.BasicAsaParser.parse(BasicAsaParser.java:184) at org.apache.metron.parsers.interfaces.MessageParser.parseOptional(MessageParser.java:54) at org.apache.metron.parsers.interfaces.MessageParser.parseOptionalResult(MessageParser.java:67) at org.apache.metron.parsers.ParserRunnerImpl.execute(ParserRunnerImpl.java:144) at org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:257) at org.apache.storm.daemon.executor$fn__10195$tuple_action_fn__10197.invoke(executor.clj:735) at org.apache.storm.daemon.executor$mk_task_receiver$fn__10114.invoke(executor.clj:466) at org.apache.storm.disruptor$clojure_handler$reify__4137.onEvent(disruptor.clj:40) at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) at org.apache.storm.daemon.executor$fn__10195$fn__10208$fn__10263.invoke(executor.clj:855) at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: [Metron] Message '2019-12-20T07:06:41-05:00 ab TESTING: Fri 20 Dec 2019 07:06:41 AM EST 2019-12-20T07:06:41-05:00 ab rsyslogd: action 'action-13-builtin:omfwd' resumed (module 'builtin:omfwd') [v8.1911.0 try https://www.rsyslog.com/e/2359 ] 2019-12-20T07:08:04-05:00 ab TESTING: Fri 20 Dec 2019 07:08:04 AM EST 2019-12-20T07:08:05-05:00 ab TESTING: Fri 20 Dec 2019 07:08:05 AM EST 2019-12-20T07:08:06-05:00 ab TESTING: Fri 20 Dec 2019 07:08:06 AM EST 2019-12-20T07:08:06-05:00 ab TESTING: Fri 20 Dec 2019 07:08:06 AM EST 2019-12-20T07:08:08-05:00 ab TESTING: Fri 20 Dec 2019 07:08:08 AM EST 2019-12-20T07:08:08-05:00 ab TESTING: Fri 20 Dec 2019 07:08:08 AM EST 2019-12-20T07:08:09-05:00 ab TESTING: Fri 20 Dec 2019 07:08:09 AM EST 2019-12-20T07:08:09-05:00 ab TESTING: Fri 20 Dec 2019 07:08:09 AM EST 2019-12-20T07:09:01-05:00 ab CRON[3174]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:09:01-05:00 ab CRON[3175]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi) 2019-12-20T07:09:01-05:00 ab CRON[3174]: pam_unix(cron:session): session closed for user root 2019-12-20T07:09:01-05:00 ab systemd[1]: Starting Clean php session files... 2019-12-20T07:09:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded. 2019-12-20T07:09:01-05:00 ab systemd[1]: Started Clean php session files. 2019-12-20T07:10:04-05:00 ab TESTING: Fri 20 Dec 2019 07:10:04 AM EST 2019-12-20T07:10:05-05:00 ab TESTING: Fri 20 Dec 2019 07:10:05 AM EST 2019-12-20T07:10:05-05:00 ab TESTING: Fri 20 Dec 2019 07:10:05 AM EST 2019-12-20T07:10:06-05:00 ab TESTING: Fri 20 Dec 2019 07:10:06 AM EST 2019-12-20T07:10:07-05:00 ab TESTING: Fri 20 Dec 2019 07:10:07 AM EST 2019-12-20T07:10:07-05:00 ab TESTING: Fri 20 Dec 2019 07:10:07 AM EST 2019-12-20T07:10:08-05:00 ab TESTING: Fri 20 Dec 2019 07:10:08 AM EST 2019-12-20T07:10:08-05:00 ab TESTING: Fri 20 Dec 2019 07:10:08 AM EST 2019-12-20T07:10:09-05:00 ab TESTING: Fri 20 Dec 2019 07:10:09 AM EST 2019-12-20T07:10:09-05:00 ab TESTING: Fri 20 Dec 2019 07:10:09 AM EST 2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST 2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST 2019-12-20T07:10:10-05:00 ab TESTING: Fri 20 Dec 2019 07:10:10 AM EST 2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST 2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST 2019-12-20T07:10:11-05:00 ab TESTING: Fri 20 Dec 2019 07:10:11 AM EST 2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST 2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST 2019-12-20T07:10:12-05:00 ab TESTING: Fri 20 Dec 2019 07:10:12 AM EST 2019-12-20T07:10:13-05:00 ab TESTING: Fri 20 Dec 2019 07:10:13 AM EST 2019-12-20T07:10:13-05:00 ab TESTING: Fri 20 Dec 2019 07:10:13 AM EST 2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST 2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST 2019-12-20T07:10:14-05:00 ab TESTING: Fri 20 Dec 2019 07:10:14 AM EST 2019-12-20T07:10:15-05:00 ab systemd[1]: Stopping System Logging Service... 2019-12-20T07:10:15-05:00 ab rsyslogd: [origin software="rsyslogd" swVersion="8.1911.0" x-pid="3071" x-info="https://www.rsyslog.com"] exiting on signal 15. 2019-12-20T07:10:15-05:00 ab systemd[1]: rsyslog.service: Succeeded. 2019-12-20T07:10:15-05:00 ab systemd[1]: Stopped System Logging Service. 2019-12-20T07:10:15-05:00 ab systemd[1]: Starting System Logging Service... 2019-12-20T07:10:15-05:00 ab rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd. [v8.1911.0] 2019-12-20T07:10:15-05:00 ab rsyslogd: [origin software="rsyslogd" swVersion="8.1911.0" x-pid="3270" x-info="https://www.rsyslog.com"] start 2019-12-20T07:10:15-05:00 ab systemd[1]: Started System Logging Service. 2019-12-20T07:10:18-05:00 ab TESTING: Fri 20 Dec 2019 07:10:18 AM EST 2019-12-20T07:15:01-05:00 ab CRON[3283]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:15:01-05:00 ab CRON[3284]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:15:01-05:00 ab CRON[3283]: pam_unix(cron:session): session closed for user root 2019-12-20T07:17:01-05:00 ab CRON[3323]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:17:01-05:00 ab CRON[3324]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 2019-12-20T07:17:01-05:00 ab CRON[3323]: pam_unix(cron:session): session closed for user root 2019-12-20T07:25:01-05:00 ab CRON[3333]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:25:01-05:00 ab CRON[3334]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:25:01-05:00 ab CRON[3333]: pam_unix(cron:session): session closed for user root 2019-12-20T07:29:38-05:00 ab snapd[666]: storehelpers.go:436: cannot refresh: snap has no updates available: "barrier", "barrier-kvm", "gtk-common-themes", "notepad-plus-plus", "snapd", "wine-platform-3-stable" 2019-12-20T07:34:26-05:00 ab smartd[665]: Device: /dev/sda [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 67 to 66 2019-12-20T07:34:26-05:00 ab smartd[665]: Device: /dev/sda [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 110 to 109 2019-12-20T07:35:01-05:00 ab CRON[3450]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:35:01-05:00 ab CRON[3451]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:35:01-05:00 ab CRON[3450]: pam_unix(cron:session): session closed for user root 2019-12-20T07:39:01-05:00 ab CRON[3460]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:39:01-05:00 ab CRON[3461]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi) 2019-12-20T07:39:01-05:00 ab CRON[3460]: pam_unix(cron:session): session closed for user root 2019-12-20T07:39:01-05:00 ab systemd[1]: Starting Clean php session files... 2019-12-20T07:39:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded. 2019-12-20T07:39:01-05:00 ab systemd[1]: Started Clean php session files. 2019-12-20T07:45:01-05:00 ab CRON[3525]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:45:01-05:00 ab CRON[3526]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:45:01-05:00 ab CRON[3525]: pam_unix(cron:session): session closed for user root 2019-12-20T07:55:01-05:00 ab CRON[3549]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T07:55:01-05:00 ab CRON[3550]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T07:55:01-05:00 ab CRON[3549]: pam_unix(cron:session): session closed for user root 2019-12-20T08:05:01-05:00 ab CRON[3575]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T08:05:01-05:00 ab CRON[3576]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) 2019-12-20T08:05:01-05:00 ab CRON[3575]: pam_unix(cron:session): session closed for user root 2019-12-20T08:09:01-05:00 ab CRON[3586]: pam_unix(cron:session): session opened for user root by (uid=0) 2019-12-20T08:09:01-05:00 ab CRON[3587]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi) 2019-12-20T08:09:01-05:00 ab CRON[3586]: pam_unix(cron:session): session closed for user root 2019-12-20T08:09:01-05:00 ab systemd[1]: Starting Clean php session files... 2019-12-20T08:09:01-05:00 ab systemd[1]: phpsessionclean.service: Succeeded. 2019-12-20T08:09:01-05:00 ab systemd[1]: Started Clean php session files. ' does not match pattern '%{CISCO_TAGGED_SYSLOG}' at org.apache.metron.parsers.asa.BasicAsaParser.parse(BasicAsaParser.java:178) ... 14 more

i need your help???? as always

Re: ingesting syslog and asa log into metron

Posted by updates on tube <ab...@gmail.com>.


On 2019/11/25 14:14:38, Simon Elliston Ball <si...@simonellistonball.com> wrote: 
> Use the nifi listen syslog processor to push Asa logs into a Kafka topic,
> then the metron asa parser to get that into your metron flow.
> 
> Simon
> 
> On Mon, 25 Nov 2019 at 14:12, updates on tube <ab...@gmail.com>
> wrote:
> 
> > hey guys first I really appreciate your urgent replies on my previous
> > posts >>
> > and for now, I went to ask how can I ingest Syslog and asa log into apache
> > metron using nifi?
> >
> -- 
> --
> simon elliston ball
> @sireb
>

Re: ingesting syslog and asa log into metron

Posted by Simon Elliston Ball <si...@simonellistonball.com>.

Use the nifi listen syslog processor to push Asa logs into a Kafka topic,
then the metron asa parser to get that into your metron flow.

Simon

On Mon, 25 Nov 2019 at 14:12, updates on tube <ab...@gmail.com>
wrote:

> hey guys first I really appreciate your urgent replies on my previous
> posts >>
> and for now, I went to ask how can I ingest Syslog and asa log into apache
> metron using nifi?
>
-- 
--
simon elliston ball
@sireb