You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Farrukh Naveed Anjum <an...@gmail.com> on 2017/10/25 04:55:19 UTC
SysLog Parser in Metron
Hi,
How can I get syslog in metron any help (pattern / parser). Kindly help ?
--
With Regards
Farrukh Naveed Anjum
Re: SysLog Parser in Metron
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Thanks, it was helpful
On Wed, Oct 25, 2017 at 7:29 PM, Ahmed Shah <Ah...@cmail.carleton.ca>
wrote:
> Hello Farrukh,
>
>
> Our team was able to report simple Dionaea alerts to Metron using syslog
> v8 (not encrypted).
>
>
>
> The source code for our project is here:
>
> https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/
>
> More specifically... syslog config files for our honeypots are here:
>
> https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/tree/master/
> SampleLogFiles/configForHP-notEnc
>
>
> More specifically... syslog config files for the Metron server are
> here:
>
> https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/tree/master/
> SampleLogFiles/configForServer-notEnc
>
>
> GROK parser pattern used:
>
> https://raw.githubusercontent.com/LTW-GCR-CSOC/csoc-
> installation-scripts/master/images/Dionaea-ManagementUI.png
>
>
> https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/blob/master/
> SampleLogFiles/README.md
>
>
>
>
> Nifi setup in Metron Server:
>
> https://raw.githubusercontent.com/LTW-GCR-CSOC/csoc-
> installation-scripts/master/images/nifiDionaeaKafka.png
>
>
>
>
> Hope it helps.
>
>
> -Ahmed
> _______________________________________________________________
> Ahmed Shah (PMP, M. Eng.)
> Cybersecurity Analyst & Developer
> GCR - Cybersecurity Operations Center
> Carleton University - cugcr.com <https://cugcr.com/tiki/lce/index.php>
>
>
> ------------------------------
> *From:* Simon Elliston Ball <si...@simonellistonball.com>
> *Sent:* October 25, 2017 3:47 AM
> *To:* user@metron.apache.org
> *Subject:* Re: SysLog Parser in Metron
>
> Short answer: grok parsers.
>
> Longer answer: syslog is more a transport, not just a log format, so it
> encapsulates a wide variety of data sources. Your best bet is probably to
> use NiFi to listen for syslog from a remote host (ListenSyslog) and then
> route each application in the syslog to a different kafka topic. That way
> you have kafka topics for each type of data you care about eg sshd, login,
> cups... whatever. From there it’s easiest to use a grok parser in metron to
> pull out the fields. There are many prebuilt patterns for the common
> services around on the web.
>
> Simon
>
> > On 25 Oct 2017, at 05:55, Farrukh Naveed Anjum <an...@gmail.com>
> wrote:
> >
> > Hi,
> >
> > How can I get syslog in metron any help (pattern / parser). Kindly help ?
> >
> > --
> > With Regards
> > Farrukh Naveed Anjum
>
--
With Regards
Farrukh Naveed Anjum
Re: SysLog Parser in Metron
Posted by Ahmed Shah <Ah...@cmail.carleton.ca>.
Hello Farrukh,
Our team was able to report simple Dionaea alerts to Metron using syslog v8 (not encrypted).
The source code for our project is here:
https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/
More specifically... syslog config files for our honeypots are here:
https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/tree/master/SampleLogFiles/configForHP-notEnc
More specifically... syslog config files for the Metron server are here:
https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/tree/master/SampleLogFiles/configForServer-notEnc
GROK parser pattern used:
https://raw.githubusercontent.com/LTW-GCR-CSOC/csoc-installation-scripts/master/images/Dionaea-ManagementUI.png
[https://raw.githubusercontent.com/LTW-GCR-CSOC/csoc-installation-scripts/master/images/Dionaea-ManagementUI.png]
https://github.com/LTW-GCR-CSOC/csoc-installation-scripts/blob/master/SampleLogFiles/README.md
Nifi setup in Metron Server:
https://raw.githubusercontent.com/LTW-GCR-CSOC/csoc-installation-scripts/master/images/nifiDionaeaKafka.png
[https://raw.githubusercontent.com/LTW-GCR-CSOC/csoc-installation-scripts/master/images/nifiDionaeaKafka.png]
Hope it helps.
-Ahmed
_______________________________________________________________
Ahmed Shah (PMP, M. Eng.)
Cybersecurity Analyst & Developer
GCR - Cybersecurity Operations Center
Carleton University - cugcr.com<https://cugcr.com/tiki/lce/index.php>
________________________________
From: Simon Elliston Ball <si...@simonellistonball.com>
Sent: October 25, 2017 3:47 AM
To: user@metron.apache.org
Subject: Re: SysLog Parser in Metron
Short answer: grok parsers.
Longer answer: syslog is more a transport, not just a log format, so it encapsulates a wide variety of data sources. Your best bet is probably to use NiFi to listen for syslog from a remote host (ListenSyslog) and then route each application in the syslog to a different kafka topic. That way you have kafka topics for each type of data you care about eg sshd, login, cups... whatever. From there it’s easiest to use a grok parser in metron to pull out the fields. There are many prebuilt patterns for the common services around on the web.
Simon
> On 25 Oct 2017, at 05:55, Farrukh Naveed Anjum <an...@gmail.com> wrote:
>
> Hi,
>
> How can I get syslog in metron any help (pattern / parser). Kindly help ?
>
> --
> With Regards
> Farrukh Naveed Anjum
Re: SysLog Parser in Metron
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
Short answer: grok parsers.
Longer answer: syslog is more a transport, not just a log format, so it encapsulates a wide variety of data sources. Your best bet is probably to use NiFi to listen for syslog from a remote host (ListenSyslog) and then route each application in the syslog to a different kafka topic. That way you have kafka topics for each type of data you care about eg sshd, login, cups... whatever. From there it’s easiest to use a grok parser in metron to pull out the fields. There are many prebuilt patterns for the common services around on the web.
Simon
> On 25 Oct 2017, at 05:55, Farrukh Naveed Anjum <an...@gmail.com> wrote:
>
> Hi,
>
> How can I get syslog in metron any help (pattern / parser). Kindly help ?
>
> --
> With Regards
> Farrukh Naveed Anjum