You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Naveen Narayanasamy <Na...@cmail.carleton.ca> on 2017/07/31 15:21:06 UTC
Integration of Honeeepi(honeypot sensor) with Metron
Hello all,
I would like to know the possibility of integrating Honeeepi omponents(Cowrie and Dionaea) with Apache Metron.As there are limited information available online,I would also like to know the different integration procedures that can be tried.
For more info about the honeeepi, please visit the link below
https://redmine.honeynet.org/projects/honeeepi/wiki
Wiki - Honeeepi - Honeynet Project Redmine<https://redmine.honeynet.org/projects/honeeepi/wiki>
redmine.honeynet.org
Introduction¶ This project is about setting up honeypots with Raspberry Pi - a credit card sized ARM Linux box. !! Raspberry Pi¶ The Raspberry Pi is a credit sized ...
Any response will be appreciated!!
Naveen
Re: Integration of Honeeepi(honeypot sensor) with Metron
Posted by Naveen Narayanasamy <Na...@cmail.carleton.ca>.
Hello Matt Foley,
Thanks a lot for your recommendations!!
Yes,honeeepi generates log files and I will start by creating a grok script.I've gone through some of the links,thanks again.
Naveen
________________________________
From: Matt Foley <ma...@apache.org>
Sent: Tuesday, August 1, 2017 1:55:24 PM
To: user@metron.apache.org
Subject: Re: Integration of Honeeepi(honeypot sensor) with Metron
The “top three” changed, so here are my preferred references for writing new parsers:
* https://cwiki.apache.org/confluence/display/METRON/2016/04/25/Metron+Tutorial+-+Fundamentals+Part+1%3A+Creating+a+New+Telemetry
* https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
* https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html
From: Matt Foley <mf...@hortonworks.com> on behalf of Matt Foley <ma...@apache.org>
Date: Monday, July 31, 2017 at 2:39 PM
To: "user@metron.apache.org" <us...@metron.apache.org>
Subject: Re: Integration of Honeeepi(honeypot sensor) with Metron
Hi Naveen,
Does Honeeepi produce a stream of logs and/or alerts, that you would like to process?
If not, you’ll need to define a “sensor” of sorts that will tell you when something interesting happens (or is happening) with the honeypot. Metron does not help with that, although it can help compare normative with aberrational event streams, thereby identifying what is “interesting”, if Honeeepi itself does not do that. The integration point with Metron will be the message stream from Honeeepi or that Honeeepi sensor, preferably piped into Kafka.
Next you need a parser for the logs from Kafka. You may be able to write a Grok script for our generic Grok parser, otherwise you can write a Metron Parser module in Java. Parsers are in the process of becoming plug-ins for Metron, but for now, the current way of creating new parsers can be found in the top three results when you google “apache metron writing a new parser”. Parsers convert messages of whatever format into a standard JSON format, which the rest of Metron knows how to deal with.
Now you’ve got your “integration”. You still need to decide what to do with the message stream. If you need to identify “interesting” vs “not interesting” events, you might plug in an ML model as one of your enrichers. When you can filter for interesting events, you can “enrich” them by raising select info in the message body into the meta-data, or adding new meta-data based on associational lookups of existing fields.
Then you add Threat Intel and do threat recognition, and feed that into your Indexing and Alerting sub-systems. That’s Metron in a thimble :-) Suggest you read the entire site-book of Metron development information at https://metron.apache.org/current-book/index.html , especially the articles with architecture diagrams at
* https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html
* https://metron.apache.org/current-book/metron-platform/metron-enrichment/index.html
* https://metron.apache.org/current-book/metron-platform/metron-indexing/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-maas-service/index.html
and the stuff about Profiling and Statistics at
* https://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-profiler-client/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-statistics/index.html
Hope this helps,
--Matt
From: Naveen Narayanasamy <Na...@cmail.carleton.ca>
Reply-To: "user@metron.apache.org" <us...@metron.apache.org>
Date: Monday, July 31, 2017 at 8:21 AM
To: "user@metron.apache.org" <us...@metron.apache.org>
Subject: Integration of Honeeepi(honeypot sensor) with Metron
Hello all,
I would like to know the possibility of integrating Honeeepi omponents(Cowrie and Dionaea) with Apache Metron.As there are limited information available online,I would also like to know the different integration procedures that can be tried.
For more info about the honeeepi, please visit the link below
https://redmine.honeynet.org/projects/honeeepi/wiki
Wiki - Honeeepi - Honeynet Project Redmine<https://redmine.honeynet.org/projects/honeeepi/wiki>
redmine.honeynet.org
Introduction¶ This project is about setting up honeypots with Raspberry Pi - a credit card sized ARM Linux box. !! Raspberry Pi¶ The Raspberry Pi is a credit sized ...
Any response will be appreciated!!
Naveen
Re: Integration of Honeeepi(honeypot sensor) with Metron
Posted by Matt Foley <ma...@apache.org>.
The “top three” changed, so here are my preferred references for writing new parsers:
* https://cwiki.apache.org/confluence/display/METRON/2016/04/25/Metron+Tutorial+-+Fundamentals+Part+1%3A+Creating+a+New+Telemetry
* https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
* https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html
From: Matt Foley <mf...@hortonworks.com> on behalf of Matt Foley <ma...@apache.org>
Date: Monday, July 31, 2017 at 2:39 PM
To: "user@metron.apache.org" <us...@metron.apache.org>
Subject: Re: Integration of Honeeepi(honeypot sensor) with Metron
Hi Naveen,
Does Honeeepi produce a stream of logs and/or alerts, that you would like to process?
If not, you’ll need to define a “sensor” of sorts that will tell you when something interesting happens (or is happening) with the honeypot. Metron does not help with that, although it can help compare normative with aberrational event streams, thereby identifying what is “interesting”, if Honeeepi itself does not do that. The integration point with Metron will be the message stream from Honeeepi or that Honeeepi sensor, preferably piped into Kafka.
Next you need a parser for the logs from Kafka. You may be able to write a Grok script for our generic Grok parser, otherwise you can write a Metron Parser module in Java. Parsers are in the process of becoming plug-ins for Metron, but for now, the current way of creating new parsers can be found in the top three results when you google “apache metron writing a new parser”. Parsers convert messages of whatever format into a standard JSON format, which the rest of Metron knows how to deal with.
Now you’ve got your “integration”. You still need to decide what to do with the message stream. If you need to identify “interesting” vs “not interesting” events, you might plug in an ML model as one of your enrichers. When you can filter for interesting events, you can “enrich” them by raising select info in the message body into the meta-data, or adding new meta-data based on associational lookups of existing fields.
Then you add Threat Intel and do threat recognition, and feed that into your Indexing and Alerting sub-systems. That’s Metron in a thimble :-) Suggest you read the entire site-book of Metron development information at https://metron.apache.org/current-book/index.html , especially the articles with architecture diagrams at
* https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html
* https://metron.apache.org/current-book/metron-platform/metron-enrichment/index.html
* https://metron.apache.org/current-book/metron-platform/metron-indexing/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-maas-service/index.html
and the stuff about Profiling and Statistics at
* https://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-profiler-client/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-statistics/index.html
Hope this helps,
--Matt
From: Naveen Narayanasamy <Na...@cmail.carleton.ca>
Reply-To: "user@metron.apache.org" <us...@metron.apache.org>
Date: Monday, July 31, 2017 at 8:21 AM
To: "user@metron.apache.org" <us...@metron.apache.org>
Subject: Integration of Honeeepi(honeypot sensor) with Metron
Hello all,
I would like to know the possibility of integrating Honeeepi omponents(Cowrie and Dionaea) with Apache Metron.As there are limited information available online,I would also like to know the different integration procedures that can be tried.
For more info about the honeeepi, please visit the link below
https://redmine.honeynet.org/projects/honeeepi/wiki
Wiki - Honeeepi - Honeynet Project Redmine
redmine.honeynet.org
Introduction¶ This project is about setting up honeypots with Raspberry Pi - a credit card sized ARM Linux box. !! Raspberry Pi¶ The Raspberry Pi is a credit sized ...
Any response will be appreciated!!
Naveen
Re: Integration of Honeeepi(honeypot sensor) with Metron
Posted by Matt Foley <ma...@apache.org>.
Hi Naveen,
Does Honeeepi produce a stream of logs and/or alerts, that you would like to process?
If not, you’ll need to define a “sensor” of sorts that will tell you when something interesting happens (or is happening) with the honeypot. Metron does not help with that, although it can help compare normative with aberrational event streams, thereby identifying what is “interesting”, if Honeeepi itself does not do that. The integration point with Metron will be the message stream from Honeeepi or that Honeeepi sensor, preferably piped into Kafka.
Next you need a parser for the logs from Kafka. You may be able to write a Grok script for our generic Grok parser, otherwise you can write a Metron Parser module in Java. Parsers are in the process of becoming plug-ins for Metron, but for now, the current way of creating new parsers can be found in the top three results when you google “apache metron writing a new parser”. Parsers convert messages of whatever format into a standard JSON format, which the rest of Metron knows how to deal with.
Now you’ve got your “integration”. You still need to decide what to do with the message stream. If you need to identify “interesting” vs “not interesting” events, you might plug in an ML model as one of your enrichers. When you can filter for interesting events, you can “enrich” them by raising select info in the message body into the meta-data, or adding new meta-data based on associational lookups of existing fields.
Then you add Threat Intel and do threat recognition, and feed that into your Indexing and Alerting sub-systems. That’s Metron in a thimble :-) Suggest you read the entire site-book of Metron development information at https://metron.apache.org/current-book/index.html , especially the articles with architecture diagrams at
* https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html
* https://metron.apache.org/current-book/metron-platform/metron-enrichment/index.html
* https://metron.apache.org/current-book/metron-platform/metron-indexing/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-maas-service/index.html
and the stuff about Profiling and Statistics at
* https://metron.apache.org/current-book/metron-analytics/metron-profiler/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-profiler-client/index.html
* https://metron.apache.org/current-book/metron-analytics/metron-statistics/index.html
Hope this helps,
--Matt
From: Naveen Narayanasamy <Na...@cmail.carleton.ca>
Reply-To: "user@metron.apache.org" <us...@metron.apache.org>
Date: Monday, July 31, 2017 at 8:21 AM
To: "user@metron.apache.org" <us...@metron.apache.org>
Subject: Integration of Honeeepi(honeypot sensor) with Metron
Hello all,
I would like to know the possibility of integrating Honeeepi omponents(Cowrie and Dionaea) with Apache Metron.As there are limited information available online,I would also like to know the different integration procedures that can be tried.
For more info about the honeeepi, please visit the link below
https://redmine.honeynet.org/projects/honeeepi/wiki
Wiki - Honeeepi - Honeynet Project Redmine
redmine.honeynet.org
Introduction¶ This project is about setting up honeypots with Raspberry Pi - a credit card sized ARM Linux box. !! Raspberry Pi¶ The Raspberry Pi is a credit sized ...
Any response will be appreciated!!
Naveen