You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Syed Hammad Tahir <ms...@itu.edu.pk> on 2017/09/20 08:04:33 UTC
System Requrements
Hello,
What would be the system required in order to run metron and analyzy a LAN
environment of almost 100 nodes using single node full development
depoloyment.
Regards.
Re: System Requrements
Posted by Matt Foley <ma...@apache.org>.
Agree with Jon you might be able to get away with a single-node, at least it will be functional enough to let you experiment and find out if you need more. However, even for an experimental system I strongly recommend you expand to 16GB of RAM, minimum. (Remember, as a test platform, full-dev expects you’ll turn off resource sinks like Elasticsearch when you’re not testing them.) And you’ll probably want more than one virtual core pretty quick. But try it out.
--Matt
From: "Zeolla@GMail.com" <ze...@gmail.com>
Reply-To: "user@metron.apache.org" <us...@metron.apache.org>
Date: Wednesday, September 20, 2017 at 11:20 AM
To: "user@metron.apache.org" <us...@metron.apache.org>
Subject: Re: System Requrements
Sounds like a pretty low volume environment. You should be good with something the size of full-dev (I think 8GB of RAM, 1vCPU, more details here). Especially if it's a VM, you should be able to expand it somewhat easily if needed.
Give it a shot and let me know how it works - happy to work with you to figure out any quirks with this install.
Jon
On Wed, Sep 20, 2017 at 7:32 AM Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
1- The nodes are endpoints (desktops and laptops connected in lan and using shared internet)
2- They are behind NAT
3- They are for one primary user each.
4- These nodes are deployed in our university labs so there is no internet exposed service.
On Wed, Sep 20, 2017 at 3:55 PM, Zeolla@GMail.com <ze...@gmail.com> wrote:
Okay, so I have some more questions then, but I'm still not sure how helpful I can be. Maybe someone else with a similar environment can chime in.
These nodes, are they servers or endpoints (laptop/desktops used for productivity - internet use, email, etc.)? Are they behind network firewalls or NAT, or are they exposed? Are they shared machines or one primary user each? If there are any internet exposed services, what are they?
Jon
On Wed, Sep 20, 2017, 06:50 Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
Actually I need to forward the specs for my IT department as soon as possible, I was thinking to get a rough idea.
Regards.
On Wed, Sep 20, 2017 at 3:43 PM, Zeolla@GMail.com <ze...@gmail.com> wrote:
This is very much something Metron can do, but scoping hardware requires more detail about the data and work to be done on the data. I would focus on setting up the sensors (custom IDS, snort) and then either gather metrics and scope Metron or just spin it up by default/with whatever you have and see how it works.
Jon
On Wed, Sep 20, 2017, 06:23 Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
Hi,
1- I want to focus more on real time analysis but lets say we start with pcap dump, I dont know at this point that how much data it can dump in 24hr period given the lan environment of 100 nodes. You can assert your assumption to answer.
2- Snort data most probably and dont know about the nukber of events yes. You can also assert your assumption here for a hypothetical scenerio to guide me.
3- I want to build an intrusion detection system and apply some machine learning algorithm on it so Guess profiling is the answer to the third question.
Based on those partial answers and your insight into this domain, kindly reply with most suitable solution with assumptions where necessary.
If you think that I am expecting something from metron which it cant do then kindly let me know.
Regards
Regards.
On Wed, Sep 20, 2017 at 3:11 PM, Zeolla@GMail.com <ze...@gmail.com> wrote:
Full dev is intended for testing, not actual use. That said, to answer your question it is more important to know (1) will you be storing pcap, (1b) if so, how much per day and for how long, (2) what data will you be sending into Metron (bro, yaf, snort, asa, etc.) and how many events per second is it, and (3) what are you planning to do with the data (profiling, MaaS, enrichments, etc.)?
Jon
On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
Hello,
What would be the system required in order to run metron and analyzy a LAN environment of almost 100 nodes using single node full development depoloyment.
Regards.
--
Jon
--
Jon
--
Jon
--
Jon
Re: System Requrements
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Sounds like a pretty low volume environment. You should be good with
something the size of full-dev (I think 8GB of RAM, 1vCPU, more details here
<https://github.com/apache/metron/blob/master/metron-deployment/packaging/packer-build/base-centos-6.7.json>).
Especially if it's a VM, you should be able to expand it somewhat easily if
needed.
Give it a shot and let me know how it works - happy to work with you to
figure out any quirks with this install.
Jon
On Wed, Sep 20, 2017 at 7:32 AM Syed Hammad Tahir <ms...@itu.edu.pk>
wrote:
> 1- The nodes are endpoints (desktops and laptops connected in lan and
> using shared internet)
> 2- They are behind NAT
> 3- They are for one primary user each.
> 4- These nodes are deployed in our university labs so there is no internet
> exposed service.
>
> On Wed, Sep 20, 2017 at 3:55 PM, Zeolla@GMail.com <ze...@gmail.com>
> wrote:
>
>> Okay, so I have some more questions then, but I'm still not sure how
>> helpful I can be. Maybe someone else with a similar environment can chime
>> in.
>>
>> These nodes, are they servers or endpoints (laptop/desktops used for
>> productivity - internet use, email, etc.)? Are they behind network
>> firewalls or NAT, or are they exposed? Are they shared machines or one
>> primary user each? If there are any internet exposed services, what are
>> they?
>>
>> Jon
>>
>> On Wed, Sep 20, 2017, 06:50 Syed Hammad Tahir <ms...@itu.edu.pk>
>> wrote:
>>
>>> Actually I need to forward the specs for my IT department as soon as
>>> possible, I was thinking to get a rough idea.
>>> Regards.
>>>
>>> On Wed, Sep 20, 2017 at 3:43 PM, Zeolla@GMail.com <ze...@gmail.com>
>>> wrote:
>>>
>>>> This is very much something Metron can do, but scoping hardware
>>>> requires more detail about the data and work to be done on the data. I
>>>> would focus on setting up the sensors (custom IDS, snort) and then either
>>>> gather metrics and scope Metron or just spin it up by default/with whatever
>>>> you have and see how it works.
>>>>
>>>> Jon
>>>>
>>>> On Wed, Sep 20, 2017, 06:23 Syed Hammad Tahir <ms...@itu.edu.pk>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> 1- I want to focus more on real time analysis but lets say we start
>>>>> with pcap dump, I dont know at this point that how much data it can dump in
>>>>> 24hr period given the lan environment of 100 nodes. You can assert your
>>>>> assumption to answer.
>>>>>
>>>>> 2- Snort data most probably and dont know about the nukber of events
>>>>> yes. You can also assert your assumption here for a hypothetical scenerio
>>>>> to guide me.
>>>>>
>>>>> 3- I want to build an intrusion detection system and apply some
>>>>> machine learning algorithm on it so Guess profiling is the answer to the
>>>>> third question.
>>>>>
>>>>> Based on those partial answers and your insight into this domain,
>>>>> kindly reply with most suitable solution with assumptions where necessary.
>>>>>
>>>>> If you think that I am expecting something from metron which it cant
>>>>> do then kindly let me know.
>>>>>
>>>>> Regards
>>>>>
>>>>> Regards.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Sep 20, 2017 at 3:11 PM, Zeolla@GMail.com <ze...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Full dev is intended for testing, not actual use. That said, to
>>>>>> answer your question it is more important to know (1) will you be storing
>>>>>> pcap, (1b) if so, how much per day and for how long, (2) what data will you
>>>>>> be sending into Metron (bro, yaf, snort, asa, etc.) and how many events per
>>>>>> second is it, and (3) what are you planning to do with the data (profiling,
>>>>>> MaaS, enrichments, etc.)?
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>> On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> What would be the system required in order to run metron and analyzy
>>>>>>> a LAN environment of almost 100 nodes using single node full development
>>>>>>> depoloyment.
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>> --
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>
>>>>> --
>>>>
>>>> Jon
>>>>
>>>
>>> --
>>
>> Jon
>>
>
> --
Jon
Re: System Requrements
Posted by Syed Hammad Tahir <ms...@itu.edu.pk>.
1- The nodes are endpoints (desktops and laptops connected in lan and using
shared internet)
2- They are behind NAT
3- They are for one primary user each.
4- These nodes are deployed in our university labs so there is no internet
exposed service.
On Wed, Sep 20, 2017 at 3:55 PM, Zeolla@GMail.com <ze...@gmail.com> wrote:
> Okay, so I have some more questions then, but I'm still not sure how
> helpful I can be. Maybe someone else with a similar environment can chime
> in.
>
> These nodes, are they servers or endpoints (laptop/desktops used for
> productivity - internet use, email, etc.)? Are they behind network
> firewalls or NAT, or are they exposed? Are they shared machines or one
> primary user each? If there are any internet exposed services, what are
> they?
>
> Jon
>
> On Wed, Sep 20, 2017, 06:50 Syed Hammad Tahir <ms...@itu.edu.pk>
> wrote:
>
>> Actually I need to forward the specs for my IT department as soon as
>> possible, I was thinking to get a rough idea.
>> Regards.
>>
>> On Wed, Sep 20, 2017 at 3:43 PM, Zeolla@GMail.com <ze...@gmail.com>
>> wrote:
>>
>>> This is very much something Metron can do, but scoping hardware requires
>>> more detail about the data and work to be done on the data. I would focus
>>> on setting up the sensors (custom IDS, snort) and then either gather
>>> metrics and scope Metron or just spin it up by default/with whatever you
>>> have and see how it works.
>>>
>>> Jon
>>>
>>> On Wed, Sep 20, 2017, 06:23 Syed Hammad Tahir <ms...@itu.edu.pk>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> 1- I want to focus more on real time analysis but lets say we start
>>>> with pcap dump, I dont know at this point that how much data it can dump in
>>>> 24hr period given the lan environment of 100 nodes. You can assert your
>>>> assumption to answer.
>>>>
>>>> 2- Snort data most probably and dont know about the nukber of events
>>>> yes. You can also assert your assumption here for a hypothetical scenerio
>>>> to guide me.
>>>>
>>>> 3- I want to build an intrusion detection system and apply some machine
>>>> learning algorithm on it so Guess profiling is the answer to the third
>>>> question.
>>>>
>>>> Based on those partial answers and your insight into this domain,
>>>> kindly reply with most suitable solution with assumptions where necessary.
>>>>
>>>> If you think that I am expecting something from metron which it cant do
>>>> then kindly let me know.
>>>>
>>>> Regards
>>>>
>>>> Regards.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Sep 20, 2017 at 3:11 PM, Zeolla@GMail.com <ze...@gmail.com>
>>>> wrote:
>>>>
>>>>> Full dev is intended for testing, not actual use. That said, to
>>>>> answer your question it is more important to know (1) will you be storing
>>>>> pcap, (1b) if so, how much per day and for how long, (2) what data will you
>>>>> be sending into Metron (bro, yaf, snort, asa, etc.) and how many events per
>>>>> second is it, and (3) what are you planning to do with the data (profiling,
>>>>> MaaS, enrichments, etc.)?
>>>>>
>>>>> Jon
>>>>>
>>>>> On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> What would be the system required in order to run metron and analyzy
>>>>>> a LAN environment of almost 100 nodes using single node full development
>>>>>> depoloyment.
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>> --
>>>>>
>>>>> Jon
>>>>>
>>>>
>>>> --
>>>
>>> Jon
>>>
>>
>> --
>
> Jon
>
Re: System Requrements
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Okay, so I have some more questions then, but I'm still not sure how
helpful I can be. Maybe someone else with a similar environment can chime
in.
These nodes, are they servers or endpoints (laptop/desktops used for
productivity - internet use, email, etc.)? Are they behind network
firewalls or NAT, or are they exposed? Are they shared machines or one
primary user each? If there are any internet exposed services, what are
they?
Jon
On Wed, Sep 20, 2017, 06:50 Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
> Actually I need to forward the specs for my IT department as soon as
> possible, I was thinking to get a rough idea.
> Regards.
>
> On Wed, Sep 20, 2017 at 3:43 PM, Zeolla@GMail.com <ze...@gmail.com>
> wrote:
>
>> This is very much something Metron can do, but scoping hardware requires
>> more detail about the data and work to be done on the data. I would focus
>> on setting up the sensors (custom IDS, snort) and then either gather
>> metrics and scope Metron or just spin it up by default/with whatever you
>> have and see how it works.
>>
>> Jon
>>
>> On Wed, Sep 20, 2017, 06:23 Syed Hammad Tahir <ms...@itu.edu.pk>
>> wrote:
>>
>>> Hi,
>>>
>>> 1- I want to focus more on real time analysis but lets say we start with
>>> pcap dump, I dont know at this point that how much data it can dump in 24hr
>>> period given the lan environment of 100 nodes. You can assert your
>>> assumption to answer.
>>>
>>> 2- Snort data most probably and dont know about the nukber of events
>>> yes. You can also assert your assumption here for a hypothetical scenerio
>>> to guide me.
>>>
>>> 3- I want to build an intrusion detection system and apply some machine
>>> learning algorithm on it so Guess profiling is the answer to the third
>>> question.
>>>
>>> Based on those partial answers and your insight into this domain, kindly
>>> reply with most suitable solution with assumptions where necessary.
>>>
>>> If you think that I am expecting something from metron which it cant do
>>> then kindly let me know.
>>>
>>> Regards
>>>
>>> Regards.
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Sep 20, 2017 at 3:11 PM, Zeolla@GMail.com <ze...@gmail.com>
>>> wrote:
>>>
>>>> Full dev is intended for testing, not actual use. That said, to answer
>>>> your question it is more important to know (1) will you be storing pcap,
>>>> (1b) if so, how much per day and for how long, (2) what data will you be
>>>> sending into Metron (bro, yaf, snort, asa, etc.) and how many events per
>>>> second is it, and (3) what are you planning to do with the data (profiling,
>>>> MaaS, enrichments, etc.)?
>>>>
>>>> Jon
>>>>
>>>> On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> What would be the system required in order to run metron and analyzy a
>>>>> LAN environment of almost 100 nodes using single node full development
>>>>> depoloyment.
>>>>>
>>>>> Regards.
>>>>>
>>>> --
>>>>
>>>> Jon
>>>>
>>>
>>> --
>>
>> Jon
>>
>
> --
Jon
Re: System Requrements
Posted by Syed Hammad Tahir <ms...@itu.edu.pk>.
Actually I need to forward the specs for my IT department as soon as
possible, I was thinking to get a rough idea.
Regards.
On Wed, Sep 20, 2017 at 3:43 PM, Zeolla@GMail.com <ze...@gmail.com> wrote:
> This is very much something Metron can do, but scoping hardware requires
> more detail about the data and work to be done on the data. I would focus
> on setting up the sensors (custom IDS, snort) and then either gather
> metrics and scope Metron or just spin it up by default/with whatever you
> have and see how it works.
>
> Jon
>
> On Wed, Sep 20, 2017, 06:23 Syed Hammad Tahir <ms...@itu.edu.pk>
> wrote:
>
>> Hi,
>>
>> 1- I want to focus more on real time analysis but lets say we start with
>> pcap dump, I dont know at this point that how much data it can dump in 24hr
>> period given the lan environment of 100 nodes. You can assert your
>> assumption to answer.
>>
>> 2- Snort data most probably and dont know about the nukber of events yes.
>> You can also assert your assumption here for a hypothetical scenerio to
>> guide me.
>>
>> 3- I want to build an intrusion detection system and apply some machine
>> learning algorithm on it so Guess profiling is the answer to the third
>> question.
>>
>> Based on those partial answers and your insight into this domain, kindly
>> reply with most suitable solution with assumptions where necessary.
>>
>> If you think that I am expecting something from metron which it cant do
>> then kindly let me know.
>>
>> Regards
>>
>> Regards.
>>
>>
>>
>>
>>
>> On Wed, Sep 20, 2017 at 3:11 PM, Zeolla@GMail.com <ze...@gmail.com>
>> wrote:
>>
>>> Full dev is intended for testing, not actual use. That said, to answer
>>> your question it is more important to know (1) will you be storing pcap,
>>> (1b) if so, how much per day and for how long, (2) what data will you be
>>> sending into Metron (bro, yaf, snort, asa, etc.) and how many events per
>>> second is it, and (3) what are you planning to do with the data (profiling,
>>> MaaS, enrichments, etc.)?
>>>
>>> Jon
>>>
>>> On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> What would be the system required in order to run metron and analyzy a
>>>> LAN environment of almost 100 nodes using single node full development
>>>> depoloyment.
>>>>
>>>> Regards.
>>>>
>>> --
>>>
>>> Jon
>>>
>>
>> --
>
> Jon
>
Re: System Requrements
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
This is very much something Metron can do, but scoping hardware requires
more detail about the data and work to be done on the data. I would focus
on setting up the sensors (custom IDS, snort) and then either gather
metrics and scope Metron or just spin it up by default/with whatever you
have and see how it works.
Jon
On Wed, Sep 20, 2017, 06:23 Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
> Hi,
>
> 1- I want to focus more on real time analysis but lets say we start with
> pcap dump, I dont know at this point that how much data it can dump in 24hr
> period given the lan environment of 100 nodes. You can assert your
> assumption to answer.
>
> 2- Snort data most probably and dont know about the nukber of events yes.
> You can also assert your assumption here for a hypothetical scenerio to
> guide me.
>
> 3- I want to build an intrusion detection system and apply some machine
> learning algorithm on it so Guess profiling is the answer to the third
> question.
>
> Based on those partial answers and your insight into this domain, kindly
> reply with most suitable solution with assumptions where necessary.
>
> If you think that I am expecting something from metron which it cant do
> then kindly let me know.
>
> Regards
>
> Regards.
>
>
>
>
>
> On Wed, Sep 20, 2017 at 3:11 PM, Zeolla@GMail.com <ze...@gmail.com>
> wrote:
>
>> Full dev is intended for testing, not actual use. That said, to answer
>> your question it is more important to know (1) will you be storing pcap,
>> (1b) if so, how much per day and for how long, (2) what data will you be
>> sending into Metron (bro, yaf, snort, asa, etc.) and how many events per
>> second is it, and (3) what are you planning to do with the data (profiling,
>> MaaS, enrichments, etc.)?
>>
>> Jon
>>
>> On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk>
>> wrote:
>>
>>> Hello,
>>>
>>> What would be the system required in order to run metron and analyzy a
>>> LAN environment of almost 100 nodes using single node full development
>>> depoloyment.
>>>
>>> Regards.
>>>
>> --
>>
>> Jon
>>
>
> --
Jon
Re: System Requrements
Posted by Syed Hammad Tahir <ms...@itu.edu.pk>.
Hi,
1- I want to focus more on real time analysis but lets say we start with
pcap dump, I dont know at this point that how much data it can dump in 24hr
period given the lan environment of 100 nodes. You can assert your
assumption to answer.
2- Snort data most probably and dont know about the nukber of events yes.
You can also assert your assumption here for a hypothetical scenerio to
guide me.
3- I want to build an intrusion detection system and apply some machine
learning algorithm on it so Guess profiling is the answer to the third
question.
Based on those partial answers and your insight into this domain, kindly
reply with most suitable solution with assumptions where necessary.
If you think that I am expecting something from metron which it cant do
then kindly let me know.
Regards
Regards.
On Wed, Sep 20, 2017 at 3:11 PM, Zeolla@GMail.com <ze...@gmail.com> wrote:
> Full dev is intended for testing, not actual use. That said, to answer
> your question it is more important to know (1) will you be storing pcap,
> (1b) if so, how much per day and for how long, (2) what data will you be
> sending into Metron (bro, yaf, snort, asa, etc.) and how many events per
> second is it, and (3) what are you planning to do with the data (profiling,
> MaaS, enrichments, etc.)?
>
> Jon
>
> On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk>
> wrote:
>
>> Hello,
>>
>> What would be the system required in order to run metron and analyzy a
>> LAN environment of almost 100 nodes using single node full development
>> depoloyment.
>>
>> Regards.
>>
> --
>
> Jon
>
Re: System Requrements
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Full dev is intended for testing, not actual use. That said, to answer
your question it is more important to know (1) will you be storing pcap,
(1b) if so, how much per day and for how long, (2) what data will you be
sending into Metron (bro, yaf, snort, asa, etc.) and how many events per
second is it, and (3) what are you planning to do with the data (profiling,
MaaS, enrichments, etc.)?
Jon
On Wed, Sep 20, 2017, 04:04 Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
> Hello,
>
> What would be the system required in order to run metron and analyzy a LAN
> environment of almost 100 nodes using single node full development
> depoloyment.
>
> Regards.
>
--
Jon