You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Syed Hammad Tahir <ms...@itu.edu.pk> on 2017/12/06 07:42:22 UTC
Basic analysis
Hi,
Can I setup custom visualization to show lets say the peak netrwork usage
traffic in a certain time?
Regards.
Re: Basic analysis
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
I suggest reading some of the kibana documentation. There are a lot of tutorials available online on kibana.
Simon
> On 6 Dec 2017, at 15:12, Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
>
> How do I do it using kibana dashboard? What would be the most easy way?
>
>
> On Wed, Dec 6, 2017 at 7:35 PM, Simon Elliston Ball <simon@simonellistonball.com <ma...@simonellistonball.com>> wrote:
> Agreed… for the users list I would just say use the Install Notebooks action, and look at the squid example on the wiki, but since it was you who asked for links, Otto, I went a bit dev list ;)
>
> Simon
>
>
>> On 6 Dec 2017, at 14:33, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>
>> The issue is the requirement for people on the user list to go to the source.
>>
>>
>> On December 6, 2017 at 09:16:39, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>
>>> No problem, I’ll grant you it’s not in the most intuitive part of the source tree to go digging in, but you can also get to the zeppelin bits via the actions button on the Metron config section (Install Notebooks)
>>>
>>> If anyone has any good ideas (or code!) for sample zeppelin notebooks that would be useful, you can add them to a specific instance of the platform via the config/zeppelin/metron location and run the action again I believe, and this would be a great place for more security people to contribute sample run books for example. There are also efforts by commercial support providers I believe to add more samples of both dashboards and use cases.
>>>
>>> Simon
>>>
>>>> On 6 Dec 2017, at 14:12, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>
>>>> Thanks Simon
>>>>
>>>>
>>>> On December 6, 2017 at 09:11:50, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>>
>>>>> In product… Install Zeppelin Notebooks, and the samples including notebooks at https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron <https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron>
>>>>>
>>>>> as of course there are similar Kibana dashboards included, which are examples of custom visualisation of metron data, there is also the run book for visualising squid data in kibana on the docs wiki https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard <https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard>
>>>>>
>>>>> Should at least get us started.
>>>>>
>>>>> Simon
>>>>>
>>>>>> On 6 Dec 2017, at 14:00, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>>>
>>>>>> Links?
>>>>>>
>>>>>>
>>>>>> On December 6, 2017 at 08:18:23, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>>>>
>>>>>>> We do already have a number of example of exactly this, but sure if someone feels like adding to those that would be great.
>>>>>>>
>>>>>>> Simon
>>>>>>>
>>>>>>>> On 6 Dec 2017, at 13:14, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>
>>>>>>>> Maybe a Jira logged for an ‘example’ notebook for this would be appropriate as well?
>>>>>>>>
>>>>>>>>
>>>>>>>> On December 6, 2017 at 07:06:30, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>>>>>>
>>>>>>>>> Yes. Consider a zeppelin notebook, or kibana dashboard for this.
>>>>>>>>>
>>>>>>>>> If you want to use these values for detection, consider building a profile based on the stats objects (see the profiler section of the documentation under analytics.
>>>>>>>>>
>>>>>>>>> Simon
>>>>>>>>>
>>>>>>>>> > On 6 Dec 2017, at 07:42, Syed Hammad Tahir <mscs16059@itu.edu.pk <ma...@itu.edu.pk>> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi,
>>>>>>>>> >
>>>>>>>>> > Can I setup custom visualization to show lets say the peak netrwork usage traffic in a certain time?
>>>>>>>>> >
>>>>>>>>> > Regards.
>
>
Re: Basic analysis
Posted by Syed Hammad Tahir <ms...@itu.edu.pk>.
How do I do it using kibana dashboard? What would be the most easy way?
On Wed, Dec 6, 2017 at 7:35 PM, Simon Elliston Ball <
simon@simonellistonball.com> wrote:
> Agreed… for the users list I would just say use the Install Notebooks
> action, and look at the squid example on the wiki, but since it was you who
> asked for links, Otto, I went a bit dev list ;)
>
> Simon
>
>
> On 6 Dec 2017, at 14:33, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is the requirement for people on the user list to go to the
> source.
>
>
> On December 6, 2017 at 09:16:39, Simon Elliston Ball (
> simon@simonellistonball.com) wrote:
>
> No problem, I’ll grant you it’s not in the most intuitive part of the
> source tree to go digging in, but you can also get to the zeppelin bits via
> the actions button on the Metron config section (Install Notebooks)
>
> If anyone has any good ideas (or code!) for sample zeppelin notebooks that
> would be useful, you can add them to a specific instance of the platform
> via the config/zeppelin/metron location and run the action again I believe,
> and this would be a great place for more security people to contribute
> sample run books for example. There are also efforts by commercial support
> providers I believe to add more samples of both dashboards and use cases.
>
> Simon
>
> On 6 Dec 2017, at 14:12, Otto Fowler <ot...@gmail.com> wrote:
>
> Thanks Simon
>
>
> On December 6, 2017 at 09:11:50, Simon Elliston Ball (
> simon@simonellistonball.com) wrote:
>
> In product… Install Zeppelin Notebooks, and the samples including
> notebooks at https://github.com/apache/metron/tree/master/metron-
> platform/metron-indexing/src/main/config/zeppelin/metron
>
> as of course there are similar Kibana dashboards included, which are
> examples of custom visualisation of metron data, there is also the run book
> for visualising squid data in kibana on the docs wiki
> https://cwiki.apache.org/confluence/display/METRON/
> Enhancing+Metron+Dashboard
>
> Should at least get us started.
>
> Simon
>
> On 6 Dec 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
>
> Links?
>
>
> On December 6, 2017 at 08:18:23, Simon Elliston Ball (
> simon@simonellistonball.com) wrote:
>
> We do already have a number of example of exactly this, but sure if
> someone feels like adding to those that would be great.
>
> Simon
>
> On 6 Dec 2017, at 13:14, Otto Fowler <ot...@gmail.com> wrote:
>
> Maybe a Jira logged for an ‘example’ notebook for this would be
> appropriate as well?
>
>
> On December 6, 2017 at 07:06:30, Simon Elliston Ball (
> simon@simonellistonball.com) wrote:
>
> Yes. Consider a zeppelin notebook, or kibana dashboard for this.
>
> If you want to use these values for detection, consider building a profile
> based on the stats objects (see the profiler section of the documentation
> under analytics.
>
> Simon
>
> > On 6 Dec 2017, at 07:42, Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
>
> >
> > Hi,
> >
> > Can I setup custom visualization to show lets say the peak netrwork
> usage traffic in a certain time?
> >
> > Regards.
>
>
>
Re: Basic analysis
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
Agreed… for the users list I would just say use the Install Notebooks action, and look at the squid example on the wiki, but since it was you who asked for links, Otto, I went a bit dev list ;)
Simon
> On 6 Dec 2017, at 14:33, Otto Fowler <ot...@gmail.com> wrote:
>
> The issue is the requirement for people on the user list to go to the source.
>
>
> On December 6, 2017 at 09:16:39, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>
>> No problem, I’ll grant you it’s not in the most intuitive part of the source tree to go digging in, but you can also get to the zeppelin bits via the actions button on the Metron config section (Install Notebooks)
>>
>> If anyone has any good ideas (or code!) for sample zeppelin notebooks that would be useful, you can add them to a specific instance of the platform via the config/zeppelin/metron location and run the action again I believe, and this would be a great place for more security people to contribute sample run books for example. There are also efforts by commercial support providers I believe to add more samples of both dashboards and use cases.
>>
>> Simon
>>
>>> On 6 Dec 2017, at 14:12, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Thanks Simon
>>>
>>>
>>> On December 6, 2017 at 09:11:50, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>
>>>> In product… Install Zeppelin Notebooks, and the samples including notebooks at https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron <https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron>
>>>>
>>>> as of course there are similar Kibana dashboards included, which are examples of custom visualisation of metron data, there is also the run book for visualising squid data in kibana on the docs wiki https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard <https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard>
>>>>
>>>> Should at least get us started.
>>>>
>>>> Simon
>>>>
>>>>> On 6 Dec 2017, at 14:00, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>>
>>>>> Links?
>>>>>
>>>>>
>>>>> On December 6, 2017 at 08:18:23, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>>>
>>>>>> We do already have a number of example of exactly this, but sure if someone feels like adding to those that would be great.
>>>>>>
>>>>>> Simon
>>>>>>
>>>>>>> On 6 Dec 2017, at 13:14, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>
>>>>>>> Maybe a Jira logged for an ‘example’ notebook for this would be appropriate as well?
>>>>>>>
>>>>>>>
>>>>>>> On December 6, 2017 at 07:06:30, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>>>>>
>>>>>>>> Yes. Consider a zeppelin notebook, or kibana dashboard for this.
>>>>>>>>
>>>>>>>> If you want to use these values for detection, consider building a profile based on the stats objects (see the profiler section of the documentation under analytics.
>>>>>>>>
>>>>>>>> Simon
>>>>>>>>
>>>>>>>> > On 6 Dec 2017, at 07:42, Syed Hammad Tahir <mscs16059@itu.edu.pk <ma...@itu.edu.pk>> wrote:
>>>>>>>> >
>>>>>>>> > Hi,
>>>>>>>> >
>>>>>>>> > Can I setup custom visualization to show lets say the peak netrwork usage traffic in a certain time?
>>>>>>>> >
>>>>>>>> > Regards.
Re: Basic analysis
Posted by Otto Fowler <ot...@gmail.com>.
The issue is the requirement for people on the user list to go to the
source.
On December 6, 2017 at 09:16:39, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
No problem, I’ll grant you it’s not in the most intuitive part of the
source tree to go digging in, but you can also get to the zeppelin bits via
the actions button on the Metron config section (Install Notebooks)
If anyone has any good ideas (or code!) for sample zeppelin notebooks that
would be useful, you can add them to a specific instance of the platform
via the config/zeppelin/metron location and run the action again I believe,
and this would be a great place for more security people to contribute
sample run books for example. There are also efforts by commercial support
providers I believe to add more samples of both dashboards and use cases.
Simon
On 6 Dec 2017, at 14:12, Otto Fowler <ot...@gmail.com> wrote:
Thanks Simon
On December 6, 2017 at 09:11:50, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
In product… Install Zeppelin Notebooks, and the samples including notebooks
at
https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron
as of course there are similar Kibana dashboards included, which are
examples of custom visualisation of metron data, there is also the run book
for visualising squid data in kibana on the docs wiki
https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard
Should at least get us started.
Simon
On 6 Dec 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
Links?
On December 6, 2017 at 08:18:23, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
We do already have a number of example of exactly this, but sure if someone
feels like adding to those that would be great.
Simon
On 6 Dec 2017, at 13:14, Otto Fowler <ot...@gmail.com> wrote:
Maybe a Jira logged for an ‘example’ notebook for this would be appropriate
as well?
On December 6, 2017 at 07:06:30, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
Yes. Consider a zeppelin notebook, or kibana dashboard for this.
If you want to use these values for detection, consider building a profile
based on the stats objects (see the profiler section of the documentation
under analytics.
Simon
> On 6 Dec 2017, at 07:42, Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
>
> Hi,
>
> Can I setup custom visualization to show lets say the peak netrwork usage
traffic in a certain time?
>
> Regards.
Re: Basic analysis
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
No problem, I’ll grant you it’s not in the most intuitive part of the source tree to go digging in, but you can also get to the zeppelin bits via the actions button on the Metron config section (Install Notebooks)
If anyone has any good ideas (or code!) for sample zeppelin notebooks that would be useful, you can add them to a specific instance of the platform via the config/zeppelin/metron location and run the action again I believe, and this would be a great place for more security people to contribute sample run books for example. There are also efforts by commercial support providers I believe to add more samples of both dashboards and use cases.
Simon
> On 6 Dec 2017, at 14:12, Otto Fowler <ot...@gmail.com> wrote:
>
> Thanks Simon
>
>
> On December 6, 2017 at 09:11:50, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>
>> In product… Install Zeppelin Notebooks, and the samples including notebooks at https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron <https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron>
>>
>> as of course there are similar Kibana dashboards included, which are examples of custom visualisation of metron data, there is also the run book for visualising squid data in kibana on the docs wiki https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard <https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard>
>>
>> Should at least get us started.
>>
>> Simon
>>
>>> On 6 Dec 2017, at 14:00, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Links?
>>>
>>>
>>> On December 6, 2017 at 08:18:23, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>
>>>> We do already have a number of example of exactly this, but sure if someone feels like adding to those that would be great.
>>>>
>>>> Simon
>>>>
>>>>> On 6 Dec 2017, at 13:14, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>>>
>>>>> Maybe a Jira logged for an ‘example’ notebook for this would be appropriate as well?
>>>>>
>>>>>
>>>>> On December 6, 2017 at 07:06:30, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>>>
>>>>>> Yes. Consider a zeppelin notebook, or kibana dashboard for this.
>>>>>>
>>>>>> If you want to use these values for detection, consider building a profile based on the stats objects (see the profiler section of the documentation under analytics.
>>>>>>
>>>>>> Simon
>>>>>>
>>>>>> > On 6 Dec 2017, at 07:42, Syed Hammad Tahir <mscs16059@itu.edu.pk <ma...@itu.edu.pk>> wrote:
>>>>>> >
>>>>>> > Hi,
>>>>>> >
>>>>>> > Can I setup custom visualization to show lets say the peak netrwork usage traffic in a certain time?
>>>>>> >
>>>>>> > Regards.
Re: Basic analysis
Posted by Otto Fowler <ot...@gmail.com>.
Thanks Simon
On December 6, 2017 at 09:11:50, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
In product… Install Zeppelin Notebooks, and the samples including notebooks
at
https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron
as of course there are similar Kibana dashboards included, which are
examples of custom visualisation of metron data, there is also the run book
for visualising squid data in kibana on the docs wiki
https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard
Should at least get us started.
Simon
On 6 Dec 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
Links?
On December 6, 2017 at 08:18:23, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
We do already have a number of example of exactly this, but sure if someone
feels like adding to those that would be great.
Simon
On 6 Dec 2017, at 13:14, Otto Fowler <ot...@gmail.com> wrote:
Maybe a Jira logged for an ‘example’ notebook for this would be appropriate
as well?
On December 6, 2017 at 07:06:30, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
Yes. Consider a zeppelin notebook, or kibana dashboard for this.
If you want to use these values for detection, consider building a profile
based on the stats objects (see the profiler section of the documentation
under analytics.
Simon
> On 6 Dec 2017, at 07:42, Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
>
> Hi,
>
> Can I setup custom visualization to show lets say the peak netrwork usage
traffic in a certain time?
>
> Regards.
Re: Basic analysis
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
In product… Install Zeppelin Notebooks, and the samples including notebooks at https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron <https://github.com/apache/metron/tree/master/metron-platform/metron-indexing/src/main/config/zeppelin/metron>
as of course there are similar Kibana dashboards included, which are examples of custom visualisation of metron data, there is also the run book for visualising squid data in kibana on the docs wiki https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard <https://cwiki.apache.org/confluence/display/METRON/Enhancing+Metron+Dashboard>
Should at least get us started.
Simon
> On 6 Dec 2017, at 14:00, Otto Fowler <ot...@gmail.com> wrote:
>
> Links?
>
>
> On December 6, 2017 at 08:18:23, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>
>> We do already have a number of example of exactly this, but sure if someone feels like adding to those that would be great.
>>
>> Simon
>>
>>> On 6 Dec 2017, at 13:14, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Maybe a Jira logged for an ‘example’ notebook for this would be appropriate as well?
>>>
>>>
>>> On December 6, 2017 at 07:06:30, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>>>
>>>> Yes. Consider a zeppelin notebook, or kibana dashboard for this.
>>>>
>>>> If you want to use these values for detection, consider building a profile based on the stats objects (see the profiler section of the documentation under analytics.
>>>>
>>>> Simon
>>>>
>>>> > On 6 Dec 2017, at 07:42, Syed Hammad Tahir <mscs16059@itu.edu.pk <ma...@itu.edu.pk>> wrote:
>>>> >
>>>> > Hi,
>>>> >
>>>> > Can I setup custom visualization to show lets say the peak netrwork usage traffic in a certain time?
>>>> >
>>>> > Regards.
Re: Basic analysis
Posted by Otto Fowler <ot...@gmail.com>.
Links?
On December 6, 2017 at 08:18:23, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
We do already have a number of example of exactly this, but sure if someone
feels like adding to those that would be great.
Simon
On 6 Dec 2017, at 13:14, Otto Fowler <ot...@gmail.com> wrote:
Maybe a Jira logged for an ‘example’ notebook for this would be appropriate
as well?
On December 6, 2017 at 07:06:30, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
Yes. Consider a zeppelin notebook, or kibana dashboard for this.
If you want to use these values for detection, consider building a profile
based on the stats objects (see the profiler section of the documentation
under analytics.
Simon
> On 6 Dec 2017, at 07:42, Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
>
> Hi,
>
> Can I setup custom visualization to show lets say the peak netrwork usage
traffic in a certain time?
>
> Regards.
Re: Basic analysis
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
We do already have a number of example of exactly this, but sure if someone feels like adding to those that would be great.
Simon
> On 6 Dec 2017, at 13:14, Otto Fowler <ot...@gmail.com> wrote:
>
> Maybe a Jira logged for an ‘example’ notebook for this would be appropriate as well?
>
>
> On December 6, 2017 at 07:06:30, Simon Elliston Ball (simon@simonellistonball.com <ma...@simonellistonball.com>) wrote:
>
>> Yes. Consider a zeppelin notebook, or kibana dashboard for this.
>>
>> If you want to use these values for detection, consider building a profile based on the stats objects (see the profiler section of the documentation under analytics.
>>
>> Simon
>>
>> > On 6 Dec 2017, at 07:42, Syed Hammad Tahir <mscs16059@itu.edu.pk <ma...@itu.edu.pk>> wrote:
>> >
>> > Hi,
>> >
>> > Can I setup custom visualization to show lets say the peak netrwork usage traffic in a certain time?
>> >
>> > Regards.
Re: Basic analysis
Posted by Otto Fowler <ot...@gmail.com>.
Maybe a Jira logged for an ‘example’ notebook for this would be appropriate
as well?
On December 6, 2017 at 07:06:30, Simon Elliston Ball (
simon@simonellistonball.com) wrote:
Yes. Consider a zeppelin notebook, or kibana dashboard for this.
If you want to use these values for detection, consider building a profile
based on the stats objects (see the profiler section of the documentation
under analytics.
Simon
> On 6 Dec 2017, at 07:42, Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
>
> Hi,
>
> Can I setup custom visualization to show lets say the peak netrwork usage
traffic in a certain time?
>
> Regards.
Re: Basic analysis
Posted by Simon Elliston Ball <si...@simonellistonball.com>.
Yes. Consider a zeppelin notebook, or kibana dashboard for this.
If you want to use these values for detection, consider building a profile based on the stats objects (see the profiler section of the documentation under analytics.
Simon
> On 6 Dec 2017, at 07:42, Syed Hammad Tahir <ms...@itu.edu.pk> wrote:
>
> Hi,
>
> Can I setup custom visualization to show lets say the peak netrwork usage traffic in a certain time?
>
> Regards.