You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@metron.apache.org by Hema malini <nh...@gmail.com> on 2019/11/06 15:27:35 UTC

Push data from elastic search to Metron alerts

Hi all,

I pushed data to elastic search indices from csv . Log data has been parsed
to CSV using customized parser and i am trying to view in metron alerts and
apply machine learning on top of those log data(Maas). I pushed
bro,snort,yaf logs to Metron .When i tried to push syslog in syslog 5424
format, i was able to see those logs getting indexed in elastic search as
well as it is present in hdfs.But i couldn't view those system logs in
metron alerts ui and in Kibana also i am not able to get indexes like
*syslog5424 though index has been created.So i created index with * pattern.

How to view syslog and custom parsed logs in csv in metron alerts.What
should be configured for data in elastic search to besent to Metron alerts
ui and for Maas as well.can someone pls help

Thanks and Regards,
Hema

Re: Push data from elastic search to Metron alerts

Posted by Hema malini <nh...@gmail.com>.

Thanks a lot Michael. As mentioned it is elastic search indices issue.
Thanks for your timely response.

On Wed, 6 Nov, 2019, 11:15 PM Michael Miklavcic, <
michael.miklavcic@gmail.com> wrote:

> It sounds like you might have some issues with Elasticsearch templates.
> See here for more detail -
> https://github.com/apache/metron/tree/master/metron-platform/metron-elasticsearch/metron-elasticsearch-common
>
> On Wed, Nov 6, 2019 at 8:25 AM Hema malini <nh...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I pushed data to elastic search indices from csv . Log data has been
>> parsed to CSV using customized parser and i am trying to view in metron
>> alerts and apply machine learning on top of those log data(Maas). I pushed
>> bro,snort,yaf logs to Metron .When i tried to push syslog in syslog 5424
>> format, i was able to see those logs getting indexed in elastic search as
>> well as it is present in hdfs.But i couldn't view those system logs in
>> metron alerts ui and in Kibana also i am not able to get indexes like
>> *syslog5424 though index has been created.So i created index with * pattern.
>>
>> How to view syslog and custom parsed logs in csv in metron alerts.What
>> should be configured for data in elastic search to besent to Metron alerts
>> ui and for Maas as well.can someone pls help
>>
>> Thanks and Regards,
>> Hema
>>
>>
>>

Re: Push data from elastic search to Metron alerts

Posted by Michael Miklavcic <mi...@gmail.com>.

It sounds like you might have some issues with Elasticsearch templates. See
here for more detail -
https://github.com/apache/metron/tree/master/metron-platform/metron-elasticsearch/metron-elasticsearch-common

On Wed, Nov 6, 2019 at 8:25 AM Hema malini <nh...@gmail.com> wrote:

> Hi all,
>
> I pushed data to elastic search indices from csv . Log data has been
> parsed to CSV using customized parser and i am trying to view in metron
> alerts and apply machine learning on top of those log data(Maas). I pushed
> bro,snort,yaf logs to Metron .When i tried to push syslog in syslog 5424
> format, i was able to see those logs getting indexed in elastic search as
> well as it is present in hdfs.But i couldn't view those system logs in
> metron alerts ui and in Kibana also i am not able to get indexes like
> *syslog5424 though index has been created.So i created index with * pattern.
>
> How to view syslog and custom parsed logs in csv in metron alerts.What
> should be configured for data in elastic search to besent to Metron alerts
> ui and for Maas as well.can someone pls help
>
> Thanks and Regards,
> Hema
>
>
>