Usecase to profile the logs data and display alerts in metron alerts-UI

[Anil Donthireddy <An...@sstech.us>]

Hi,

I am trying to develop an usecase with metron.

The end-to-end usecase is, we need to calculate the count of number of events happened for each ip's. Based on the count reaching above the threshold count generate an alerts in the merton alerts-UI.

For that, I have the events data coming to "indexing" kafka topic. I created a profile in ${METRON_HOME}/config/zookeeper/profiler.json as below.

**************************
{
                "profiles": [
                {
                                "profile": "demo_iplogon_failed",
                                "foreach": "ip_address",
                                "onlyif": "source.type == 'demo_windowsnxlog' and event_id == '4625'",
                                "init": {
                                                "count": "0"
                                },
                                "update": {
                                                "count": "count + 1"
                                },
                                "result": {
                                                "profile" : "count",
                                                "triage": {
                                                                "logon_failed_count": "count"
                                                }
                                }
                }
                ]
}
***********************
After reloading the zookeeper config files, I see the indices created in ElasticSearch with format profile_index_yyyy.MM.dd.HH and the snapshot of record is as below

[cid:image001.png@01D3C15A.D7935210]

I see all the records in the profiler_index_* pattern have is_alert="true". But I don't see these records turn up as alerts in the metron-alerts UI.

I would like to figure out the reason for alerts not turning up in the metron-alerts UI ?

I am seeing is_alerts="true" for all the records under profiler_index_*. How can I configure to set is_alert="true" only when the count exceeds the threshold value ?

Thanks,
Anil.

Re: Usecase to profile the logs data and display alerts in metron alerts-UI

[Nick Allen <ni...@nickallen.org>]

> I would like to figure out the reason for alerts not turning up in the
metron-alerts UI ?

I believe it is because the Alerts UI has not been configured to look at
the `profiler_index_*` indices that have been created. Right now, the
Alerts UI only looks at the indices that have been created for each sensor.


> I am seeing is_alerts=”true” for all the records under profiler_index_*.

The `is_alerts` value is set to true when those messages are generated by
the Profiler. The purpose of sending messages back into Kafka from the
Profiler, is to enable use of the Threat Triage mechanism. That is why they
are always set to true.


> How can I configure to set is_alert=”true” only when the count exceeds
the threshold value ?

You would do this in a slightly different way.  You need to define a rule
in Threat Triage that increase the threat score based on that count
exceeding a threshold.


On Wed, Mar 21, 2018 at 11:53 AM, Anil Donthireddy <
Anil.Donthireddy@sstech.us> wrote:

> Hi,
>
>
>
> I am trying to develop an usecase with metron.
>
>
>
> The end-to-end usecase is, we need to calculate the count of number of
> events happened for each ip’s. Based on the count reaching above the
> threshold count generate an alerts in the merton alerts-UI.
>
>
>
> For that, I have the events data coming to “indexing” kafka topic. I
> created a profile in ${METRON_HOME}/config/zookeeper/profiler.json as
> below.
>
>
>
> **************************
>
> {
>
>                 "profiles": [
>
>                 {
>
>                                 "profile": "demo_iplogon_failed",
>
>                                 "foreach": "ip_address",
>
>                                 "onlyif": "source.type ==
> 'demo_windowsnxlog' and event_id == '4625'",
>
>                                 "init": {
>
>                                                 "count": "0"
>
>                                 },
>
>                                 "update": {
>
>                                                 "count": "count + 1"
>
>                                 },
>
>                                 "result": {
>
>                                                 "profile" : "count",
>
>                                                 "triage": {
>
>
> "logon_failed_count": "count"
>
>                                                 }
>
>                                 }
>
>                 }
>
>                 ]
>
> }
>
> ***********************
>
> After reloading the zookeeper config files, I see the indices created in
> ElasticSearch with format profile_index_yyyy.MM.dd.HH and the snapshot of
> record is as below
>
>
>
>
>
> I see all the records in the profiler_index_* pattern have
> is_alert=”true”. But I don’t see these records turn up as alerts in the
> metron-alerts UI.
>
>
>
> I would like to figure out the reason for alerts not turning up in the
> metron-alerts UI ?
>
>
>
> I am seeing is_alerts=”true” for all the records under profiler_index_*.
> How can I configure to set is_alert=”true” only when the count exceeds the
> threshold value ?
>
>
>
> Thanks,
>
> Anil.
>