You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Laurens Vets <la...@daemon.be> on 2017/09/25 17:46:25 UTC

Not seeing any Metron alerts.

I have the following configuration:

"threatIntel": {
   "fieldMap": {},
     "fieldToTypeMap": {},
     "config": {},
     "triageConfig": {
       "riskLevelRules": [
         {
           "name": "Rule1",
           "comment": "Checks whatever 1.",
           "rule": "test == \"false\"",
           "score": 20,
           "reason": null
         },
         {
           "name": "Rule1",
           "comment": "Checks whatever 2.",
           "rule": "test2 == \"False\"",
           "score": 20,
           "reason": null
         },
         {
           "name": "Rule3",
           "comment": "Checks whatever 2.",
           "rule": "test3 == \"No\"",
           "score": 20,
           "reason": null
         }
       ],
       "aggregator": "SUM",
       "aggregationConfig": {}
     }
},

I have no additional configuration in enrichment besides filling a 
specific with true or false based on a Stellar expression.

I expected that when events would match my above rules, the _score field 
would be filled in. That does not seem to be the case.

Does anyone know what I might be missing?

Re: Not seeing any Metron alerts.

Posted by Nick Allen <ni...@nickallen.org>.
And, this PR just went into master.  Hopefully this will help in the
future.  Let me know how I can make it better.

On Mon, Sep 25, 2017 at 4:54 PM, Nick Allen <ni...@nickallen.org> wrote:

> Just as a side note, based on PR #733 [1], you can also simulate/debug
> these types of
> ​Threat Triage ​
> rules
> ​directly ​
> in the Stellar REPL.
> ​  ​
>
> ​(1) There are a few different Threat Triage functions that you can use.
> Feel free to explore each.
>
> [Stellar]>>> %functions THREAT
> THREAT_TRIAGE_ADD, THREAT_TRIAGE_CONFIG, THREAT_TRIAGE_INIT,
> THREAT_TRIAGE_PRINT, THREAT_TRIAGE_REMOVE, THREAT_TRIAGE_SCORE,
> THREAT_TRIAGE_SET_AGGREGATOR​
>
>
> ​(2) Mock-up a telemetry message that we will triage and score.
>
> [Stellar]>>> msg := SHELL_EDIT()
> [Stellar]>>> msg
> {
> "test3": "No"
> }​
>
>
> (
> ​3​
> ) I
> ​copied
>  the rules that you included in your email.
> ​  You can either pass in the entire configuration like this or build up
> the rule set incrementally using THREAT_TRIAGE_ADD.​
>
> [Stellar]>>> conf := SHELL_EDIT()
> [Stellar]>>> conf
> {
>   "enrichment" : {
>     "fieldMap" : { },
>     "fieldToTypeMap" : { },
>     "config" : { }
>   },
>   "threatIntel": {
>    "fieldMap": {},
>      "fieldToTypeMap": {},
>      "config": {},
>      "triageConfig": {
>        "riskLevelRules": [
>          {
>            "name": "Rule1",
>            "comment": "Checks whatever 1.",
>            "rule": "test == \"false\"",
>            "score": 20,
>            "reason": null
>          },
>          {
>            "name": "Rule1",
>            "comment": "Checks whatever 2.",
>            "rule": "test2 == \"False\"",
>            "score": 20,
>            "reason": null
>          },
>          {
>            "name": "Rule3",
>            "comment": "Checks whatever 2.",
>            "rule": "test3 == \"No\"",
>            "score": 20,
>            "reason": null
>          }
>        ],
>        "aggregator": "SUM",
>        "aggregationConfig": {}
>      }
>    },
>    "configuration": {}
> }
>
>
>
> ​(4) Initialize Threat Triage and review your rule set.
>
> [Stellar]>>> t := THREAT_TRIAGE_INIT(conf)
> [Stellar]>>> THREAT_TRIAGE_PRINT(t)
> ╔═══════╤════════════════════╤══════════════════╤═══════╤════════╗
> ║ Name  │ Comment            │ Triage Rule      │ Score │ Reason ║
> ╠═══════╪════════════════════╪══════════════════╪═══════╪════════╣
> ║ Rule1 │ Checks whatever 1. │ test == "false"  │ 20    │        ║
> ╟───────┼────────────────────┼──────────────────┼───────┼────────╢
> ║ Rule1 │ Checks whatever 2. │ test2 == "False" │ 20    │        ║
> ╟───────┼────────────────────┼──────────────────┼───────┼────────╢
> ║ Rule3 │ Checks whatever 2. │ test3 == "No"    │ 20    │        ║
> ╚═══════╧════════════════════╧══════════════════╧═══════╧════════╝
> Aggregation: SUM
>
> ​​
> ​(5) Score the message.  From this you can see in detail that the total
> threat triage score is 20, which was the sum of a single rule that fired;
> in this case Rule3.
>
> [Stellar]>>> THREAT_TRIAGE_SCORE(msg, t)
> {score=20.0, aggregator=SUM, rules=[{score=20, name=Rule3, rule=test3 ==
> "No", comment=Checks whatever 2.}]} ​
>
>
>
> ​I also noticed as I was putting together this demo that the Triage
> Debugger does not honor the `is_alert` field and so behaves slightly
> differently than when running in the Enrichment topology.  I should fix
> that. :)​
>
>
>
> [1] https://github.com/apache/metron/pull/733
>
>
>
>
> On Mon, Sep 25, 2017 at 1:46 PM Laurens Vets <la...@daemon.be> wrote:
>
>> I have the following configuration:
>>
>> "threatIntel": {
>>    "fieldMap": {},
>>      "fieldToTypeMap": {},
>>      "config": {},
>>      "triageConfig": {
>>        "riskLevelRules": [
>>          {
>>            "name": "Rule1",
>>            "comment": "Checks whatever 1.",
>>            "rule": "test == \"false\"",
>>            "score": 20,
>>            "reason": null
>>          },
>>          {
>>            "name": "Rule1",
>>            "comment": "Checks whatever 2.",
>>            "rule": "test2 == \"False\"",
>>            "score": 20,
>>            "reason": null
>>          },
>>          {
>>            "name": "Rule3",
>>            "comment": "Checks whatever 2.",
>>            "rule": "test3 == \"No\"",
>>            "score": 20,
>>            "reason": null
>>          }
>>        ],
>>        "aggregator": "SUM",
>>        "aggregationConfig": {}
>>      }
>> },
>>
>> I have no additional configuration in enrichment besides filling a
>> specific with true or false based on a Stellar expression.
>>
>> I expected that when events would match my above rules, the _score field
>> would be filled in. That does not seem to be the case.
>>
>> Does anyone know what I might be missing?
>>
>

Re: Not seeing any Metron alerts.

Posted by Nick Allen <ni...@nickallen.org>.
Just as a side note, based on PR #733 [1], you can also simulate/debug
these types of
​Threat Triage ​
rules
​directly ​
in the Stellar REPL.
​  ​

​(1) There are a few different Threat Triage functions that you can use.
Feel free to explore each.

[Stellar]>>> %functions THREAT
THREAT_TRIAGE_ADD, THREAT_TRIAGE_CONFIG, THREAT_TRIAGE_INIT,
THREAT_TRIAGE_PRINT, THREAT_TRIAGE_REMOVE, THREAT_TRIAGE_SCORE,
THREAT_TRIAGE_SET_AGGREGATOR​


​(2) Mock-up a telemetry message that we will triage and score.

[Stellar]>>> msg := SHELL_EDIT()
[Stellar]>>> msg
{
"test3": "No"
}​


(
​3​
) I
​copied
 the rules that you included in your email.
​  You can either pass in the entire configuration like this or build up
the rule set incrementally using THREAT_TRIAGE_ADD.​

[Stellar]>>> conf := SHELL_EDIT()
[Stellar]>>> conf
{
  "enrichment" : {
    "fieldMap" : { },
    "fieldToTypeMap" : { },
    "config" : { }
  },
  "threatIntel": {
   "fieldMap": {},
     "fieldToTypeMap": {},
     "config": {},
     "triageConfig": {
       "riskLevelRules": [
         {
           "name": "Rule1",
           "comment": "Checks whatever 1.",
           "rule": "test == \"false\"",
           "score": 20,
           "reason": null
         },
         {
           "name": "Rule1",
           "comment": "Checks whatever 2.",
           "rule": "test2 == \"False\"",
           "score": 20,
           "reason": null
         },
         {
           "name": "Rule3",
           "comment": "Checks whatever 2.",
           "rule": "test3 == \"No\"",
           "score": 20,
           "reason": null
         }
       ],
       "aggregator": "SUM",
       "aggregationConfig": {}
     }
   },
   "configuration": {}
}



​(4) Initialize Threat Triage and review your rule set.

[Stellar]>>> t := THREAT_TRIAGE_INIT(conf)
[Stellar]>>> THREAT_TRIAGE_PRINT(t)
╔═══════╤════════════════════╤══════════════════╤═══════╤════════╗
║ Name  │ Comment            │ Triage Rule      │ Score │ Reason ║
╠═══════╪════════════════════╪══════════════════╪═══════╪════════╣
║ Rule1 │ Checks whatever 1. │ test == "false"  │ 20    │        ║
╟───────┼────────────────────┼──────────────────┼───────┼────────╢
║ Rule1 │ Checks whatever 2. │ test2 == "False" │ 20    │        ║
╟───────┼────────────────────┼──────────────────┼───────┼────────╢
║ Rule3 │ Checks whatever 2. │ test3 == "No"    │ 20    │        ║
╚═══════╧════════════════════╧══════════════════╧═══════╧════════╝
Aggregation: SUM

​​
​(5) Score the message.  From this you can see in detail that the total
threat triage score is 20, which was the sum of a single rule that fired;
in this case Rule3.

[Stellar]>>> THREAT_TRIAGE_SCORE(msg, t)
{score=20.0, aggregator=SUM, rules=[{score=20, name=Rule3, rule=test3 ==
"No", comment=Checks whatever 2.}]} ​



​I also noticed as I was putting together this demo that the Triage
Debugger does not honor the `is_alert` field and so behaves slightly
differently than when running in the Enrichment topology.  I should fix
that. :)​



[1] https://github.com/apache/metron/pull/733




On Mon, Sep 25, 2017 at 1:46 PM Laurens Vets <la...@daemon.be> wrote:

> I have the following configuration:
>
> "threatIntel": {
>    "fieldMap": {},
>      "fieldToTypeMap": {},
>      "config": {},
>      "triageConfig": {
>        "riskLevelRules": [
>          {
>            "name": "Rule1",
>            "comment": "Checks whatever 1.",
>            "rule": "test == \"false\"",
>            "score": 20,
>            "reason": null
>          },
>          {
>            "name": "Rule1",
>            "comment": "Checks whatever 2.",
>            "rule": "test2 == \"False\"",
>            "score": 20,
>            "reason": null
>          },
>          {
>            "name": "Rule3",
>            "comment": "Checks whatever 2.",
>            "rule": "test3 == \"No\"",
>            "score": 20,
>            "reason": null
>          }
>        ],
>        "aggregator": "SUM",
>        "aggregationConfig": {}
>      }
> },
>
> I have no additional configuration in enrichment besides filling a
> specific with true or false based on a Stellar expression.
>
> I expected that when events would match my above rules, the _score field
> would be filled in. That does not seem to be the case.
>
> Does anyone know what I might be missing?
>

Re: Not seeing any Metron alerts.

Posted by Laurens Vets <la...@daemon.be>.
I was under the impression that besides setting the is_alert field,
there would now also be a threat:triage:level created by my
riskLevelRules rules? A SUMmed 'score' as it were? 

On 2017-09-25 16:46, Zeolla@GMail.com wrote:

> I was quickly reading through this on my mobile device so sorry if I'm off base here, but it may be because threat.triage.level is changed to threat:triage:level just before indexing due to the inability to use a period in keys on older versions of ES.  Not sure exactly what you mean by you don't get a threat.triage.level field. 
> 
> Jon 
> 
> On Mon, Sep 25, 2017, 19:34 Laurens Vets <la...@daemon.be> wrote: 
> 
>> Next problem:
>> 
>> I'm setting the "is_alert" field to true. It shows up in Kibana, but I
>> don't get a threat.triage.level field which means that either my
>> riskLevelRules rules don't trigger or something else goes wrong.
>> 
>> How and where can I look for additional information on why my rules
>> might not be working? (Metron UI accepts my JSON without issues)
>> 
>> On 2017-09-25 13:39, Laurens Vets wrote:
>>> Thanks!
>>> 
>>> On 2017-09-25 13:16, Simon Elliston Ball wrote:
>>>> The second statement overwrites the first, but also uses the previous
>>>> value.
>>>> 
>>>> Technically that is an or. Note this construct is designed to allow
>>>> multiple different trigger conditions to make is_alert true, hence the
>>>> second one being is_alert := is_alert || something_else.
>>>> 
>>>> && is bitwise and
>>>> || is bitwise or
>>>> 
>>>> Simon
>>>> 
>>>>> On 25 Sep 2017, at 21:12, Laurens Vets <la...@daemon.be> wrote:
>>>>> 
>>>>> Thanks! Followup question, the below is_alert 'rules' in the snippet
>>>>> from
>>>>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html,
>>>>> are those an AND or OR?
>>>>> 
>>>>> "threatIntel": {
>>>>> "fieldMap": {
>>>>> "stellar" : {
>>>>> "config" : [
>>>>> "geo_distance_distr:= STATS_MERGE( PROFILE_GET(
>>>>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2,
>>>>> 'MINUTES')))",
>>>>> "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>>>>> "dist_sd := STATS_SD(geo_distance_distr)",
>>>>> "geo_outlier := ABS(dist_median - geo_distance) >=
>>>>> 5*dist_sd",
>>>>> "is_alert := exists(is_alert) && is_alert",
>>>>> "is_alert := is_alert || (geo_outlier != null && geo_outlier
>>>>> == true)",
>>>>> "geo_distance_distr := null"
>>>>> ]
>>>>> }
>>>>> },
>>>>> 
>>>>> For instance, can the 2nd is_alert line overwrite the value assigned
>>>>> in the first is_alert rule?
>>>>> 
>>>>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>>>>>> Usually you would have the is_alert set based on more complex rules,
>>>>>> and then potentially have different rules to determine the
>>>>>> importance
>>>>>> of the alert, so they do tend to serve different purposes.
>>>>>> For example a triage rule might be set on levels of an indicator
>>>>>> after
>>>>>> is_alert has been triggered by a simple presence of a non-zero
>>>>>> result
>>>>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>>>>>> different rule levels. We're adding the ability to make score a
>>>>>> stellar statement which simplifies this further by allowing score to
>>>>>> be a function, but thresholds are still useful to determine the text
>>>>>> content of the alert for example.
>>>>>> Simon
>>>>>>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
>>>>>>> Oh, I didn't know I had to set is_alert to True.
>>>>>>> Doesn't that mean that we have to add all rules twice? First to
>>>>>>> check whether is_alert needs to be set to True. Next to apply the
>>>>>>> actual scores?
>>>>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>>>>>>> the _score field is actually an elastic search matching score
>>>>>>>> field,
>>>>>>>> and is not relevant to metron. You should see the scores in the
>>>>>>>> threat:triage:score field. However, your rules will only be run if
>>>>>>>> the
>>>>>>>> telemetry has is_alert set true, so you should ensure that the
>>>>>>>> enrichment phase sets is_alert: true somewhere for alerts you want
>>>>>>>> to
>>>>>>>> go to triage?
>>>>>>>> Simon
>>>>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>>>>>>>> I have the following configuration:
>>>>>>>>> "threatIntel": {
>>>>>>>>> "fieldMap": {},
>>>>>>>>> "fieldToTypeMap": {},
>>>>>>>>> "config": {},
>>>>>>>>> "triageConfig": {
>>>>>>>>> "riskLevelRules": [
>>>>>>>>> {
>>>>>>>>> "name": "Rule1",
>>>>>>>>> "comment": "Checks whatever 1.",
>>>>>>>>> "rule": "test == \"false\"",
>>>>>>>>> "score": 20,
>>>>>>>>> "reason": null
>>>>>>>>> },
>>>>>>>>> {
>>>>>>>>> "name": "Rule1",
>>>>>>>>> "comment": "Checks whatever 2.",
>>>>>>>>> "rule": "test2 == \"False\"",
>>>>>>>>> "score": 20,
>>>>>>>>> "reason": null
>>>>>>>>> },
>>>>>>>>> {
>>>>>>>>> "name": "Rule3",
>>>>>>>>> "comment": "Checks whatever 2.",
>>>>>>>>> "rule": "test3 == \"No\"",
>>>>>>>>> "score": 20,
>>>>>>>>> "reason": null
>>>>>>>>> }
>>>>>>>>> ],
>>>>>>>>> "aggregator": "SUM",
>>>>>>>>> "aggregationConfig": {}
>>>>>>>>> }
>>>>>>>>> },
>>>>>>>>> I have no additional configuration in enrichment besides filling
>>>>>>>>> a specific with true or false based on a Stellar expression.
>>>>>>>>> I expected that when events would match my above rules, the
>>>>>>>>> _score field would be filled in. That does not seem to be the
>>>>>>>>> case.
>>>>>>>>> Does anyone know what I might be missing?
> 
> -- 
> 
> Jon

Re: Not seeing any Metron alerts.

Posted by Simon Elliston Ball <si...@simonellistonball.com>.
Probably, though there are things (unlikely things) you can do to templates that would prevent that. 

> On 26 Sep 2017, at 17:25, Laurens Vets <la...@daemon.be> wrote:
> 
> Why would I need to update my ES template? I should see the field (possibly with the wrong type) anyways in the event after I refreshed the fields in Kibana right?
> 
> On 2017-09-26 09:16, Simon Elliston Ball wrote:
> 
>> There should be, though you may need to update your templates in ES if you've got any custom templates there, and make sure you refresh the fields in kibana's index config. 
>>  
>> Simon
>>  
>> 
>>> On 26 Sep 2017, at 17:13, Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>>> 
>>> After setting is_alert to true, this field is now shown in my event in Kibana. I would expect there also to be a field "threat:triage:level" in those same events (if my rules work?)
>>> 
>>> On 2017-09-25 16:46, Zeolla@GMail.com <ma...@GMail.com> wrote:
>>> 
>>> I was quickly reading through this on my mobile device so sorry if I'm off base here, but it may be because threat.triage.level is changed to threat:triage:level just before indexing due to the inability to use a period in keys on older versions of ES.  Not sure exactly what you mean by you don't get a threat.triage.level field.
>>> 
>>> Jon
>>> 
>>> 
>>> On Mon, Sep 25, 2017, 19:34 Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>>> Next problem:
>>> 
>>> I'm setting the "is_alert" field to true. It shows up in Kibana, but I
>>> don't get a threat.triage.level field which means that either my
>>> riskLevelRules rules don't trigger or something else goes wrong.
>>> 
>>> How and where can I look for additional information on why my rules
>>> might not be working? (Metron UI accepts my JSON without issues)
>>> 
>>> On 2017-09-25 13:39, Laurens Vets wrote:
>>> > Thanks!
>>> >
>>> > On 2017-09-25 13:16, Simon Elliston Ball wrote:
>>> >> The second statement overwrites the first, but also uses the previous
>>> >> value.
>>> >>
>>> >> Technically that is an or. Note this construct is designed to allow
>>> >> multiple different trigger conditions to make is_alert true, hence the
>>> >> second one being is_alert := is_alert || something_else.
>>> >>
>>> >> && is bitwise and
>>> >> || is bitwise or
>>> >>
>>> >> Simon
>>> >>
>>> >>> On 25 Sep 2017, at 21:12, Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>>> >>>
>>> >>> Thanks! Followup question, the below is_alert 'rules' in the snippet
>>> >>> from
>>> >>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html <http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html>,
>>> >>> are those an AND or OR?
>>> >>>
>>> >>>  "threatIntel": {
>>> >>>    "fieldMap": {
>>> >>>      "stellar" : {
>>> >>>        "config" : [
>>> >>>          "geo_distance_distr:= STATS_MERGE( PROFILE_GET(
>>> >>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2,
>>> >>> 'MINUTES')))",
>>> >>>          "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>>> >>>          "dist_sd := STATS_SD(geo_distance_distr)",
>>> >>>          "geo_outlier := ABS(dist_median - geo_distance) >=
>>> >>> 5*dist_sd",
>>> >>>          "is_alert := exists(is_alert) && is_alert",
>>> >>>          "is_alert := is_alert || (geo_outlier != null && geo_outlier
>>> >>> == true)",
>>> >>>          "geo_distance_distr := null"
>>> >>>        ]
>>> >>>      }
>>> >>>    },
>>> >>>
>>> >>> For instance, can the 2nd is_alert line overwrite the value assigned
>>> >>> in the first is_alert rule?
>>> >>>
>>> >>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>>> >>>> Usually you would have the is_alert set based on more complex rules,
>>> >>>> and then potentially have different rules to determine the
>>> >>>> importance
>>> >>>> of the alert, so they do tend to serve different purposes.
>>> >>>> For example a triage rule might be set on levels of an indicator
>>> >>>> after
>>> >>>> is_alert has been triggered by a simple presence of a non-zero
>>> >>>> result
>>> >>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>>> >>>> different rule levels. We're adding the ability to make score a
>>> >>>> stellar statement which simplifies this further by allowing score to
>>> >>>> be a function, but thresholds are still useful to determine the text
>>> >>>> content of the alert for example.
>>> >>>> Simon
>>> >>>>> On 25 Sep 2017, at 19:09, Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>>> >>>>> Oh, I didn't know I had to set is_alert to True.
>>> >>>>> Doesn't that mean that we have to add all rules twice? First to
>>> >>>>> check whether is_alert needs to be set to True. Next to apply the
>>> >>>>> actual scores?
>>> >>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>> >>>>>> the _score field is actually an elastic search matching score
>>> >>>>>> field,
>>> >>>>>> and is not relevant to metron. You should see the scores in the
>>> >>>>>> threat:triage:score field. However, your rules will only be run if
>>> >>>>>> the
>>> >>>>>> telemetry has is_alert set true, so you should ensure that the
>>> >>>>>> enrichment phase sets is_alert: true somewhere for alerts you want
>>> >>>>>> to
>>> >>>>>> go to triage?
>>> >>>>>> Simon
>>> >>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>>> >>>>>>> I have the following configuration:
>>> >>>>>>> "threatIntel": {
>>> >>>>>>> "fieldMap": {},
>>> >>>>>>>  "fieldToTypeMap": {},
>>> >>>>>>>  "config": {},
>>> >>>>>>>  "triageConfig": {
>>> >>>>>>>    "riskLevelRules": [
>>> >>>>>>>      {
>>> >>>>>>>        "name": "Rule1",
>>> >>>>>>>        "comment": "Checks whatever 1.",
>>> >>>>>>>        "rule": "test == \"false\"",
>>> >>>>>>>        "score": 20,
>>> >>>>>>>        "reason": null
>>> >>>>>>>      },
>>> >>>>>>>      {
>>> >>>>>>>        "name": "Rule1",
>>> >>>>>>>        "comment": "Checks whatever 2.",
>>> >>>>>>>        "rule": "test2 == \"False\"",
>>> >>>>>>>        "score": 20,
>>> >>>>>>>        "reason": null
>>> >>>>>>>      },
>>> >>>>>>>      {
>>> >>>>>>>        "name": "Rule3",
>>> >>>>>>>        "comment": "Checks whatever 2.",
>>> >>>>>>>        "rule": "test3 == \"No\"",
>>> >>>>>>>        "score": 20,
>>> >>>>>>>        "reason": null
>>> >>>>>>>      }
>>> >>>>>>>    ],
>>> >>>>>>>    "aggregator": "SUM",
>>> >>>>>>>    "aggregationConfig": {}
>>> >>>>>>>  }
>>> >>>>>>> },
>>> >>>>>>> I have no additional configuration in enrichment besides filling
>>> >>>>>>> a specific with true or false based on a Stellar expression.
>>> >>>>>>> I expected that when events would match my above rules, the
>>> >>>>>>> _score field would be filled in. That does not seem to be the
>>> >>>>>>> case.
>>> >>>>>>> Does anyone know what I might be missing?
>>> --
>>> Jon
>>> 
>>> 
>>> 
> 


Re: Not seeing any Metron alerts.

Posted by Laurens Vets <la...@daemon.be>.
Why would I need to update my ES template? I should see the field
(possibly with the wrong type) anyways in the event after I refreshed
the fields in Kibana right? 

On 2017-09-26 09:16, Simon Elliston Ball wrote:

> There should be, though you may need to update your templates in ES if you've got any custom templates there, and make sure you refresh the fields in kibana's index config.  
> 
> Simon 
> 
> On 26 Sep 2017, at 17:13, Laurens Vets <la...@daemon.be> wrote: 
> 
> After setting is_alert to true, this field is now shown in my event in Kibana. I would expect there also to be a field "threat:triage:level" in those same events (if my rules work?) 
> 
> On 2017-09-25 16:46, Zeolla@GMail.com wrote: 
> 
> I was quickly reading through this on my mobile device so sorry if I'm off base here, but it may be because threat.triage.level is changed to threat:triage:level just before indexing due to the inability to use a period in keys on older versions of ES.  Not sure exactly what you mean by you don't get a threat.triage.level field. 
> 
> Jon 
> 
> On Mon, Sep 25, 2017, 19:34 Laurens Vets <la...@daemon.be> wrote: Next problem:
> 
> I'm setting the "is_alert" field to true. It shows up in Kibana, but I
> don't get a threat.triage.level field which means that either my
> riskLevelRules rules don't trigger or something else goes wrong.
> 
> How and where can I look for additional information on why my rules
> might not be working? (Metron UI accepts my JSON without issues)
> 
> On 2017-09-25 13:39, Laurens Vets wrote:
>> Thanks!
>> 
>> On 2017-09-25 13:16, Simon Elliston Ball wrote:
>>> The second statement overwrites the first, but also uses the previous
>>> value.
>>> 
>>> Technically that is an or. Note this construct is designed to allow
>>> multiple different trigger conditions to make is_alert true, hence the
>>> second one being is_alert := is_alert || something_else.
>>> 
>>> && is bitwise and
>>> || is bitwise or
>>> 
>>> Simon
>>> 
>>>> On 25 Sep 2017, at 21:12, Laurens Vets <la...@daemon.be> wrote:
>>>> 
>>>> Thanks! Followup question, the below is_alert 'rules' in the snippet
>>>> from
>>>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html,
>>>> are those an AND or OR?
>>>> 
>>>> "threatIntel": {
>>>> "fieldMap": {
>>>> "stellar" : {
>>>> "config" : [
>>>> "geo_distance_distr:= STATS_MERGE( PROFILE_GET(
>>>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2,
>>>> 'MINUTES')))",
>>>> "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>>>> "dist_sd := STATS_SD(geo_distance_distr)",
>>>> "geo_outlier := ABS(dist_median - geo_distance) >=
>>>> 5*dist_sd",
>>>> "is_alert := exists(is_alert) && is_alert",
>>>> "is_alert := is_alert || (geo_outlier != null && geo_outlier
>>>> == true)",
>>>> "geo_distance_distr := null"
>>>> ]
>>>> }
>>>> },
>>>> 
>>>> For instance, can the 2nd is_alert line overwrite the value assigned
>>>> in the first is_alert rule?
>>>> 
>>>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>>>>> Usually you would have the is_alert set based on more complex rules,
>>>>> and then potentially have different rules to determine the
>>>>> importance
>>>>> of the alert, so they do tend to serve different purposes.
>>>>> For example a triage rule might be set on levels of an indicator
>>>>> after
>>>>> is_alert has been triggered by a simple presence of a non-zero
>>>>> result
>>>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>>>>> different rule levels. We're adding the ability to make score a
>>>>> stellar statement which simplifies this further by allowing score to
>>>>> be a function, but thresholds are still useful to determine the text
>>>>> content of the alert for example.
>>>>> Simon
>>>>>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
>>>>>> Oh, I didn't know I had to set is_alert to True.
>>>>>> Doesn't that mean that we have to add all rules twice? First to
>>>>>> check whether is_alert needs to be set to True. Next to apply the
>>>>>> actual scores?
>>>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>>>>>> the _score field is actually an elastic search matching score
>>>>>>> field,
>>>>>>> and is not relevant to metron. You should see the scores in the
>>>>>>> threat:triage:score field. However, your rules will only be run if
>>>>>>> the
>>>>>>> telemetry has is_alert set true, so you should ensure that the
>>>>>>> enrichment phase sets is_alert: true somewhere for alerts you want
>>>>>>> to
>>>>>>> go to triage?
>>>>>>> Simon
>>>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>>>>>>> I have the following configuration:
>>>>>>>> "threatIntel": {
>>>>>>>> "fieldMap": {},
>>>>>>>> "fieldToTypeMap": {},
>>>>>>>> "config": {},
>>>>>>>> "triageConfig": {
>>>>>>>> "riskLevelRules": [
>>>>>>>> {
>>>>>>>> "name": "Rule1",
>>>>>>>> "comment": "Checks whatever 1.",
>>>>>>>> "rule": "test == \"false\"",
>>>>>>>> "score": 20,
>>>>>>>> "reason": null
>>>>>>>> },
>>>>>>>> {
>>>>>>>> "name": "Rule1",
>>>>>>>> "comment": "Checks whatever 2.",
>>>>>>>> "rule": "test2 == \"False\"",
>>>>>>>> "score": 20,
>>>>>>>> "reason": null
>>>>>>>> },
>>>>>>>> {
>>>>>>>> "name": "Rule3",
>>>>>>>> "comment": "Checks whatever 2.",
>>>>>>>> "rule": "test3 == \"No\"",
>>>>>>>> "score": 20,
>>>>>>>> "reason": null
>>>>>>>> }
>>>>>>>> ],
>>>>>>>> "aggregator": "SUM",
>>>>>>>> "aggregationConfig": {}
>>>>>>>> }
>>>>>>>> },
>>>>>>>> I have no additional configuration in enrichment besides filling
>>>>>>>> a specific with true or false based on a Stellar expression.
>>>>>>>> I expected that when events would match my above rules, the
>>>>>>>> _score field would be filled in. That does not seem to be the
>>>>>>>> case.
>>>>>>>> Does anyone know what I might be missing? 
> -- 
> 
> Jon

Re: Not seeing any Metron alerts.

Posted by Simon Elliston Ball <si...@simonellistonball.com>.
There should be, though you may need to update your templates in ES if you’ve got any custom templates there, and make sure you refresh the fields in kibana’s index config. 

Simon


> On 26 Sep 2017, at 17:13, Laurens Vets <la...@daemon.be> wrote:
> 
> After setting is_alert to true, this field is now shown in my event in Kibana. I would expect there also to be a field "threat:triage:level" in those same events (if my rules work?)
> 
> On 2017-09-25 16:46, Zeolla@GMail.com wrote:
> 
>> I was quickly reading through this on my mobile device so sorry if I'm off base here, but it may be because threat.triage.level is changed to threat:triage:level just before indexing due to the inability to use a period in keys on older versions of ES.  Not sure exactly what you mean by you don't get a threat.triage.level field.
>> 
>> Jon
>> 
>> 
>> On Mon, Sep 25, 2017, 19:34 Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>> Next problem:
>> 
>> I'm setting the "is_alert" field to true. It shows up in Kibana, but I
>> don't get a threat.triage.level field which means that either my
>> riskLevelRules rules don't trigger or something else goes wrong.
>> 
>> How and where can I look for additional information on why my rules
>> might not be working? (Metron UI accepts my JSON without issues)
>> 
>> On 2017-09-25 13:39, Laurens Vets wrote:
>> > Thanks!
>> >
>> > On 2017-09-25 13:16, Simon Elliston Ball wrote:
>> >> The second statement overwrites the first, but also uses the previous
>> >> value.
>> >>
>> >> Technically that is an or. Note this construct is designed to allow
>> >> multiple different trigger conditions to make is_alert true, hence the
>> >> second one being is_alert := is_alert || something_else.
>> >>
>> >> && is bitwise and
>> >> || is bitwise or
>> >>
>> >> Simon
>> >>
>> >>> On 25 Sep 2017, at 21:12, Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>> >>>
>> >>> Thanks! Followup question, the below is_alert 'rules' in the snippet
>> >>> from
>> >>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html <http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html>,
>> >>> are those an AND or OR?
>> >>>
>> >>>  "threatIntel": {
>> >>>    "fieldMap": {
>> >>>      "stellar" : {
>> >>>        "config" : [
>> >>>          "geo_distance_distr:= STATS_MERGE( PROFILE_GET(
>> >>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2,
>> >>> 'MINUTES')))",
>> >>>          "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>> >>>          "dist_sd := STATS_SD(geo_distance_distr)",
>> >>>          "geo_outlier := ABS(dist_median - geo_distance) >=
>> >>> 5*dist_sd",
>> >>>          "is_alert := exists(is_alert) && is_alert",
>> >>>          "is_alert := is_alert || (geo_outlier != null && geo_outlier
>> >>> == true)",
>> >>>          "geo_distance_distr := null"
>> >>>        ]
>> >>>      }
>> >>>    },
>> >>>
>> >>> For instance, can the 2nd is_alert line overwrite the value assigned
>> >>> in the first is_alert rule?
>> >>>
>> >>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>> >>>> Usually you would have the is_alert set based on more complex rules,
>> >>>> and then potentially have different rules to determine the
>> >>>> importance
>> >>>> of the alert, so they do tend to serve different purposes.
>> >>>> For example a triage rule might be set on levels of an indicator
>> >>>> after
>> >>>> is_alert has been triggered by a simple presence of a non-zero
>> >>>> result
>> >>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>> >>>> different rule levels. We're adding the ability to make score a
>> >>>> stellar statement which simplifies this further by allowing score to
>> >>>> be a function, but thresholds are still useful to determine the text
>> >>>> content of the alert for example.
>> >>>> Simon
>> >>>>> On 25 Sep 2017, at 19:09, Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>> >>>>> Oh, I didn't know I had to set is_alert to True.
>> >>>>> Doesn't that mean that we have to add all rules twice? First to
>> >>>>> check whether is_alert needs to be set to True. Next to apply the
>> >>>>> actual scores?
>> >>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>> >>>>>> the _score field is actually an elastic search matching score
>> >>>>>> field,
>> >>>>>> and is not relevant to metron. You should see the scores in the
>> >>>>>> threat:triage:score field. However, your rules will only be run if
>> >>>>>> the
>> >>>>>> telemetry has is_alert set true, so you should ensure that the
>> >>>>>> enrichment phase sets is_alert: true somewhere for alerts you want
>> >>>>>> to
>> >>>>>> go to triage?
>> >>>>>> Simon
>> >>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <laurens@daemon.be <ma...@daemon.be>> wrote:
>> >>>>>>> I have the following configuration:
>> >>>>>>> "threatIntel": {
>> >>>>>>> "fieldMap": {},
>> >>>>>>>  "fieldToTypeMap": {},
>> >>>>>>>  "config": {},
>> >>>>>>>  "triageConfig": {
>> >>>>>>>    "riskLevelRules": [
>> >>>>>>>      {
>> >>>>>>>        "name": "Rule1",
>> >>>>>>>        "comment": "Checks whatever 1.",
>> >>>>>>>        "rule": "test == \"false\"",
>> >>>>>>>        "score": 20,
>> >>>>>>>        "reason": null
>> >>>>>>>      },
>> >>>>>>>      {
>> >>>>>>>        "name": "Rule1",
>> >>>>>>>        "comment": "Checks whatever 2.",
>> >>>>>>>        "rule": "test2 == \"False\"",
>> >>>>>>>        "score": 20,
>> >>>>>>>        "reason": null
>> >>>>>>>      },
>> >>>>>>>      {
>> >>>>>>>        "name": "Rule3",
>> >>>>>>>        "comment": "Checks whatever 2.",
>> >>>>>>>        "rule": "test3 == \"No\"",
>> >>>>>>>        "score": 20,
>> >>>>>>>        "reason": null
>> >>>>>>>      }
>> >>>>>>>    ],
>> >>>>>>>    "aggregator": "SUM",
>> >>>>>>>    "aggregationConfig": {}
>> >>>>>>>  }
>> >>>>>>> },
>> >>>>>>> I have no additional configuration in enrichment besides filling
>> >>>>>>> a specific with true or false based on a Stellar expression.
>> >>>>>>> I expected that when events would match my above rules, the
>> >>>>>>> _score field would be filled in. That does not seem to be the
>> >>>>>>> case.
>> >>>>>>> Does anyone know what I might be missing?
>> --
>> Jon
>> 
> 


Re: Not seeing any Metron alerts.

Posted by Laurens Vets <la...@daemon.be>.
After setting is_alert to true, this field is now shown in my event in
Kibana. I would expect there also to be a field "threat:triage:level" in
those same events (if my rules work?) 

On 2017-09-25 16:46, Zeolla@GMail.com wrote:

> I was quickly reading through this on my mobile device so sorry if I'm off base here, but it may be because threat.triage.level is changed to threat:triage:level just before indexing due to the inability to use a period in keys on older versions of ES.  Not sure exactly what you mean by you don't get a threat.triage.level field. 
> 
> Jon 
> 
> On Mon, Sep 25, 2017, 19:34 Laurens Vets <la...@daemon.be> wrote: 
> 
>> Next problem:
>> 
>> I'm setting the "is_alert" field to true. It shows up in Kibana, but I
>> don't get a threat.triage.level field which means that either my
>> riskLevelRules rules don't trigger or something else goes wrong.
>> 
>> How and where can I look for additional information on why my rules
>> might not be working? (Metron UI accepts my JSON without issues)
>> 
>> On 2017-09-25 13:39, Laurens Vets wrote:
>>> Thanks!
>>> 
>>> On 2017-09-25 13:16, Simon Elliston Ball wrote:
>>>> The second statement overwrites the first, but also uses the previous
>>>> value.
>>>> 
>>>> Technically that is an or. Note this construct is designed to allow
>>>> multiple different trigger conditions to make is_alert true, hence the
>>>> second one being is_alert := is_alert || something_else.
>>>> 
>>>> && is bitwise and
>>>> || is bitwise or
>>>> 
>>>> Simon
>>>> 
>>>>> On 25 Sep 2017, at 21:12, Laurens Vets <la...@daemon.be> wrote:
>>>>> 
>>>>> Thanks! Followup question, the below is_alert 'rules' in the snippet
>>>>> from
>>>>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html,
>>>>> are those an AND or OR?
>>>>> 
>>>>> "threatIntel": {
>>>>> "fieldMap": {
>>>>> "stellar" : {
>>>>> "config" : [
>>>>> "geo_distance_distr:= STATS_MERGE( PROFILE_GET(
>>>>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2,
>>>>> 'MINUTES')))",
>>>>> "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>>>>> "dist_sd := STATS_SD(geo_distance_distr)",
>>>>> "geo_outlier := ABS(dist_median - geo_distance) >=
>>>>> 5*dist_sd",
>>>>> "is_alert := exists(is_alert) && is_alert",
>>>>> "is_alert := is_alert || (geo_outlier != null && geo_outlier
>>>>> == true)",
>>>>> "geo_distance_distr := null"
>>>>> ]
>>>>> }
>>>>> },
>>>>> 
>>>>> For instance, can the 2nd is_alert line overwrite the value assigned
>>>>> in the first is_alert rule?
>>>>> 
>>>>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>>>>>> Usually you would have the is_alert set based on more complex rules,
>>>>>> and then potentially have different rules to determine the
>>>>>> importance
>>>>>> of the alert, so they do tend to serve different purposes.
>>>>>> For example a triage rule might be set on levels of an indicator
>>>>>> after
>>>>>> is_alert has been triggered by a simple presence of a non-zero
>>>>>> result
>>>>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>>>>>> different rule levels. We're adding the ability to make score a
>>>>>> stellar statement which simplifies this further by allowing score to
>>>>>> be a function, but thresholds are still useful to determine the text
>>>>>> content of the alert for example.
>>>>>> Simon
>>>>>>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
>>>>>>> Oh, I didn't know I had to set is_alert to True.
>>>>>>> Doesn't that mean that we have to add all rules twice? First to
>>>>>>> check whether is_alert needs to be set to True. Next to apply the
>>>>>>> actual scores?
>>>>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>>>>>>> the _score field is actually an elastic search matching score
>>>>>>>> field,
>>>>>>>> and is not relevant to metron. You should see the scores in the
>>>>>>>> threat:triage:score field. However, your rules will only be run if
>>>>>>>> the
>>>>>>>> telemetry has is_alert set true, so you should ensure that the
>>>>>>>> enrichment phase sets is_alert: true somewhere for alerts you want
>>>>>>>> to
>>>>>>>> go to triage?
>>>>>>>> Simon
>>>>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>>>>>>>> I have the following configuration:
>>>>>>>>> "threatIntel": {
>>>>>>>>> "fieldMap": {},
>>>>>>>>> "fieldToTypeMap": {},
>>>>>>>>> "config": {},
>>>>>>>>> "triageConfig": {
>>>>>>>>> "riskLevelRules": [
>>>>>>>>> {
>>>>>>>>> "name": "Rule1",
>>>>>>>>> "comment": "Checks whatever 1.",
>>>>>>>>> "rule": "test == \"false\"",
>>>>>>>>> "score": 20,
>>>>>>>>> "reason": null
>>>>>>>>> },
>>>>>>>>> {
>>>>>>>>> "name": "Rule1",
>>>>>>>>> "comment": "Checks whatever 2.",
>>>>>>>>> "rule": "test2 == \"False\"",
>>>>>>>>> "score": 20,
>>>>>>>>> "reason": null
>>>>>>>>> },
>>>>>>>>> {
>>>>>>>>> "name": "Rule3",
>>>>>>>>> "comment": "Checks whatever 2.",
>>>>>>>>> "rule": "test3 == \"No\"",
>>>>>>>>> "score": 20,
>>>>>>>>> "reason": null
>>>>>>>>> }
>>>>>>>>> ],
>>>>>>>>> "aggregator": "SUM",
>>>>>>>>> "aggregationConfig": {}
>>>>>>>>> }
>>>>>>>>> },
>>>>>>>>> I have no additional configuration in enrichment besides filling
>>>>>>>>> a specific with true or false based on a Stellar expression.
>>>>>>>>> I expected that when events would match my above rules, the
>>>>>>>>> _score field would be filled in. That does not seem to be the
>>>>>>>>> case.
>>>>>>>>> Does anyone know what I might be missing?
> 
> -- 
> 
> Jon

Re: Not seeing any Metron alerts.

Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
I was quickly reading through this on my mobile device so sorry if I'm off
base here, but it may be because threat.triage.level is changed to
threat:triage:level just before indexing due to the inability to use a
period in keys on older versions of ES.  Not sure exactly what you mean by
you don't get a threat.triage.level field.

Jon

On Mon, Sep 25, 2017, 19:34 Laurens Vets <la...@daemon.be> wrote:

> Next problem:
>
> I'm setting the "is_alert" field to true. It shows up in Kibana, but I
> don't get a threat.triage.level field which means that either my
> riskLevelRules rules don't trigger or something else goes wrong.
>
> How and where can I look for additional information on why my rules
> might not be working? (Metron UI accepts my JSON without issues)
>
> On 2017-09-25 13:39, Laurens Vets wrote:
> > Thanks!
> >
> > On 2017-09-25 13:16, Simon Elliston Ball wrote:
> >> The second statement overwrites the first, but also uses the previous
> >> value.
> >>
> >> Technically that is an or. Note this construct is designed to allow
> >> multiple different trigger conditions to make is_alert true, hence the
> >> second one being is_alert := is_alert || something_else.
> >>
> >> && is bitwise and
> >> || is bitwise or
> >>
> >> Simon
> >>
> >>> On 25 Sep 2017, at 21:12, Laurens Vets <la...@daemon.be> wrote:
> >>>
> >>> Thanks! Followup question, the below is_alert 'rules' in the snippet
> >>> from
> >>>
> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html
> ,
> >>> are those an AND or OR?
> >>>
> >>>  "threatIntel": {
> >>>    "fieldMap": {
> >>>      "stellar" : {
> >>>        "config" : [
> >>>          "geo_distance_distr:= STATS_MERGE( PROFILE_GET(
> >>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2,
> >>> 'MINUTES')))",
> >>>          "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
> >>>          "dist_sd := STATS_SD(geo_distance_distr)",
> >>>          "geo_outlier := ABS(dist_median - geo_distance) >=
> >>> 5*dist_sd",
> >>>          "is_alert := exists(is_alert) && is_alert",
> >>>          "is_alert := is_alert || (geo_outlier != null && geo_outlier
> >>> == true)",
> >>>          "geo_distance_distr := null"
> >>>        ]
> >>>      }
> >>>    },
> >>>
> >>> For instance, can the 2nd is_alert line overwrite the value assigned
> >>> in the first is_alert rule?
> >>>
> >>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
> >>>> Usually you would have the is_alert set based on more complex rules,
> >>>> and then potentially have different rules to determine the
> >>>> importance
> >>>> of the alert, so they do tend to serve different purposes.
> >>>> For example a triage rule might be set on levels of an indicator
> >>>> after
> >>>> is_alert has been triggered by a simple presence of a non-zero
> >>>> result
> >>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
> >>>> different rule levels. We’re adding the ability to make score a
> >>>> stellar statement which simplifies this further by allowing score to
> >>>> be a function, but thresholds are still useful to determine the text
> >>>> content of the alert for example.
> >>>> Simon
> >>>>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
> >>>>> Oh, I didn't know I had to set is_alert to True.
> >>>>> Doesn't that mean that we have to add all rules twice? First to
> >>>>> check whether is_alert needs to be set to True. Next to apply the
> >>>>> actual scores?
> >>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
> >>>>>> the _score field is actually an elastic search matching score
> >>>>>> field,
> >>>>>> and is not relevant to metron. You should see the scores in the
> >>>>>> threat:triage:score field. However, your rules will only be run if
> >>>>>> the
> >>>>>> telemetry has is_alert set true, so you should ensure that the
> >>>>>> enrichment phase sets is_alert: true somewhere for alerts you want
> >>>>>> to
> >>>>>> go to triage?
> >>>>>> Simon
> >>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
> >>>>>>> I have the following configuration:
> >>>>>>> "threatIntel": {
> >>>>>>> "fieldMap": {},
> >>>>>>>  "fieldToTypeMap": {},
> >>>>>>>  "config": {},
> >>>>>>>  "triageConfig": {
> >>>>>>>    "riskLevelRules": [
> >>>>>>>      {
> >>>>>>>        "name": "Rule1",
> >>>>>>>        "comment": "Checks whatever 1.",
> >>>>>>>        "rule": "test == \"false\"",
> >>>>>>>        "score": 20,
> >>>>>>>        "reason": null
> >>>>>>>      },
> >>>>>>>      {
> >>>>>>>        "name": "Rule1",
> >>>>>>>        "comment": "Checks whatever 2.",
> >>>>>>>        "rule": "test2 == \"False\"",
> >>>>>>>        "score": 20,
> >>>>>>>        "reason": null
> >>>>>>>      },
> >>>>>>>      {
> >>>>>>>        "name": "Rule3",
> >>>>>>>        "comment": "Checks whatever 2.",
> >>>>>>>        "rule": "test3 == \"No\"",
> >>>>>>>        "score": 20,
> >>>>>>>        "reason": null
> >>>>>>>      }
> >>>>>>>    ],
> >>>>>>>    "aggregator": "SUM",
> >>>>>>>    "aggregationConfig": {}
> >>>>>>>  }
> >>>>>>> },
> >>>>>>> I have no additional configuration in enrichment besides filling
> >>>>>>> a specific with true or false based on a Stellar expression.
> >>>>>>> I expected that when events would match my above rules, the
> >>>>>>> _score field would be filled in. That does not seem to be the
> >>>>>>> case.
> >>>>>>> Does anyone know what I might be missing?
>
-- 

Jon

Re: Not seeing any Metron alerts.

Posted by Laurens Vets <la...@daemon.be>.
Next problem:

I'm setting the "is_alert" field to true. It shows up in Kibana, but I 
don't get a threat.triage.level field which means that either my 
riskLevelRules rules don't trigger or something else goes wrong.

How and where can I look for additional information on why my rules 
might not be working? (Metron UI accepts my JSON without issues)

On 2017-09-25 13:39, Laurens Vets wrote:
> Thanks!
> 
> On 2017-09-25 13:16, Simon Elliston Ball wrote:
>> The second statement overwrites the first, but also uses the previous 
>> value.
>> 
>> Technically that is an or. Note this construct is designed to allow
>> multiple different trigger conditions to make is_alert true, hence the
>> second one being is_alert := is_alert || something_else.
>> 
>> && is bitwise and
>> || is bitwise or
>> 
>> Simon
>> 
>>> On 25 Sep 2017, at 21:12, Laurens Vets <la...@daemon.be> wrote:
>>> 
>>> Thanks! Followup question, the below is_alert 'rules' in the snippet 
>>> from 
>>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html, 
>>> are those an AND or OR?
>>> 
>>>  "threatIntel": {
>>>    "fieldMap": {
>>>      "stellar" : {
>>>        "config" : [
>>>          "geo_distance_distr:= STATS_MERGE( PROFILE_GET( 
>>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, 
>>> 'MINUTES')))",
>>>          "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>>>          "dist_sd := STATS_SD(geo_distance_distr)",
>>>          "geo_outlier := ABS(dist_median - geo_distance) >= 
>>> 5*dist_sd",
>>>          "is_alert := exists(is_alert) && is_alert",
>>>          "is_alert := is_alert || (geo_outlier != null && geo_outlier 
>>> == true)",
>>>          "geo_distance_distr := null"
>>>        ]
>>>      }
>>>    },
>>> 
>>> For instance, can the 2nd is_alert line overwrite the value assigned 
>>> in the first is_alert rule?
>>> 
>>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>>>> Usually you would have the is_alert set based on more complex rules,
>>>> and then potentially have different rules to determine the 
>>>> importance
>>>> of the alert, so they do tend to serve different purposes.
>>>> For example a triage rule might be set on levels of an indicator 
>>>> after
>>>> is_alert has been triggered by a simple presence of a non-zero 
>>>> result
>>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>>>> different rule levels. We’re adding the ability to make score a
>>>> stellar statement which simplifies this further by allowing score to
>>>> be a function, but thresholds are still useful to determine the text
>>>> content of the alert for example.
>>>> Simon
>>>>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
>>>>> Oh, I didn't know I had to set is_alert to True.
>>>>> Doesn't that mean that we have to add all rules twice? First to 
>>>>> check whether is_alert needs to be set to True. Next to apply the 
>>>>> actual scores?
>>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>>>>> the _score field is actually an elastic search matching score 
>>>>>> field,
>>>>>> and is not relevant to metron. You should see the scores in the
>>>>>> threat:triage:score field. However, your rules will only be run if 
>>>>>> the
>>>>>> telemetry has is_alert set true, so you should ensure that the
>>>>>> enrichment phase sets is_alert: true somewhere for alerts you want 
>>>>>> to
>>>>>> go to triage?
>>>>>> Simon
>>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>>>>>> I have the following configuration:
>>>>>>> "threatIntel": {
>>>>>>> "fieldMap": {},
>>>>>>>  "fieldToTypeMap": {},
>>>>>>>  "config": {},
>>>>>>>  "triageConfig": {
>>>>>>>    "riskLevelRules": [
>>>>>>>      {
>>>>>>>        "name": "Rule1",
>>>>>>>        "comment": "Checks whatever 1.",
>>>>>>>        "rule": "test == \"false\"",
>>>>>>>        "score": 20,
>>>>>>>        "reason": null
>>>>>>>      },
>>>>>>>      {
>>>>>>>        "name": "Rule1",
>>>>>>>        "comment": "Checks whatever 2.",
>>>>>>>        "rule": "test2 == \"False\"",
>>>>>>>        "score": 20,
>>>>>>>        "reason": null
>>>>>>>      },
>>>>>>>      {
>>>>>>>        "name": "Rule3",
>>>>>>>        "comment": "Checks whatever 2.",
>>>>>>>        "rule": "test3 == \"No\"",
>>>>>>>        "score": 20,
>>>>>>>        "reason": null
>>>>>>>      }
>>>>>>>    ],
>>>>>>>    "aggregator": "SUM",
>>>>>>>    "aggregationConfig": {}
>>>>>>>  }
>>>>>>> },
>>>>>>> I have no additional configuration in enrichment besides filling 
>>>>>>> a specific with true or false based on a Stellar expression.
>>>>>>> I expected that when events would match my above rules, the 
>>>>>>> _score field would be filled in. That does not seem to be the 
>>>>>>> case.
>>>>>>> Does anyone know what I might be missing?

Re: Not seeing any Metron alerts.

Posted by Laurens Vets <la...@daemon.be>.
Thanks!

On 2017-09-25 13:16, Simon Elliston Ball wrote:
> The second statement overwrites the first, but also uses the previous 
> value.
> 
> Technically that is an or. Note this construct is designed to allow
> multiple different trigger conditions to make is_alert true, hence the
> second one being is_alert := is_alert || something_else.
> 
> && is bitwise and
> || is bitwise or
> 
> Simon
> 
>> On 25 Sep 2017, at 21:12, Laurens Vets <la...@daemon.be> wrote:
>> 
>> Thanks! Followup question, the below is_alert 'rules' in the snippet 
>> from 
>> http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html, 
>> are those an AND or OR?
>> 
>>  "threatIntel": {
>>    "fieldMap": {
>>      "stellar" : {
>>        "config" : [
>>          "geo_distance_distr:= STATS_MERGE( PROFILE_GET( 
>> 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, 
>> 'MINUTES')))",
>>          "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>>          "dist_sd := STATS_SD(geo_distance_distr)",
>>          "geo_outlier := ABS(dist_median - geo_distance) >= 
>> 5*dist_sd",
>>          "is_alert := exists(is_alert) && is_alert",
>>          "is_alert := is_alert || (geo_outlier != null && geo_outlier 
>> == true)",
>>          "geo_distance_distr := null"
>>        ]
>>      }
>>    },
>> 
>> For instance, can the 2nd is_alert line overwrite the value assigned 
>> in the first is_alert rule?
>> 
>> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>>> Usually you would have the is_alert set based on more complex rules,
>>> and then potentially have different rules to determine the importance
>>> of the alert, so they do tend to serve different purposes.
>>> For example a triage rule might be set on levels of an indicator 
>>> after
>>> is_alert has been triggered by a simple presence of a non-zero result
>>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>>> different rule levels. We’re adding the ability to make score a
>>> stellar statement which simplifies this further by allowing score to
>>> be a function, but thresholds are still useful to determine the text
>>> content of the alert for example.
>>> Simon
>>>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
>>>> Oh, I didn't know I had to set is_alert to True.
>>>> Doesn't that mean that we have to add all rules twice? First to 
>>>> check whether is_alert needs to be set to True. Next to apply the 
>>>> actual scores?
>>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>>>> the _score field is actually an elastic search matching score 
>>>>> field,
>>>>> and is not relevant to metron. You should see the scores in the
>>>>> threat:triage:score field. However, your rules will only be run if 
>>>>> the
>>>>> telemetry has is_alert set true, so you should ensure that the
>>>>> enrichment phase sets is_alert: true somewhere for alerts you want 
>>>>> to
>>>>> go to triage?
>>>>> Simon
>>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>>>>> I have the following configuration:
>>>>>> "threatIntel": {
>>>>>> "fieldMap": {},
>>>>>>  "fieldToTypeMap": {},
>>>>>>  "config": {},
>>>>>>  "triageConfig": {
>>>>>>    "riskLevelRules": [
>>>>>>      {
>>>>>>        "name": "Rule1",
>>>>>>        "comment": "Checks whatever 1.",
>>>>>>        "rule": "test == \"false\"",
>>>>>>        "score": 20,
>>>>>>        "reason": null
>>>>>>      },
>>>>>>      {
>>>>>>        "name": "Rule1",
>>>>>>        "comment": "Checks whatever 2.",
>>>>>>        "rule": "test2 == \"False\"",
>>>>>>        "score": 20,
>>>>>>        "reason": null
>>>>>>      },
>>>>>>      {
>>>>>>        "name": "Rule3",
>>>>>>        "comment": "Checks whatever 2.",
>>>>>>        "rule": "test3 == \"No\"",
>>>>>>        "score": 20,
>>>>>>        "reason": null
>>>>>>      }
>>>>>>    ],
>>>>>>    "aggregator": "SUM",
>>>>>>    "aggregationConfig": {}
>>>>>>  }
>>>>>> },
>>>>>> I have no additional configuration in enrichment besides filling a 
>>>>>> specific with true or false based on a Stellar expression.
>>>>>> I expected that when events would match my above rules, the _score 
>>>>>> field would be filled in. That does not seem to be the case.
>>>>>> Does anyone know what I might be missing?

Re: Not seeing any Metron alerts.

Posted by Simon Elliston Ball <si...@simonellistonball.com>.
The second statement overwrites the first, but also uses the previous value. 

Technically that is an or. Note this construct is designed to allow multiple different trigger conditions to make is_alert true, hence the second one being is_alert := is_alert || something_else. 

&& is bitwise and
|| is bitwise or

Simon

> On 25 Sep 2017, at 21:12, Laurens Vets <la...@daemon.be> wrote:
> 
> Thanks! Followup question, the below is_alert 'rules' in the snippet from http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html, are those an AND or OR?
> 
>  "threatIntel": {
>    "fieldMap": {
>      "stellar" : {
>        "config" : [
>          "geo_distance_distr:= STATS_MERGE( PROFILE_GET( 'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, 'MINUTES')))",
>          "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
>          "dist_sd := STATS_SD(geo_distance_distr)",
>          "geo_outlier := ABS(dist_median - geo_distance) >= 5*dist_sd",
>          "is_alert := exists(is_alert) && is_alert",
>          "is_alert := is_alert || (geo_outlier != null && geo_outlier == true)",
>          "geo_distance_distr := null"
>        ]
>      }
>    },
> 
> For instance, can the 2nd is_alert line overwrite the value assigned in the first is_alert rule?
> 
> On 2017-09-25 11:13, Simon Elliston Ball wrote:
>> Usually you would have the is_alert set based on more complex rules,
>> and then potentially have different rules to determine the importance
>> of the alert, so they do tend to serve different purposes.
>> For example a triage rule might be set on levels of an indicator after
>> is_alert has been triggered by a simple presence of a non-zero result
>> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
>> different rule levels. We’re adding the ability to make score a
>> stellar statement which simplifies this further by allowing score to
>> be a function, but thresholds are still useful to determine the text
>> content of the alert for example.
>> Simon
>>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
>>> Oh, I didn't know I had to set is_alert to True.
>>> Doesn't that mean that we have to add all rules twice? First to check whether is_alert needs to be set to True. Next to apply the actual scores?
>>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>>> the _score field is actually an elastic search matching score field,
>>>> and is not relevant to metron. You should see the scores in the
>>>> threat:triage:score field. However, your rules will only be run if the
>>>> telemetry has is_alert set true, so you should ensure that the
>>>> enrichment phase sets is_alert: true somewhere for alerts you want to
>>>> go to triage?
>>>> Simon
>>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>>>> I have the following configuration:
>>>>> "threatIntel": {
>>>>> "fieldMap": {},
>>>>>  "fieldToTypeMap": {},
>>>>>  "config": {},
>>>>>  "triageConfig": {
>>>>>    "riskLevelRules": [
>>>>>      {
>>>>>        "name": "Rule1",
>>>>>        "comment": "Checks whatever 1.",
>>>>>        "rule": "test == \"false\"",
>>>>>        "score": 20,
>>>>>        "reason": null
>>>>>      },
>>>>>      {
>>>>>        "name": "Rule1",
>>>>>        "comment": "Checks whatever 2.",
>>>>>        "rule": "test2 == \"False\"",
>>>>>        "score": 20,
>>>>>        "reason": null
>>>>>      },
>>>>>      {
>>>>>        "name": "Rule3",
>>>>>        "comment": "Checks whatever 2.",
>>>>>        "rule": "test3 == \"No\"",
>>>>>        "score": 20,
>>>>>        "reason": null
>>>>>      }
>>>>>    ],
>>>>>    "aggregator": "SUM",
>>>>>    "aggregationConfig": {}
>>>>>  }
>>>>> },
>>>>> I have no additional configuration in enrichment besides filling a specific with true or false based on a Stellar expression.
>>>>> I expected that when events would match my above rules, the _score field would be filled in. That does not seem to be the case.
>>>>> Does anyone know what I might be missing?


Re: Not seeing any Metron alerts.

Posted by Laurens Vets <la...@daemon.be>.
Thanks! Followup question, the below is_alert 'rules' in the snippet 
from 
http://apache.website-solution.net/metron/0.4.1/site-book/use-cases/geographic_login_outliers/index.html, 
are those an AND or OR?

   "threatIntel": {
     "fieldMap": {
       "stellar" : {
         "config" : [
           "geo_distance_distr:= STATS_MERGE( PROFILE_GET( 
'geo_distribution_from_centroid', 'global', PROFILE_FIXED( 2, 
'MINUTES')))",
           "dist_median := STATS_PERCENTILE(geo_distance_distr, 50.0)",
           "dist_sd := STATS_SD(geo_distance_distr)",
           "geo_outlier := ABS(dist_median - geo_distance) >= 5*dist_sd",
           "is_alert := exists(is_alert) && is_alert",
           "is_alert := is_alert || (geo_outlier != null && geo_outlier 
== true)",
           "geo_distance_distr := null"
         ]
       }
     },

For instance, can the 2nd is_alert line overwrite the value assigned in 
the first is_alert rule?

On 2017-09-25 11:13, Simon Elliston Ball wrote:
> Usually you would have the is_alert set based on more complex rules,
> and then potentially have different rules to determine the importance
> of the alert, so they do tend to serve different purposes.
> 
> For example a triage rule might be set on levels of an indicator after
> is_alert has been triggered by a simple presence of a non-zero result
> for that indicator, e.g. is it 2x std_devs, or 4x std_devs as
> different rule levels. We’re adding the ability to make score a
> stellar statement which simplifies this further by allowing score to
> be a function, but thresholds are still useful to determine the text
> content of the alert for example.
> 
> Simon
> 
>> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
>> 
>> Oh, I didn't know I had to set is_alert to True.
>> 
>> Doesn't that mean that we have to add all rules twice? First to check 
>> whether is_alert needs to be set to True. Next to apply the actual 
>> scores?
>> 
>> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>>> the _score field is actually an elastic search matching score field,
>>> and is not relevant to metron. You should see the scores in the
>>> threat:triage:score field. However, your rules will only be run if 
>>> the
>>> telemetry has is_alert set true, so you should ensure that the
>>> enrichment phase sets is_alert: true somewhere for alerts you want to
>>> go to triage?
>>> Simon
>>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>>> I have the following configuration:
>>>> "threatIntel": {
>>>> "fieldMap": {},
>>>>   "fieldToTypeMap": {},
>>>>   "config": {},
>>>>   "triageConfig": {
>>>>     "riskLevelRules": [
>>>>       {
>>>>         "name": "Rule1",
>>>>         "comment": "Checks whatever 1.",
>>>>         "rule": "test == \"false\"",
>>>>         "score": 20,
>>>>         "reason": null
>>>>       },
>>>>       {
>>>>         "name": "Rule1",
>>>>         "comment": "Checks whatever 2.",
>>>>         "rule": "test2 == \"False\"",
>>>>         "score": 20,
>>>>         "reason": null
>>>>       },
>>>>       {
>>>>         "name": "Rule3",
>>>>         "comment": "Checks whatever 2.",
>>>>         "rule": "test3 == \"No\"",
>>>>         "score": 20,
>>>>         "reason": null
>>>>       }
>>>>     ],
>>>>     "aggregator": "SUM",
>>>>     "aggregationConfig": {}
>>>>   }
>>>> },
>>>> I have no additional configuration in enrichment besides filling a 
>>>> specific with true or false based on a Stellar expression.
>>>> I expected that when events would match my above rules, the _score 
>>>> field would be filled in. That does not seem to be the case.
>>>> Does anyone know what I might be missing?

Re: Not seeing any Metron alerts.

Posted by Simon Elliston Ball <si...@simonellistonball.com>.
Usually you would have the is_alert set based on more complex rules, and then potentially have different rules to determine the importance of the alert, so they do tend to serve different purposes. 

For example a triage rule might be set on levels of an indicator after is_alert has been triggered by a simple presence of a non-zero result for that indicator, e.g. is it 2x std_devs, or 4x std_devs as different rule levels. We’re adding the ability to make score a stellar statement which simplifies this further by allowing score to be a function, but thresholds are still useful to determine the text content of the alert for example. 

Simon

> On 25 Sep 2017, at 19:09, Laurens Vets <la...@daemon.be> wrote:
> 
> Oh, I didn't know I had to set is_alert to True.
> 
> Doesn't that mean that we have to add all rules twice? First to check whether is_alert needs to be set to True. Next to apply the actual scores?
> 
> On 2017-09-25 11:00, Simon Elliston Ball wrote:
>> the _score field is actually an elastic search matching score field,
>> and is not relevant to metron. You should see the scores in the
>> threat:triage:score field. However, your rules will only be run if the
>> telemetry has is_alert set true, so you should ensure that the
>> enrichment phase sets is_alert: true somewhere for alerts you want to
>> go to triage?
>> Simon
>>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>>> I have the following configuration:
>>> "threatIntel": {
>>> "fieldMap": {},
>>>   "fieldToTypeMap": {},
>>>   "config": {},
>>>   "triageConfig": {
>>>     "riskLevelRules": [
>>>       {
>>>         "name": "Rule1",
>>>         "comment": "Checks whatever 1.",
>>>         "rule": "test == \"false\"",
>>>         "score": 20,
>>>         "reason": null
>>>       },
>>>       {
>>>         "name": "Rule1",
>>>         "comment": "Checks whatever 2.",
>>>         "rule": "test2 == \"False\"",
>>>         "score": 20,
>>>         "reason": null
>>>       },
>>>       {
>>>         "name": "Rule3",
>>>         "comment": "Checks whatever 2.",
>>>         "rule": "test3 == \"No\"",
>>>         "score": 20,
>>>         "reason": null
>>>       }
>>>     ],
>>>     "aggregator": "SUM",
>>>     "aggregationConfig": {}
>>>   }
>>> },
>>> I have no additional configuration in enrichment besides filling a specific with true or false based on a Stellar expression.
>>> I expected that when events would match my above rules, the _score field would be filled in. That does not seem to be the case.
>>> Does anyone know what I might be missing?


Re: Not seeing any Metron alerts.

Posted by Laurens Vets <la...@daemon.be>.
Oh, I didn't know I had to set is_alert to True.

Doesn't that mean that we have to add all rules twice? First to check 
whether is_alert needs to be set to True. Next to apply the actual 
scores?

On 2017-09-25 11:00, Simon Elliston Ball wrote:
> the _score field is actually an elastic search matching score field,
> and is not relevant to metron. You should see the scores in the
> threat:triage:score field. However, your rules will only be run if the
> telemetry has is_alert set true, so you should ensure that the
> enrichment phase sets is_alert: true somewhere for alerts you want to
> go to triage?
> 
> Simon
> 
>> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
>> 
>> I have the following configuration:
>> 
>> "threatIntel": {
>>  "fieldMap": {},
>>    "fieldToTypeMap": {},
>>    "config": {},
>>    "triageConfig": {
>>      "riskLevelRules": [
>>        {
>>          "name": "Rule1",
>>          "comment": "Checks whatever 1.",
>>          "rule": "test == \"false\"",
>>          "score": 20,
>>          "reason": null
>>        },
>>        {
>>          "name": "Rule1",
>>          "comment": "Checks whatever 2.",
>>          "rule": "test2 == \"False\"",
>>          "score": 20,
>>          "reason": null
>>        },
>>        {
>>          "name": "Rule3",
>>          "comment": "Checks whatever 2.",
>>          "rule": "test3 == \"No\"",
>>          "score": 20,
>>          "reason": null
>>        }
>>      ],
>>      "aggregator": "SUM",
>>      "aggregationConfig": {}
>>    }
>> },
>> 
>> I have no additional configuration in enrichment besides filling a 
>> specific with true or false based on a Stellar expression.
>> 
>> I expected that when events would match my above rules, the _score 
>> field would be filled in. That does not seem to be the case.
>> 
>> Does anyone know what I might be missing?

Re: Not seeing any Metron alerts.

Posted by Simon Elliston Ball <si...@simonellistonball.com>.
the _score field is actually an elastic search matching score field, and is not relevant to metron. You should see the scores in the threat:triage:score field. However, your rules will only be run if the telemetry has is_alert set true, so you should ensure that the enrichment phase sets is_alert: true somewhere for alerts you want to go to triage? 

Simon

> On 25 Sep 2017, at 18:46, Laurens Vets <la...@daemon.be> wrote:
> 
> I have the following configuration:
> 
> "threatIntel": {
>  "fieldMap": {},
>    "fieldToTypeMap": {},
>    "config": {},
>    "triageConfig": {
>      "riskLevelRules": [
>        {
>          "name": "Rule1",
>          "comment": "Checks whatever 1.",
>          "rule": "test == \"false\"",
>          "score": 20,
>          "reason": null
>        },
>        {
>          "name": "Rule1",
>          "comment": "Checks whatever 2.",
>          "rule": "test2 == \"False\"",
>          "score": 20,
>          "reason": null
>        },
>        {
>          "name": "Rule3",
>          "comment": "Checks whatever 2.",
>          "rule": "test3 == \"No\"",
>          "score": 20,
>          "reason": null
>        }
>      ],
>      "aggregator": "SUM",
>      "aggregationConfig": {}
>    }
> },
> 
> I have no additional configuration in enrichment besides filling a specific with true or false based on a Stellar expression.
> 
> I expected that when events would match my above rules, the _score field would be filled in. That does not seem to be the case.
> 
> Does anyone know what I might be missing?