You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Sanket Sharma <sa...@dukstra.com> on 2019/11/21 18:20:56 UTC
Switching alert status
Hi,
I was wondering if there is a way to switch the alert status on or off based on rule or condition? For instance, I have two different subnets where I’m monitoring network traffic. One is an open network and the other is a private/secured network.
I would like to set ‘is_alert’ to true if the src is subnet two and set it to false when its subnet one. The end goal is to only have certain alerts displayed in the alerts UI based on conditions. Is it possible?
Best regards,
Sanket
Re: Switching alert status
Posted by Sanket Sharma <sa...@dukstra.com>.
Here's the full work log in one place:
1. Enrichment configuration:
"enrichment": {
"fieldMap": {
"stellar": {
"config": {
"is_alert": "not(ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT , 'cyber:verified-accounts', 'd'))"
}
}
},
"fieldToTypeMap": {},
"config": {}
}
USER_ACCOUNT is a field in the incoming JSON message. We try and look it up in the enrichment table 'cyber:verified-accounts'. We only need to raise the alert if that user id is not present in the HBase table.
2. Stellar Tests (Have redacted the full account ids):
[Stellar]>>> USER_ACCOUNT := "xxxxxxxxxxx43254"
xxxxxxxxxxx43254
[Stellar]>>> is_alert := not(ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd'))
false
[Stellar]>>> USER_ACCOUNT := "xxxxxxxxxxx70293"
xxxxxxxxxxx70293
[Stellar]>>> is_alert := not(ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd'))
false
[Stellar]>>>
3. Incoming JSON message:
{
...........................,
...........................,
...........................,
"USER_ACCOUNT": "xxxxxxxxxxx43254",
...........................,
...........................,
...........................,
...........................,
...........................,
...........................,
...........................,
...........................,
...........................,
}
In the alerts UI is_alert is set to true for both of them and I can see other fields from the verified-accounts HBase table:
[cid:f331790b-8f00-4a53-a199-30a307accc9d]
So it seems that Stellar REPL is working as expected but the Enrichments from configuration are returning true for everything. I am trying to restart the topology and see if that makes a difference.
Thank you very much for your help and prompt responses.
Best regards,
Sanket
________________________________
From: Nick Allen <ni...@nickallen.org>
Sent: Monday, December 2, 2019 10:28 PM
To: Sanket Sharma <sa...@dukstra.com>
Cc: user@metron.apache.org <us...@metron.apache.org>
Subject: Re: Switching alert status
Quick thought... Should USER_ACCOUNT be a string; 'USER_ACCOUNT'? Otherwise it is expecting that there is a field in your message named USER_ACCOUNT. If there is no such field, the value of the variable USER_ACCOUNT will be null. Try that out in the REPL and see if it explains what you are seeing.
On Mon, Dec 2, 2019 at 3:24 PM Nick Allen <ni...@nickallen.org>> wrote:
> On the alert screen, for each record, I can see the fields from the enrichment as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax exception. Not sure what am I missing?
You should debug issues like this using the Stellar REPL. Run the Stellar REPL and try out the Stellar expressions that you mentioned. In most cases, it should be fairly easy to determine what is wrong. My original response included the steps to do this.
If for some reason it is not clear after you have tried these expressions in the REPL, respond back with what you are seeing in the REPL in contract to what you are seeing in the streaming topology and I can try to help further.
On Mon, Dec 2, 2019 at 3:11 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Thanks Nick. Your help is greatly appreciated. Based on the feedback and documentation, I was able to setup a streaming enrichment:
$METRON_HOME/config/zookeeper/parsers/VERIFIED_ACCOUNTS.json
{
"parserClassName" : "org.apache.metron.parsers.json.JSONMapParser",
"writerClassName" : "org.apache.metron.writer.hbase.SimpleHbaseEnrichmentWriter",
"sensorTopic":"verified-accounts",
"parserConfig":
{
"shew.table" : "cyber:verified-accounts",
"shew.cf<http://shew.cf>" : "d",
"shew.keyColumns" : "USER_ACCOUNT",
"shew.enrichmentType" : "VERIFIED_ACCOUNTS"
}
}
Then, I configured a stellar enrichment as below. The idea was that an alert should be triggered if the account does not exist in the enrichment (enrichment is basically a whitelist of user accounts):
$METRON_HOME/config/zookeeper/enrichment/VERIFIED_ACCOUNTS.json
{
"enrichment": {
"fieldMap": {
"stellar": {
"config": {
"is_alert": "ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') == false"
}
}
},
"fieldToTypeMap": {},
"config": {}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {},
"config": {},
"triageConfig": {
"riskLevelRules": [],
"aggregator": "MAX",
"aggregationConfig": {}
}
},
"configuration": {}
}
When I run the above sensors, it seems like ENRICHMENT_EXISTS actually joins the records on the given key instead of returning true or false. On the alert screen, for each record, I can see the fields from the enrichment as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax exception.
Not sure what am I missing?
You are spot on. That is exactly what we are looking for. I'll go ahead and open a JIRA to discuss it further.
Thanks once again for your detailed feedback and responses. Much appreciated.
Best regards,
Sanket
________________________________
From: Nick Allen <ni...@nickallen.org>>
Sent: Monday, December 2, 2019 9:09 PM
To: user@metron.apache.org<ma...@metron.apache.org> <us...@metron.apache.org>>
Subject: Re: Switching alert status
> I was able to create the alerts as suggested. My understanding is that stellar rules (and scores) configured in the config UI will only be evaluated if the is_alert flag is set to true. Is that correct?
Threat Triage is what gives you the "scores" that you are referring to. The purpose of Threat Triage is to assess a threat and output an overall threat score. That score can then be used to prioritize which threats need to be addressed first.
Threat Triage only runs on messages where there is a field named "is_alert" with a Stellar expression that evaluates to true. This allows you to avoid the expense of Threat Triage, in cases where you know it is not needed.
> Is there perhaps a way to toggle the is_alert flag based on the threat triage score ... or alternatively a way to calculate score without setting the is_alert flag to true?
The "is_alert" field is used as a flag to indicate which messages should undergo Threat Triage processing. Once a message completes Threat Triage and gets a score, changing or removing the "is_alert" field does not do anything.
> The use case is that we still want to score incoming records (based on stellar rules) but don’t want them displaying in the alerts UI unless they have a score... We could potentially filter the records from the alerts UI I suppose or clean up the Elastic index periodically, but wondering if something like this can be done out of the box?
If you just want to filter messages from the Alerts UI, then you can add a search filter to filter out messages that do not have a score or have a really low score. A user can filter and sort messages however they like within the Alerts UI, but this may not be exactly what you want.
If you want to not index messages (into Solr or Elasticsearch) unless the message has a score, then this is not directly possible out-of-the-box. Right now we index all messages into the search indices.
It sounds like you might want to only index a subset of your messages. If you are so inclined, feel free to open a JIRA to discuss that as a new feature.
Hope this helps
On Wed, Nov 27, 2019 at 6:48 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Excellent! Thank you so much for the assistance.
I was able to create the alerts as suggested. My understanding is that stellar rules (and scores) configured in the config UI will only be evaluated if the is_alert flag is set to true. Is that correct?
Is there perhaps a way to toggle the is_alert flag based on the threat triage score (or alternatively a way to calculate score without setting the is_alert flag to true?) The use case is that we still want to score incoming records (based on stellar rules) but don’t want them displaying in the alerts UI unless they have a score. We could potentially filter the records from the alerts UI I suppose or clean up the Elastic index periodically, but wondering if something like this can done out of the box?
Best regards,
Sanket
From: Nick Allen <ni...@nickallen.org>>
Reply to: "user@metron.apache.org<ma...@metron.apache.org>" <us...@metron.apache.org>>
Date: Thursday, 21 November 2019 at 20:45
To: "user@metron.apache.org<ma...@metron.apache.org>" <us...@metron.apache.org>>
Subject: Re: Switching alert status
Yes. You would create an enrichment using a Stellar expression that looks something like this.
is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
I can create, test and load that enrichment using the Stellar REPL.
[root@node1 0.7.2]# source /etc/default/metron
[root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
Let's test out the expression to make sure it does what we would expect. You can make the expression as complex as you need for your use case using Stellar<https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>.
[Stellar]>>> ip_src_addr := "192.168.0.22"
192.168.0.22
[Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
true
Need more help?
[Stellar]>>> ?IN_SUBNET
IN_SUBNET
Description: Returns true if an IP is within a subnet range.
Arguments:
ip - The IP address in string form
cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24<http://192.168.0.0/24>)
Returns: True if the IP address is within at least one of the network ranges and false if otherwise
Then I can create and save the enrichment configuration necessary to apply your Stellar expression.
[Stellar]>>> bro := SHELL_EDIT()
{
"enrichment" : {
"fieldMap": {
"stellar" : {
"config" : {
"is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24'<http://192.168.0.0/24'>)"
}
}
}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {}
}
}
[Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
Hope this helps. Good luck.
On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Hi,
I was wondering if there is a way to switch the alert status on or off based on rule or condition? For instance, I have two different subnets where I’m monitoring network traffic. One is an open network and the other is a private/secured network.
I would like to set ‘is_alert’ to true if the src is subnet two and set it to false when its subnet one. The end goal is to only have certain alerts displayed in the alerts UI based on conditions. Is it possible?
Best regards,
Sanket
Re: Switching alert status
Posted by Nick Allen <ni...@nickallen.org>.
Quick thought... Should USER_ACCOUNT be a string; 'USER_ACCOUNT'?
Otherwise it is expecting that there is a field in your message named
USER_ACCOUNT. If there is no such field, the value of the variable
USER_ACCOUNT will be null. Try that out in the REPL and see if it explains
what you are seeing.
On Mon, Dec 2, 2019 at 3:24 PM Nick Allen <ni...@nickallen.org> wrote:
> > On the alert screen, for each record, I can see the fields from the
> enrichment as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS',
> USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax
> exception. Not sure what am I missing?
>
> You should debug issues like this using the Stellar REPL. Run the Stellar
> REPL and try out the Stellar expressions that you mentioned. In most
> cases, it should be fairly easy to determine what is wrong. My original
> response included the steps to do this.
>
> If for some reason it is not clear after you have tried these expressions
> in the REPL, respond back with what you are seeing in the REPL in contract
> to what you are seeing in the streaming topology and I can try to help
> further.
>
>
>
>
>
> On Mon, Dec 2, 2019 at 3:11 PM Sanket Sharma <sa...@dukstra.com>
> wrote:
>
>> Thanks Nick. Your help is greatly appreciated. Based on the feedback and
>> documentation, I was able to setup a streaming enrichment:
>> $METRON_HOME/config/zookeeper/parsers/VERIFIED_ACCOUNTS.json
>> {
>> "parserClassName" : "org.apache.metron.parsers.json.JSONMapParser",
>> "writerClassName" :
>> "org.apache.metron.writer.hbase.SimpleHbaseEnrichmentWriter",
>> "sensorTopic":"verified-accounts",
>> "parserConfig":
>> {
>> "shew.table" : "cyber:verified-accounts",
>> "shew.cf" : "d",
>> "shew.keyColumns" : "USER_ACCOUNT",
>> "shew.enrichmentType" : "VERIFIED_ACCOUNTS"
>> }
>> }
>> Then, I configured a stellar enrichment as below. The idea was that an
>> alert should be triggered if the account does not exist in the enrichment
>> (enrichment is basically a whitelist of user accounts):
>> $METRON_HOME/config/zookeeper/enrichment/VERIFIED_ACCOUNTS.json
>> {
>> "enrichment": {
>> "fieldMap": {
>> "stellar": {
>> "config": {
>> "is_alert": "ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS',
>> USER_ACCOUNT, 'cyber:verified-accounts', 'd') == false"
>> }
>> }
>> },
>> "fieldToTypeMap": {},
>> "config": {}
>> },
>> "threatIntel": {
>> "fieldMap": {},
>> "fieldToTypeMap": {},
>> "config": {},
>> "triageConfig": {
>> "riskLevelRules": [],
>> "aggregator": "MAX",
>> "aggregationConfig": {}
>> }
>> },
>> "configuration": {}
>> }
>>
>> When I run the above sensors, it seems like ENRICHMENT_EXISTS actually
>> joins the records on the given key instead of returning true or false. On
>> the alert screen, for each record, I can see the fields from the enrichment
>> as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS',
>> USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax
>> exception.
>> Not sure what am I missing?
>>
>> You are spot on. That is exactly what we are looking for. I'll go ahead
>> and open a JIRA to discuss it further.
>>
>> Thanks once again for your detailed feedback and responses. Much
>> appreciated.
>>
>> Best regards,
>> Sanket
>> ------------------------------
>> *From:* Nick Allen <ni...@nickallen.org>
>> *Sent:* Monday, December 2, 2019 9:09 PM
>> *To:* user@metron.apache.org <us...@metron.apache.org>
>> *Subject:* Re: Switching alert status
>>
>> > I was able to create the alerts as suggested. My understanding is that
>> stellar rules (and scores) configured in the config UI will only be
>> evaluated if the is_alert flag is set to true. Is that correct?
>>
>> Threat Triage is what gives you the "scores" that you are referring to.
>> The purpose of Threat Triage is to assess a threat and output an overall
>> threat score. That score can then be used to prioritize which threats need
>> to be addressed first.
>>
>> Threat Triage only runs on messages where there is a field named
>> "is_alert" with a Stellar expression that evaluates to true. This allows
>> you to avoid the expense of Threat Triage, in cases where you know it is
>> not needed.
>>
>>
>> > Is there perhaps a way to toggle the is_alert flag based on the threat
>> triage score ... or alternatively a way to calculate score without
>> setting the is_alert flag to true?
>>
>> The "is_alert" field is used as a flag to indicate which messages should
>> undergo Threat Triage processing. Once a message completes Threat Triage
>> and gets a score, changing or removing the "is_alert" field does not do
>> anything.
>>
>>
>> > The use case is that we still want to score incoming records (based on
>> stellar rules) but don’t want them displaying in the alerts UI unless they
>> have a score... We could potentially filter the records from the alerts
>> UI I suppose or clean up the Elastic index periodically, but wondering if
>> something like this can be done out of the box?
>>
>> If you just want to filter messages from the Alerts UI, then you can add
>> a search filter to filter out messages that do not have a score or have a
>> really low score. A user can filter and sort messages however they like
>> within the Alerts UI, but this may not be exactly what you want.
>>
>> If you want to *not* index messages (into Solr or Elasticsearch) unless
>> the message has a score, then this is not directly possible
>> out-of-the-box. Right now we index all messages into the search indices.
>>
>> It sounds like you might want to only index a subset of your messages.
>> If you are so inclined, feel free to open a JIRA to discuss that as a new
>> feature.
>>
>>
>> Hope this helps
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Nov 27, 2019 at 6:48 PM Sanket Sharma <sa...@dukstra.com>
>> wrote:
>>
>> Excellent! Thank you so much for the assistance.
>>
>>
>>
>> I was able to create the alerts as suggested. My understanding is that
>> stellar rules (and scores) configured in the config UI will only be
>> evaluated if the is_alert flag is set to true. Is that correct?
>>
>>
>>
>> Is there perhaps a way to toggle the is_alert flag based on the threat
>> triage score (or alternatively a way to calculate score without setting the
>> is_alert flag to true?) The use case is that we still want to score
>> incoming records (based on stellar rules) but don’t want them displaying in
>> the alerts UI unless they have a score. We could potentially filter the
>> records from the alerts UI I suppose or clean up the Elastic index
>> periodically, but wondering if something like this can done out of the box?
>>
>>
>>
>> Best regards,
>>
>> Sanket
>>
>>
>>
>> *From: *Nick Allen <ni...@nickallen.org>
>> *Reply to: *"user@metron.apache.org" <us...@metron.apache.org>
>> *Date: *Thursday, 21 November 2019 at 20:45
>> *To: *"user@metron.apache.org" <us...@metron.apache.org>
>> *Subject: *Re: Switching alert status
>>
>>
>>
>> Yes. You would create an enrichment using a Stellar expression that
>> looks something like this.
>>
>>
>>
>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
>>
>>
>>
>> I can create, test and load that enrichment using the Stellar REPL.
>>
>>
>>
>> [root@node1 0.7.2]# source /etc/default/metron
>>
>> [root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
>>
>>
>>
>> Let's test out the expression to make sure it does what we would expect.
>> You can make the expression as complex as you need for your use case using
>> Stellar
>> <https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>
>> .
>>
>>
>>
>> [Stellar]>>> ip_src_addr := "192.168.0.22"
>> 192.168.0.22
>> [Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
>> true
>>
>> Need more help?
>>
>>
>>
>> [Stellar]>>> ?IN_SUBNET
>> IN_SUBNET
>> Description: Returns true if an IP is within a subnet range.
>>
>> Arguments:
>> ip - The IP address in string form
>> cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24)
>>
>> Returns: True if the IP address is within at least one of the network ranges and false if otherwise
>>
>>
>>
>> Then I can create and save the enrichment configuration necessary to
>> apply your Stellar expression.
>>
>>
>>
>> [Stellar]>>> bro := SHELL_EDIT()
>>
>> {
>>
>> "enrichment" : {
>>
>> "fieldMap": {
>>
>> "stellar" : {
>>
>> "config" : {
>>
>> "is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24')"
>>
>> }
>>
>> }
>>
>> }
>>
>> },
>>
>> "threatIntel": {
>>
>> "fieldMap": {},
>>
>> "fieldToTypeMap": {}
>>
>> }
>>
>> }
>>
>> [Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
>>
>>
>>
>> Hope this helps. Good luck.
>>
>>
>>
>>
>>
>> On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>
>> wrote:
>>
>> Hi,
>>
>>
>>
>> I was wondering if there is a way to switch the alert status on or off
>> based on rule or condition? For instance, I have two different subnets
>> where I’m monitoring network traffic. One is an open network and the other
>> is a private/secured network.
>>
>>
>>
>> I would like to set ‘is_alert’ to true if the src is subnet two and set
>> it to false when its subnet one. The end goal is to only have certain
>> alerts displayed in the alerts UI based on conditions. Is it possible?
>>
>>
>>
>>
>>
>> Best regards,
>>
>> Sanket
>>
>>
Re: Switching alert status
Posted by Sanket Sharma <sa...@dukstra.com>.
Thanks. Busy trying that now. Will update shortly.
________________________________
From: Nick Allen <ni...@nickallen.org>
Sent: Monday, December 2, 2019 10:24 PM
To: Sanket Sharma <sa...@dukstra.com>
Cc: user@metron.apache.org <us...@metron.apache.org>
Subject: Re: Switching alert status
> On the alert screen, for each record, I can see the fields from the enrichment as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax exception. Not sure what am I missing?
You should debug issues like this using the Stellar REPL. Run the Stellar REPL and try out the Stellar expressions that you mentioned. In most cases, it should be fairly easy to determine what is wrong. My original response included the steps to do this.
If for some reason it is not clear after you have tried these expressions in the REPL, respond back with what you are seeing in the REPL in contract to what you are seeing in the streaming topology and I can try to help further.
On Mon, Dec 2, 2019 at 3:11 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Thanks Nick. Your help is greatly appreciated. Based on the feedback and documentation, I was able to setup a streaming enrichment:
$METRON_HOME/config/zookeeper/parsers/VERIFIED_ACCOUNTS.json
{
"parserClassName" : "org.apache.metron.parsers.json.JSONMapParser",
"writerClassName" : "org.apache.metron.writer.hbase.SimpleHbaseEnrichmentWriter",
"sensorTopic":"verified-accounts",
"parserConfig":
{
"shew.table" : "cyber:verified-accounts",
"shew.cf<http://shew.cf>" : "d",
"shew.keyColumns" : "USER_ACCOUNT",
"shew.enrichmentType" : "VERIFIED_ACCOUNTS"
}
}
Then, I configured a stellar enrichment as below. The idea was that an alert should be triggered if the account does not exist in the enrichment (enrichment is basically a whitelist of user accounts):
$METRON_HOME/config/zookeeper/enrichment/VERIFIED_ACCOUNTS.json
{
"enrichment": {
"fieldMap": {
"stellar": {
"config": {
"is_alert": "ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') == false"
}
}
},
"fieldToTypeMap": {},
"config": {}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {},
"config": {},
"triageConfig": {
"riskLevelRules": [],
"aggregator": "MAX",
"aggregationConfig": {}
}
},
"configuration": {}
}
When I run the above sensors, it seems like ENRICHMENT_EXISTS actually joins the records on the given key instead of returning true or false. On the alert screen, for each record, I can see the fields from the enrichment as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax exception.
Not sure what am I missing?
You are spot on. That is exactly what we are looking for. I'll go ahead and open a JIRA to discuss it further.
Thanks once again for your detailed feedback and responses. Much appreciated.
Best regards,
Sanket
________________________________
From: Nick Allen <ni...@nickallen.org>>
Sent: Monday, December 2, 2019 9:09 PM
To: user@metron.apache.org<ma...@metron.apache.org> <us...@metron.apache.org>>
Subject: Re: Switching alert status
> I was able to create the alerts as suggested. My understanding is that stellar rules (and scores) configured in the config UI will only be evaluated if the is_alert flag is set to true. Is that correct?
Threat Triage is what gives you the "scores" that you are referring to. The purpose of Threat Triage is to assess a threat and output an overall threat score. That score can then be used to prioritize which threats need to be addressed first.
Threat Triage only runs on messages where there is a field named "is_alert" with a Stellar expression that evaluates to true. This allows you to avoid the expense of Threat Triage, in cases where you know it is not needed.
> Is there perhaps a way to toggle the is_alert flag based on the threat triage score ... or alternatively a way to calculate score without setting the is_alert flag to true?
The "is_alert" field is used as a flag to indicate which messages should undergo Threat Triage processing. Once a message completes Threat Triage and gets a score, changing or removing the "is_alert" field does not do anything.
> The use case is that we still want to score incoming records (based on stellar rules) but don’t want them displaying in the alerts UI unless they have a score... We could potentially filter the records from the alerts UI I suppose or clean up the Elastic index periodically, but wondering if something like this can be done out of the box?
If you just want to filter messages from the Alerts UI, then you can add a search filter to filter out messages that do not have a score or have a really low score. A user can filter and sort messages however they like within the Alerts UI, but this may not be exactly what you want.
If you want to not index messages (into Solr or Elasticsearch) unless the message has a score, then this is not directly possible out-of-the-box. Right now we index all messages into the search indices.
It sounds like you might want to only index a subset of your messages. If you are so inclined, feel free to open a JIRA to discuss that as a new feature.
Hope this helps
On Wed, Nov 27, 2019 at 6:48 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Excellent! Thank you so much for the assistance.
I was able to create the alerts as suggested. My understanding is that stellar rules (and scores) configured in the config UI will only be evaluated if the is_alert flag is set to true. Is that correct?
Is there perhaps a way to toggle the is_alert flag based on the threat triage score (or alternatively a way to calculate score without setting the is_alert flag to true?) The use case is that we still want to score incoming records (based on stellar rules) but don’t want them displaying in the alerts UI unless they have a score. We could potentially filter the records from the alerts UI I suppose or clean up the Elastic index periodically, but wondering if something like this can done out of the box?
Best regards,
Sanket
From: Nick Allen <ni...@nickallen.org>>
Reply to: "user@metron.apache.org<ma...@metron.apache.org>" <us...@metron.apache.org>>
Date: Thursday, 21 November 2019 at 20:45
To: "user@metron.apache.org<ma...@metron.apache.org>" <us...@metron.apache.org>>
Subject: Re: Switching alert status
Yes. You would create an enrichment using a Stellar expression that looks something like this.
is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
I can create, test and load that enrichment using the Stellar REPL.
[root@node1 0.7.2]# source /etc/default/metron
[root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
Let's test out the expression to make sure it does what we would expect. You can make the expression as complex as you need for your use case using Stellar<https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>.
[Stellar]>>> ip_src_addr := "192.168.0.22"
192.168.0.22
[Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
true
Need more help?
[Stellar]>>> ?IN_SUBNET
IN_SUBNET
Description: Returns true if an IP is within a subnet range.
Arguments:
ip - The IP address in string form
cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24<http://192.168.0.0/24>)
Returns: True if the IP address is within at least one of the network ranges and false if otherwise
Then I can create and save the enrichment configuration necessary to apply your Stellar expression.
[Stellar]>>> bro := SHELL_EDIT()
{
"enrichment" : {
"fieldMap": {
"stellar" : {
"config" : {
"is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24'<http://192.168.0.0/24'>)"
}
}
}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {}
}
}
[Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
Hope this helps. Good luck.
On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Hi,
I was wondering if there is a way to switch the alert status on or off based on rule or condition? For instance, I have two different subnets where I’m monitoring network traffic. One is an open network and the other is a private/secured network.
I would like to set ‘is_alert’ to true if the src is subnet two and set it to false when its subnet one. The end goal is to only have certain alerts displayed in the alerts UI based on conditions. Is it possible?
Best regards,
Sanket
Re: Switching alert status
Posted by Nick Allen <ni...@nickallen.org>.
> On the alert screen, for each record, I can see the fields from the
enrichment as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS',
USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax
exception. Not sure what am I missing?
You should debug issues like this using the Stellar REPL. Run the Stellar
REPL and try out the Stellar expressions that you mentioned. In most
cases, it should be fairly easy to determine what is wrong. My original
response included the steps to do this.
If for some reason it is not clear after you have tried these expressions
in the REPL, respond back with what you are seeing in the REPL in contract
to what you are seeing in the streaming topology and I can try to help
further.
On Mon, Dec 2, 2019 at 3:11 PM Sanket Sharma <sa...@dukstra.com>
wrote:
> Thanks Nick. Your help is greatly appreciated. Based on the feedback and
> documentation, I was able to setup a streaming enrichment:
> $METRON_HOME/config/zookeeper/parsers/VERIFIED_ACCOUNTS.json
> {
> "parserClassName" : "org.apache.metron.parsers.json.JSONMapParser",
> "writerClassName" :
> "org.apache.metron.writer.hbase.SimpleHbaseEnrichmentWriter",
> "sensorTopic":"verified-accounts",
> "parserConfig":
> {
> "shew.table" : "cyber:verified-accounts",
> "shew.cf" : "d",
> "shew.keyColumns" : "USER_ACCOUNT",
> "shew.enrichmentType" : "VERIFIED_ACCOUNTS"
> }
> }
> Then, I configured a stellar enrichment as below. The idea was that an
> alert should be triggered if the account does not exist in the enrichment
> (enrichment is basically a whitelist of user accounts):
> $METRON_HOME/config/zookeeper/enrichment/VERIFIED_ACCOUNTS.json
> {
> "enrichment": {
> "fieldMap": {
> "stellar": {
> "config": {
> "is_alert": "ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS',
> USER_ACCOUNT, 'cyber:verified-accounts', 'd') == false"
> }
> }
> },
> "fieldToTypeMap": {},
> "config": {}
> },
> "threatIntel": {
> "fieldMap": {},
> "fieldToTypeMap": {},
> "config": {},
> "triageConfig": {
> "riskLevelRules": [],
> "aggregator": "MAX",
> "aggregationConfig": {}
> }
> },
> "configuration": {}
> }
>
> When I run the above sensors, it seems like ENRICHMENT_EXISTS actually
> joins the records on the given key instead of returning true or false. On
> the alert screen, for each record, I can see the fields from the enrichment
> as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS',
> USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax
> exception.
> Not sure what am I missing?
>
> You are spot on. That is exactly what we are looking for. I'll go ahead
> and open a JIRA to discuss it further.
>
> Thanks once again for your detailed feedback and responses. Much
> appreciated.
>
> Best regards,
> Sanket
> ------------------------------
> *From:* Nick Allen <ni...@nickallen.org>
> *Sent:* Monday, December 2, 2019 9:09 PM
> *To:* user@metron.apache.org <us...@metron.apache.org>
> *Subject:* Re: Switching alert status
>
> > I was able to create the alerts as suggested. My understanding is that
> stellar rules (and scores) configured in the config UI will only be
> evaluated if the is_alert flag is set to true. Is that correct?
>
> Threat Triage is what gives you the "scores" that you are referring to.
> The purpose of Threat Triage is to assess a threat and output an overall
> threat score. That score can then be used to prioritize which threats need
> to be addressed first.
>
> Threat Triage only runs on messages where there is a field named
> "is_alert" with a Stellar expression that evaluates to true. This allows
> you to avoid the expense of Threat Triage, in cases where you know it is
> not needed.
>
>
> > Is there perhaps a way to toggle the is_alert flag based on the threat
> triage score ... or alternatively a way to calculate score without
> setting the is_alert flag to true?
>
> The "is_alert" field is used as a flag to indicate which messages should
> undergo Threat Triage processing. Once a message completes Threat Triage
> and gets a score, changing or removing the "is_alert" field does not do
> anything.
>
>
> > The use case is that we still want to score incoming records (based on
> stellar rules) but don’t want them displaying in the alerts UI unless they
> have a score... We could potentially filter the records from the alerts
> UI I suppose or clean up the Elastic index periodically, but wondering if
> something like this can be done out of the box?
>
> If you just want to filter messages from the Alerts UI, then you can add
> a search filter to filter out messages that do not have a score or have a
> really low score. A user can filter and sort messages however they like
> within the Alerts UI, but this may not be exactly what you want.
>
> If you want to *not* index messages (into Solr or Elasticsearch) unless
> the message has a score, then this is not directly possible
> out-of-the-box. Right now we index all messages into the search indices.
>
> It sounds like you might want to only index a subset of your messages. If
> you are so inclined, feel free to open a JIRA to discuss that as a new
> feature.
>
>
> Hope this helps
>
>
>
>
>
>
>
>
> On Wed, Nov 27, 2019 at 6:48 PM Sanket Sharma <sa...@dukstra.com>
> wrote:
>
> Excellent! Thank you so much for the assistance.
>
>
>
> I was able to create the alerts as suggested. My understanding is that
> stellar rules (and scores) configured in the config UI will only be
> evaluated if the is_alert flag is set to true. Is that correct?
>
>
>
> Is there perhaps a way to toggle the is_alert flag based on the threat
> triage score (or alternatively a way to calculate score without setting the
> is_alert flag to true?) The use case is that we still want to score
> incoming records (based on stellar rules) but don’t want them displaying in
> the alerts UI unless they have a score. We could potentially filter the
> records from the alerts UI I suppose or clean up the Elastic index
> periodically, but wondering if something like this can done out of the box?
>
>
>
> Best regards,
>
> Sanket
>
>
>
> *From: *Nick Allen <ni...@nickallen.org>
> *Reply to: *"user@metron.apache.org" <us...@metron.apache.org>
> *Date: *Thursday, 21 November 2019 at 20:45
> *To: *"user@metron.apache.org" <us...@metron.apache.org>
> *Subject: *Re: Switching alert status
>
>
>
> Yes. You would create an enrichment using a Stellar expression that looks
> something like this.
>
>
>
> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
>
>
>
> I can create, test and load that enrichment using the Stellar REPL.
>
>
>
> [root@node1 0.7.2]# source /etc/default/metron
>
> [root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
>
>
>
> Let's test out the expression to make sure it does what we would expect.
> You can make the expression as complex as you need for your use case using
> Stellar
> <https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>
> .
>
>
>
> [Stellar]>>> ip_src_addr := "192.168.0.22"
> 192.168.0.22
> [Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
> true
>
> Need more help?
>
>
>
> [Stellar]>>> ?IN_SUBNET
> IN_SUBNET
> Description: Returns true if an IP is within a subnet range.
>
> Arguments:
> ip - The IP address in string form
> cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24)
>
> Returns: True if the IP address is within at least one of the network ranges and false if otherwise
>
>
>
> Then I can create and save the enrichment configuration necessary to apply
> your Stellar expression.
>
>
>
> [Stellar]>>> bro := SHELL_EDIT()
>
> {
>
> "enrichment" : {
>
> "fieldMap": {
>
> "stellar" : {
>
> "config" : {
>
> "is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24')"
>
> }
>
> }
>
> }
>
> },
>
> "threatIntel": {
>
> "fieldMap": {},
>
> "fieldToTypeMap": {}
>
> }
>
> }
>
> [Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
>
>
>
> Hope this helps. Good luck.
>
>
>
>
>
> On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>
> wrote:
>
> Hi,
>
>
>
> I was wondering if there is a way to switch the alert status on or off
> based on rule or condition? For instance, I have two different subnets
> where I’m monitoring network traffic. One is an open network and the other
> is a private/secured network.
>
>
>
> I would like to set ‘is_alert’ to true if the src is subnet two and set it
> to false when its subnet one. The end goal is to only have certain alerts
> displayed in the alerts UI based on conditions. Is it possible?
>
>
>
>
>
> Best regards,
>
> Sanket
>
>
Re: Switching alert status
Posted by Sanket Sharma <sa...@dukstra.com>.
Thanks Nick. Your help is greatly appreciated. Based on the feedback and documentation, I was able to setup a streaming enrichment:
$METRON_HOME/config/zookeeper/parsers/VERIFIED_ACCOUNTS.json
{
"parserClassName" : "org.apache.metron.parsers.json.JSONMapParser",
"writerClassName" : "org.apache.metron.writer.hbase.SimpleHbaseEnrichmentWriter",
"sensorTopic":"verified-accounts",
"parserConfig":
{
"shew.table" : "cyber:verified-accounts",
"shew.cf" : "d",
"shew.keyColumns" : "USER_ACCOUNT",
"shew.enrichmentType" : "VERIFIED_ACCOUNTS"
}
}
Then, I configured a stellar enrichment as below. The idea was that an alert should be triggered if the account does not exist in the enrichment (enrichment is basically a whitelist of user accounts):
$METRON_HOME/config/zookeeper/enrichment/VERIFIED_ACCOUNTS.json
{
"enrichment": {
"fieldMap": {
"stellar": {
"config": {
"is_alert": "ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') == false"
}
}
},
"fieldToTypeMap": {},
"config": {}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {},
"config": {},
"triageConfig": {
"riskLevelRules": [],
"aggregator": "MAX",
"aggregationConfig": {}
}
},
"configuration": {}
}
When I run the above sensors, it seems like ENRICHMENT_EXISTS actually joins the records on the given key instead of returning true or false. On the alert screen, for each record, I can see the fields from the enrichment as well. I've tried "not ENRICHMENT_EXISTS('VERIFIED_ACCOUNTS', USER_ACCOUNT, 'cyber:verified-accounts', 'd') " but that throws a syntax exception.
Not sure what am I missing?
You are spot on. That is exactly what we are looking for. I'll go ahead and open a JIRA to discuss it further.
Thanks once again for your detailed feedback and responses. Much appreciated.
Best regards,
Sanket
________________________________
From: Nick Allen <ni...@nickallen.org>
Sent: Monday, December 2, 2019 9:09 PM
To: user@metron.apache.org <us...@metron.apache.org>
Subject: Re: Switching alert status
> I was able to create the alerts as suggested. My understanding is that stellar rules (and scores) configured in the config UI will only be evaluated if the is_alert flag is set to true. Is that correct?
Threat Triage is what gives you the "scores" that you are referring to. The purpose of Threat Triage is to assess a threat and output an overall threat score. That score can then be used to prioritize which threats need to be addressed first.
Threat Triage only runs on messages where there is a field named "is_alert" with a Stellar expression that evaluates to true. This allows you to avoid the expense of Threat Triage, in cases where you know it is not needed.
> Is there perhaps a way to toggle the is_alert flag based on the threat triage score ... or alternatively a way to calculate score without setting the is_alert flag to true?
The "is_alert" field is used as a flag to indicate which messages should undergo Threat Triage processing. Once a message completes Threat Triage and gets a score, changing or removing the "is_alert" field does not do anything.
> The use case is that we still want to score incoming records (based on stellar rules) but don’t want them displaying in the alerts UI unless they have a score... We could potentially filter the records from the alerts UI I suppose or clean up the Elastic index periodically, but wondering if something like this can be done out of the box?
If you just want to filter messages from the Alerts UI, then you can add a search filter to filter out messages that do not have a score or have a really low score. A user can filter and sort messages however they like within the Alerts UI, but this may not be exactly what you want.
If you want to not index messages (into Solr or Elasticsearch) unless the message has a score, then this is not directly possible out-of-the-box. Right now we index all messages into the search indices.
It sounds like you might want to only index a subset of your messages. If you are so inclined, feel free to open a JIRA to discuss that as a new feature.
Hope this helps
On Wed, Nov 27, 2019 at 6:48 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Excellent! Thank you so much for the assistance.
I was able to create the alerts as suggested. My understanding is that stellar rules (and scores) configured in the config UI will only be evaluated if the is_alert flag is set to true. Is that correct?
Is there perhaps a way to toggle the is_alert flag based on the threat triage score (or alternatively a way to calculate score without setting the is_alert flag to true?) The use case is that we still want to score incoming records (based on stellar rules) but don’t want them displaying in the alerts UI unless they have a score. We could potentially filter the records from the alerts UI I suppose or clean up the Elastic index periodically, but wondering if something like this can done out of the box?
Best regards,
Sanket
From: Nick Allen <ni...@nickallen.org>>
Reply to: "user@metron.apache.org<ma...@metron.apache.org>" <us...@metron.apache.org>>
Date: Thursday, 21 November 2019 at 20:45
To: "user@metron.apache.org<ma...@metron.apache.org>" <us...@metron.apache.org>>
Subject: Re: Switching alert status
Yes. You would create an enrichment using a Stellar expression that looks something like this.
is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
I can create, test and load that enrichment using the Stellar REPL.
[root@node1 0.7.2]# source /etc/default/metron
[root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
Let's test out the expression to make sure it does what we would expect. You can make the expression as complex as you need for your use case using Stellar<https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>.
[Stellar]>>> ip_src_addr := "192.168.0.22"
192.168.0.22
[Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
true
Need more help?
[Stellar]>>> ?IN_SUBNET
IN_SUBNET
Description: Returns true if an IP is within a subnet range.
Arguments:
ip - The IP address in string form
cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24<http://192.168.0.0/24>)
Returns: True if the IP address is within at least one of the network ranges and false if otherwise
Then I can create and save the enrichment configuration necessary to apply your Stellar expression.
[Stellar]>>> bro := SHELL_EDIT()
{
"enrichment" : {
"fieldMap": {
"stellar" : {
"config" : {
"is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24'<http://192.168.0.0/24'>)"
}
}
}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {}
}
}
[Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
Hope this helps. Good luck.
On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Hi,
I was wondering if there is a way to switch the alert status on or off based on rule or condition? For instance, I have two different subnets where I’m monitoring network traffic. One is an open network and the other is a private/secured network.
I would like to set ‘is_alert’ to true if the src is subnet two and set it to false when its subnet one. The end goal is to only have certain alerts displayed in the alerts UI based on conditions. Is it possible?
Best regards,
Sanket
Re: Switching alert status
Posted by Nick Allen <ni...@nickallen.org>.
> I was able to create the alerts as suggested. My understanding is that
stellar rules (and scores) configured in the config UI will only be
evaluated if the is_alert flag is set to true. Is that correct?
Threat Triage is what gives you the "scores" that you are referring to.
The purpose of Threat Triage is to assess a threat and output an overall
threat score. That score can then be used to prioritize which threats need
to be addressed first.
Threat Triage only runs on messages where there is a field named "is_alert"
with a Stellar expression that evaluates to true. This allows you to avoid
the expense of Threat Triage, in cases where you know it is not needed.
> Is there perhaps a way to toggle the is_alert flag based on the threat
triage score ... or alternatively a way to calculate score without setting
the is_alert flag to true?
The "is_alert" field is used as a flag to indicate which messages should
undergo Threat Triage processing. Once a message completes Threat Triage
and gets a score, changing or removing the "is_alert" field does not do
anything.
> The use case is that we still want to score incoming records (based on
stellar rules) but don’t want them displaying in the alerts UI unless they
have a score... We could potentially filter the records from the alerts UI
I suppose or clean up the Elastic index periodically, but wondering if
something like this can be done out of the box?
If you just want to filter messages from the Alerts UI, then you can add a
search filter to filter out messages that do not have a score or have a
really low score. A user can filter and sort messages however they like
within the Alerts UI, but this may not be exactly what you want.
If you want to *not* index messages (into Solr or Elasticsearch) unless the
message has a score, then this is not directly possible out-of-the-box.
Right now we index all messages into the search indices.
It sounds like you might want to only index a subset of your messages. If
you are so inclined, feel free to open a JIRA to discuss that as a new
feature.
Hope this helps
On Wed, Nov 27, 2019 at 6:48 PM Sanket Sharma <sa...@dukstra.com>
wrote:
> Excellent! Thank you so much for the assistance.
>
>
>
> I was able to create the alerts as suggested. My understanding is that
> stellar rules (and scores) configured in the config UI will only be
> evaluated if the is_alert flag is set to true. Is that correct?
>
>
>
> Is there perhaps a way to toggle the is_alert flag based on the threat
> triage score (or alternatively a way to calculate score without setting the
> is_alert flag to true?) The use case is that we still want to score
> incoming records (based on stellar rules) but don’t want them displaying in
> the alerts UI unless they have a score. We could potentially filter the
> records from the alerts UI I suppose or clean up the Elastic index
> periodically, but wondering if something like this can done out of the box?
>
>
>
> Best regards,
>
> Sanket
>
>
>
> *From: *Nick Allen <ni...@nickallen.org>
> *Reply to: *"user@metron.apache.org" <us...@metron.apache.org>
> *Date: *Thursday, 21 November 2019 at 20:45
> *To: *"user@metron.apache.org" <us...@metron.apache.org>
> *Subject: *Re: Switching alert status
>
>
>
> Yes. You would create an enrichment using a Stellar expression that looks
> something like this.
>
>
>
> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
>
>
>
> I can create, test and load that enrichment using the Stellar REPL.
>
>
>
> [root@node1 0.7.2]# source /etc/default/metron
>
> [root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
>
>
>
> Let's test out the expression to make sure it does what we would expect.
> You can make the expression as complex as you need for your use case using
> Stellar
> <https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>
> .
>
>
>
> [Stellar]>>> ip_src_addr := "192.168.0.22"
> 192.168.0.22
> [Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
> true
>
> Need more help?
>
>
>
> [Stellar]>>> ?IN_SUBNET
> IN_SUBNET
> Description: Returns true if an IP is within a subnet range.
>
> Arguments:
> ip - The IP address in string form
> cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24)
>
> Returns: True if the IP address is within at least one of the network ranges and false if otherwise
>
>
>
> Then I can create and save the enrichment configuration necessary to apply
> your Stellar expression.
>
>
>
> [Stellar]>>> bro := SHELL_EDIT()
>
> {
>
> "enrichment" : {
>
> "fieldMap": {
>
> "stellar" : {
>
> "config" : {
>
> "is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24')"
>
> }
>
> }
>
> }
>
> },
>
> "threatIntel": {
>
> "fieldMap": {},
>
> "fieldToTypeMap": {}
>
> }
>
> }
>
> [Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
>
>
>
> Hope this helps. Good luck.
>
>
>
>
>
> On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>
> wrote:
>
> Hi,
>
>
>
> I was wondering if there is a way to switch the alert status on or off
> based on rule or condition? For instance, I have two different subnets
> where I’m monitoring network traffic. One is an open network and the other
> is a private/secured network.
>
>
>
> I would like to set ‘is_alert’ to true if the src is subnet two and set it
> to false when its subnet one. The end goal is to only have certain alerts
> displayed in the alerts UI based on conditions. Is it possible?
>
>
>
>
>
> Best regards,
>
> Sanket
>
>
Re: Switching alert status
Posted by Sanket Sharma <sa...@dukstra.com>.
Excellent! Thank you so much for the assistance.
I was able to create the alerts as suggested. My understanding is that stellar rules (and scores) configured in the config UI will only be evaluated if the is_alert flag is set to true. Is that correct?
Is there perhaps a way to toggle the is_alert flag based on the threat triage score (or alternatively a way to calculate score without setting the is_alert flag to true?) The use case is that we still want to score incoming records (based on stellar rules) but don’t want them displaying in the alerts UI unless they have a score. We could potentially filter the records from the alerts UI I suppose or clean up the Elastic index periodically, but wondering if something like this can done out of the box?
Best regards,
Sanket
From: Nick Allen <ni...@nickallen.org>
Reply to: "user@metron.apache.org" <us...@metron.apache.org>
Date: Thursday, 21 November 2019 at 20:45
To: "user@metron.apache.org" <us...@metron.apache.org>
Subject: Re: Switching alert status
Yes. You would create an enrichment using a Stellar expression that looks something like this.
is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
I can create, test and load that enrichment using the Stellar REPL.
[root@node1 0.7.2]# source /etc/default/metron
[root@node1 0.7.2]# $METRON_HOME/bin/stellar -z $ZOOKEEPER
Let's test out the expression to make sure it does what we would expect. You can make the expression as complex as you need for your use case using Stellar<https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>.
[Stellar]>>> ip_src_addr := "192.168.0.22"
192.168.0.22
[Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24<http://192.168.0.0/24>')
true
Need more help?
[Stellar]>>> ?IN_SUBNET
IN_SUBNET
Description: Returns true if an IP is within a subnet range.
Arguments:
ip - The IP address in string form
cidr+ - One or more IP ranges specified in CIDR notation (for example 192.168.0.0/24<http://192.168.0.0/24>)
Returns: True if the IP address is within at least one of the network ranges and false if otherwise
Then I can create and save the enrichment configuration necessary to apply your Stellar expression.
[Stellar]>>> bro := SHELL_EDIT()
{
"enrichment" : {
"fieldMap": {
"stellar" : {
"config" : {
"is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24'<http://192.168.0.0/24'>)"
}
}
}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {}
}
}
[Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
Hope this helps. Good luck.
On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>> wrote:
Hi,
I was wondering if there is a way to switch the alert status on or off based on rule or condition? For instance, I have two different subnets where I’m monitoring network traffic. One is an open network and the other is a private/secured network.
I would like to set ‘is_alert’ to true if the src is subnet two and set it to false when its subnet one. The end goal is to only have certain alerts displayed in the alerts UI based on conditions. Is it possible?
Best regards,
Sanket
Re: Switching alert status
Posted by Nick Allen <ni...@nickallen.org>.
Yes. You would create an enrichment using a Stellar expression that looks
something like this.
is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
I can create, test and load that enrichment using the Stellar REPL.
[root@node1 0.7.2]# source /etc/default/metron[root@node1 0.7.2]#
$METRON_HOME/bin/stellar -z $ZOOKEEPER
Let's test out the expression to make sure it does what we would expect.
You can make the expression as complex as you need for your use case using
Stellar
<https://metron.apache.org/current-book/metron-stellar/stellar-common/index.html>
.
[Stellar]>>> ip_src_addr := "192.168.0.22"
192.168.0.22
[Stellar]>>> is_alert := IN_SUBNET(ip_src_addr, '192.168.0.0/24')
true
Need more help?
[Stellar]>>> ?IN_SUBNET
IN_SUBNET
Description: Returns true if an IP is within a subnet range.
Arguments:
ip - The IP address in string form
cidr+ - One or more IP ranges specified in CIDR notation (for example
192.168.0.0/24)
Returns: True if the IP address is within at least one of the network
ranges and false if otherwise
Then I can create and save the enrichment configuration necessary to apply
your Stellar expression.
[Stellar]>>> bro := SHELL_EDIT()
{
"enrichment" : {
"fieldMap": {
"stellar" : {
"config" : {
"is_alert" : "IN_SUBNET(ip_src_addr, '192.168.0.0/24')"
}
}
}
},
"threatIntel": {
"fieldMap": {},
"fieldToTypeMap": {}
}
}[Stellar]>>> CONFIG_PUT("ENRICHMENT", bro, "bro")
Hope this helps. Good luck.
On Thu, Nov 21, 2019 at 1:21 PM Sanket Sharma <sa...@dukstra.com>
wrote:
> Hi,
>
>
>
> I was wondering if there is a way to switch the alert status on or off
> based on rule or condition? For instance, I have two different subnets
> where I’m monitoring network traffic. One is an open network and the other
> is a private/secured network.
>
>
>
> I would like to set ‘is_alert’ to true if the src is subnet two and set it
> to false when its subnet one. The end goal is to only have certain alerts
> displayed in the alerts UI based on conditions. Is it possible?
>
>
>
>
>
> Best regards,
>
> Sanket
>