You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by justinleet <gi...@git.apache.org> on 2017/10/23 12:08:54 UTC
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
GitHub user justinleet opened a pull request:
https://github.com/apache/metron/pull/811
METRON-1272: Hide child alerts from searches and grouping if they belong to meta alerts
## Contributor Comments
Adds the ability to hide alerts already contained in a meta alert from searches. It'll also hide it from the group mechanism (as they've already been sliced and diced).
This happens by adding a "metaalerts" field to our alerts, which is a simple list of GUIDs. This does require partial updates to alerts when a meta alert is created or has the status changed from active to inactive or vice versa. Alerts can potentially be updated when the "alert" field of a meta alert is changed (i.e. an alert is added or removed), but this should be limited to alerts that are actually affected. Given that there are practical limits (iirc, ~1000) alerts that can be in a given meta alert and this is a manual action, it shouldn't be particularly bad.
A batchUpdate is added to the IndexDao to avoid making a ton of updates to ES. It's added accordingly through the implementors (HBase just uses the List<Put> method, if we want something else let me know, I haven't touched that stuff in awhile). ElasticsearchMetaAlertDao throws an UnsupportedOperationException both to avoid having to implement it and also because creating meta alerts should generally be a single action as a result of manual intervention, not a bulk operation.
Right now, there is an update to the templates to make them "not_analyzed". This is unfortunately due to our GUIDs being analyzed fields. I could potentially delete and reinsert the hyphens appropriately if we care to avoid that and are willing to absorb a bit of a kludge.
I owe a test plan for this, along with a couple unit tests (particularly around grouping, and shifting the status of the meta alert and ensuring queries work as expected). Having said that, it should be fairly testable now on full dev, just by creating a couple meta alerts with a couple child alerts and running appropriate searches.
Also definitely owe some documentation, particularly if we need to keep the field in the templates (and add it as a requirement).
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [ ] Have you written or updated unit tests and or integration tests to verify your changes?
- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/justinleet/metron meta_hiding
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/811.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #811
----
commit b03267b6a0aa71fabedcc0fed1f768b64676b914
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T11:26:57Z
Untested hiding in both grouping and search
commit e54865e203a57111e2f2485a2e6e0d49e0e9fd2a
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T13:37:18Z
better
commit 58f1edba1d900d7e66687ee1a05e08cae95b4663
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T15:16:40Z
Maybe fixing creating meta alert
commit 7e97694fa8eb3a750d2daaf2d41bc52a2150fdb0
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T16:10:42Z
Adding an alert should be working now
commit 28bb5c13199a483d9c8c51bb782293050a429fe9
Author: merrimanr <me...@gmail.com>
Date: 2017-10-17T16:15:24Z
initial commit
commit 3aa4c158c11c1a72e2e80a9fa2e98a6f1be57348
Author: merrimanr <me...@gmail.com>
Date: 2017-10-17T16:16:07Z
Merge remote-tracking branch 'mirror/master' into METRON-1255
commit bc923540e3a33225d458db9505045cf062d553f2
Author: merrimanr <me...@gmail.com>
Date: 2017-10-17T18:02:49Z
resolved merge conflicts
commit 76f29057ecbd694f513ab6083e4ef921882f9944
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T22:37:51Z
create alert updates happen in batch. Not adding calculations on, though
commit 7508e111f27a890c686b814450c57a17265ae77d
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T23:57:11Z
Fixing create
commit 646b8d6bfae681dabfd5f9b192633593f54fe5cd
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T02:26:59Z
Moving bulkUpdate to interface. Fixing other bugs
commit bfe179bbe09f24e7be7edf4d87b45aef88cf7805
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T02:57:25Z
refactoring
commit b186a116b5c00af0e104728b0040b0e330eb2881
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T03:43:22Z
refactoring
commit 05cc0cc1368fbbc5708ae09404acfddf1606ee04
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T03:53:00Z
Merge branch 'master' into meta_hiding
commit cb6569b9346a319f55d3df12b91c310d402c257c
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T12:02:29Z
Merge branch 'METRON-1255' into meta_hiding
commit 22896fb841bd6831876352d75572037f147117ba
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T13:37:26Z
Renaming method from bulk to batch
commit 22e7462f2a6cae917bffc5c4b39f10b579848e63
Author: merrimanr <me...@gmail.com>
Date: 2017-10-18T19:53:43Z
initial commit
commit 319e130d13ca58bb6ebead96250fed9a7fe5cfae
Author: merrimanr <me...@gmail.com>
Date: 2017-10-18T21:28:04Z
Merge remote-tracking branch 'mirror/master' into metaalert-alert-update
# Conflicts:
# metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
# metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
commit 7bb7d946fc1b400a3ff0ed1a79a029650609387a
Author: merrimanr <me...@gmail.com>
Date: 2017-10-18T21:43:28Z
fixed a couple findUpdatedDoc statements
commit b926bfd99560dbbfc3fbd6c88a00e2b3ccee6515
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-19T12:56:33Z
remove comment
commit b5290d1b6e0e295c35ffd4e283c23245896a9fbe
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-19T14:06:27Z
Changing group to just hide alerts in a meta alert
commit 395a5a6e546e8539fc191a07f93aa48517d4acc8
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-19T14:11:50Z
Merge branch 'master' into meta_hiding
commit e669a7f98a42dfec8d70621f299f347d3566a110
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-20T19:28:57Z
Merge branch 'METRON-1262' into meta_hiding
commit 860cb846140a91ad028623f14e6c2d545f89905e
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-21T22:28:01Z
Finally get unit tests working. At least it was a unit test problem
commit 68bce64dfd6c1674464d7c9a57e2685d034a0644
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:29:18Z
Fixing / adjusting test and apparently lost the hiding in a merge
commit ff99ad1730a5b650e02d6c02433915666664a35a
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:34:19Z
Improving search test
commit 00203d26f928afd613744af6885c03be6a057de3
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:35:44Z
Adding comment
commit 3061ecdc6c19c223e5dddfef9f7b412fdbcd7fce
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:53:02Z
Merge branch 'master' into meta_hiding
----
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
> I find that extremely confusing as a user of the tool.
To explain that a bit more (and continuing with that same basic example)...
As a user I created a meta-alert where the hostname is "ip-addr.es". Since I created a meta-alert around that specific hostname, that must be a pretty important host name. It is probably something I am investigating right now.
Now imagine I ask my Tier III to take a look at that weird hostname. He's going to do something like this to attempt to find that problematic hostname.
It appears that the hostname "ip-addr.es" is completely missing. It is as if we lost data. This is the kind of work flow that I think is very confusing.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
I am seeing another issue that may or may not be related. It seems that when I am using the "group by" functionality, I cannot see meta-alerts at all.
(1) If I am not using the "group by", I can see the meta-alert perfectly fine.
(2) Now I want to group by host. I click the "host" group by widget, but there are no results. I am left thinking... Where did the meta-alert go? The only way I can see the meta-alert is to not use the "group by" functionality.
(3) And the UI screenshots match what is returned by the underlying API. In the case of the missing meta-alert, this is the request/response.
Request:
```
{
"indices": [
"websphere",
"snort",
"asa",
"bro",
"yaf",
"metaalert"
],
"scoreField": "threat:triage:score",
"groups": [
{
"field": "host",
"order": {
"sortOrder": "desc",
"groupOrderType": "term"
}
}
],
"query": "(host:ip\\-addr.es OR alert.host:ip\\-addr.es)"
}
```
Response:
```
{"groupedBy":"host","groupResults":[]}
```
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/811
### Testing
Testing purely based on the REST API. Having said that, if you want to do like @nickwallen did and pull in https://github.com/apache/metron/pull/803 into the same branch, you should be able to follow his examples that he posted screenshots for. Having done this myself, there should be no merge conflicts.
Throughout the following instructions, make sure to substitute GUIDs and index appropriately.
Create a meta alert.
```
{
"groups": [
"string"
],
"guidToIndices": {
"c4060f38-aa20-4670-a85f-9d6e9fe0931a":"bro_index_2017.10.25.13"
}
}
```
Retrieve the meta alert. Either use curl or use `findOne` endpoint on the returned GUID from the previous response.
```
curl -XGET "node1:9200/metaalert_index/metaalert_doc/54f979ec-1491-4c8d-9292-db9202cdba86?pretty"
```
Ensure the contained alert has the appropriate guid, e.g.
```
...
"guid": "c4060f38-aa20-4670-a85f-9d6e9fe0931a",
...
```
Retrieve the child alert, again through curl or `findOne`
```
curl -XGET "node1:9200/bro_index_2017.10.25.13/bro_doc/c4060f38-aa20-4670-a85f-9d6e9fe0931a?pretty"
```
Ensure the meta alerts field contains a link to the meta alert we created.
```
"metaalerts": [
"54f979ec-1491-4c8d-9292-db9202cdba86"
]
```
Run a search that should match both and ensure only the meta alert is returned
```
{
"fields": [
"*"
],
"from": 0,
"size": 10,
"indices": [
"metaalert",
"bro"
],
"query": "guid:\"c4060f38-aa20-4670-a85f-9d6e9fe0931a\" OR alert.guid:\"c4060f38-aa20-4670-a85f-9d6e9fe0931a\""
}
```
returns
```
{
"total": 1,
"results": [
{
"id": "54f979ec-1491-4c8d-9292-db9202cdba86",
"source": {
"_timestamp": 1508942592157
},
"score": 1,
"index": "metaalert_index"
}
]
}
```
This can also be run where multiple matches happen
```
{
"fields": [
"*"
],
"from": 0,
"size": 10,
"indices": [
"metaalert",
"bro"
],
"query": "ip_dst_addr:192.168.66.121 OR alert.ip_dst_addr:192.168.66.121"
}
```
Results are omitted for size, but the child alert should not show up, but the meta alert should. Just do a find on the results if there are a lot or modify the query.
Updating Status
A status update will carry through for all the child alerts. I.e. going from "active" to "inactive" and vice-versa will manage the links.
```
{
"guid": "54f979ec-1491-4c8d-9292-db9202cdba86",
"index": "metaalert",
"patch": [{
"op": "replace"
, "path": "/status"
, "value": "inactive"
}],
"sensorType": "metaalert",
"source": {}
}
```
Retrieving the meta alert will now give an "inactive" status
Retrieving the alert itself again will now have the metaalerts field as follows (assuming this alert isn't in any other meta alerts):
```
"metaalerts": []
```
It should also show up in our search again (but the metaalert won't!)
Try changing it back to active and ensure the child is linked again and is hidden from the results.
```
{
"guid": "54f979ec-1491-4c8d-9292-db9202cdba86",
"index": "metaalert",
"patch": [{
"op": "replace"
, "path": "/status"
, "value": "active"
}],
"sensorType": "metaalert",
"source": {}
}
```
The link in the alert should also be back.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/811
Master is merged in, so this Travis run should be legit
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by merrimanr <gi...@git.apache.org>.
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/811
I believe excluding metaalerts from the group by view is the desired behavior.
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146847834
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java ---
@@ -295,19 +341,199 @@ protected Document buildCreateDocument(MultiGetResponse multiGetResponse, List<S
/**
* Process an update to a meta alert itself.
* @param update The update Document to be applied
- * @param index The optional index to update to
* @throws IOException If there's a problem running the update
*/
- protected void handleMetaUpdate(Document update, Optional<String> index) throws IOException {
- // We have an update to a meta alert itself
- // If we've updated the alerts field (i.e add/remove), recalculate meta alert scores.
+ protected void handleMetaUpdate(Document update) throws IOException {
--- End diff --
I can help debug further. I just wanted to get the report of the problem out there first. I'll research how to kick-up logging on the API so we can actually see what's going on.
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146965770
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java ---
@@ -295,19 +341,199 @@ protected Document buildCreateDocument(MultiGetResponse multiGetResponse, List<S
/**
* Process an update to a meta alert itself.
* @param update The update Document to be applied
- * @param index The optional index to update to
* @throws IOException If there's a problem running the update
*/
- protected void handleMetaUpdate(Document update, Optional<String> index) throws IOException {
- // We have an update to a meta alert itself
- // If we've updated the alerts field (i.e add/remove), recalculate meta alert scores.
+ protected void handleMetaUpdate(Document update) throws IOException {
--- End diff --
I added a comment on https://github.com/apache/metron/pull/803/, because the 999 limit is actually only established there, it's not something in master.
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146826488
--- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template ---
@@ -154,6 +154,10 @@
"alert": {
"type": "nested"
},
+ "metaalerts": {
--- End diff --
I was actually double checking if these changes were actually needed when you commented on this. I thought it was, but after actually implementing it, it's not. We don't search on this field or use it as anything other than a list of source strings, so we don't actually care that the analyzer may slice and dice the GUID.
I just removed those changes from the PR, because while it may be nicer ES-wise to have them defined, I'd personally prefer to keep the overhead on new sensors as low as possible. Let me know if that's alright with you and we can figure out if we want to add it back and document it or not.
Basically, to check this, do the following:
Create a meta alert, note the id (e.g. "bcf91b5b-1b14-42c1-98be-ef405ed48c70")
```
{
"groups": [
"string"
],
"guidToIndices": {
"7764f686-f0b0-4657-90d4-42601a4e5ef1":"bro_index_2017.10.24.20"
}
}
```
Pull back the child alert. It'll have a meta alert field similar to this:
```
"metaalerts": [
"bcf91b5b-1b14-42c1-98be-ef405ed48c70"
]
```
Create another meta alert, with the same alert as a child. Again, note the id ("1685d5ac-c3c7-4bdb-b4f1-ba9a17a00947")
```
{
"groups": [
"string"
],
"guidToIndices": {
"7764f686-f0b0-4657-90d4-42601a4e5ef1":"bro_index_2017.10.24.20",
"0241c002-2198-4fa4-a2e6-a33725fd9901":"bro_index_2017.10.24.20"
}
}
```
Pull back the child alert again. It'll have a meta alert field similar to this:
```
"metaalerts": [
"1685d5ac-c3c7-4bdb-b4f1-ba9a17a00947",
"bcf91b5b-1b14-42c1-98be-ef405ed48c70"
]
```
The second meta alert should properly have populated the field.
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146712417
--- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template ---
@@ -154,6 +154,10 @@
"alert": {
"type": "nested"
},
+ "metaalerts": {
--- End diff --
Do we need to mention the template changes in Upgrading.md?
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/811
@nickwallen I added testing around the status stuff, and a couple docs. Let me know if there's anything else that should be updated (or isn't clear or whatever else).
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/811
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
It appears to me that the alerts contained within a meta-alert are not contributing to the facet counts returned by a search request. I think we still do want that to happen. Let me explain with an example and screenshots.
Note: I am testing this together with the UI changes in #803 .
1. First, I isolate 10 alerts with a specific host name that I want to work with. I have turned off ingest so that no additional alerts should appear during the running of this example. I can see that the facet counts are what I would expect.
2. Next, I group by host so that I can create my meta-alert.
3. Immediately after creating the meta-alert, I do not immediately see it. I think this is a problem with the UI itself not refreshing after creating the alert. This might need fixed in #803 .
4. If I then trigger another search, I do see the meta-alert. Great!
5. Next I just expand the meta-alert to validate that the 10 original alerts were added. You can see from this screenshot that the facet counts all show 0. This tells me that the facet counts are not including meta-alerts.
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146712573
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java ---
@@ -174,21 +175,34 @@ protected SearchResponse search(SearchRequest searchRequest, QueryBuilder queryB
@Override
public GroupResponse group(GroupRequest groupRequest) throws InvalidSearchException {
- if(client == null) {
+ return group(groupRequest, new QueryStringQueryBuilder(groupRequest.getQuery()));
+ }
+
+ /**
+ * Defers to a provided {@link org.elasticsearch.index.query.QueryBuilder} for the query.
+ * @param groupRequest The request defining the parameters of the grouping
+ * @param queryBuilder The actual query to be run. Intended for if the SearchRequest requires wrapping
+ * @return The results of the query
+ * @throws InvalidSearchException When the query is malformed or the current state doesn't allow search
+ */
--- End diff --
Thanks for the javadoc!
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet closed the pull request at:
https://github.com/apache/metron/pull/811
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by justinleet <gi...@git.apache.org>.
GitHub user justinleet reopened a pull request:
https://github.com/apache/metron/pull/811
METRON-1272: Hide child alerts from searches and grouping if they belong to meta alerts
## Contributor Comments
Adds the ability to hide alerts already contained in a meta alert from searches. It'll also hide it from the group mechanism (as they've already been sliced and diced).
This happens by adding a "metaalerts" field to our alerts, which is a simple list of GUIDs. This does require partial updates to alerts when a meta alert is created or has the status changed from active to inactive or vice versa. Alerts can potentially be updated when the "alert" field of a meta alert is changed (i.e. an alert is added or removed), but this should be limited to alerts that are actually affected. Given that there are practical limits (iirc, ~1000) alerts that can be in a given meta alert and this is a manual action, it shouldn't be particularly bad.
A batchUpdate is added to the IndexDao to avoid making a ton of updates to ES. It's added accordingly through the implementors (HBase just uses the List<Put> method, if we want something else let me know, I haven't touched that stuff in awhile). ElasticsearchMetaAlertDao throws an UnsupportedOperationException both to avoid having to implement it and also because creating meta alerts should generally be a single action as a result of manual intervention, not a bulk operation.
Right now, there is an update to the templates to make them "not_analyzed". This is unfortunately due to our GUIDs being analyzed fields. I could potentially delete and reinsert the hyphens appropriately if we care to avoid that and are willing to absorb a bit of a kludge.
I owe a test plan for this, along with a couple unit tests (particularly ~around grouping, and~ shifting the status of the meta alert and ensuring queries work as expected). Having said that, it should be fairly testable now on full dev, just by creating a couple meta alerts with a couple child alerts and running appropriate searches.
Also definitely owe some documentation, particularly if we need to keep the field in the templates (and add it as a requirement).
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions.
Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [ ] Have you written or updated unit tests and or integration tests to verify your changes?
- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/justinleet/metron meta_hiding
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/811.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #811
----
commit b03267b6a0aa71fabedcc0fed1f768b64676b914
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T11:26:57Z
Untested hiding in both grouping and search
commit e54865e203a57111e2f2485a2e6e0d49e0e9fd2a
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T13:37:18Z
better
commit 58f1edba1d900d7e66687ee1a05e08cae95b4663
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T15:16:40Z
Maybe fixing creating meta alert
commit 7e97694fa8eb3a750d2daaf2d41bc52a2150fdb0
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T16:10:42Z
Adding an alert should be working now
commit 28bb5c13199a483d9c8c51bb782293050a429fe9
Author: merrimanr <me...@gmail.com>
Date: 2017-10-17T16:15:24Z
initial commit
commit 3aa4c158c11c1a72e2e80a9fa2e98a6f1be57348
Author: merrimanr <me...@gmail.com>
Date: 2017-10-17T16:16:07Z
Merge remote-tracking branch 'mirror/master' into METRON-1255
commit bc923540e3a33225d458db9505045cf062d553f2
Author: merrimanr <me...@gmail.com>
Date: 2017-10-17T18:02:49Z
resolved merge conflicts
commit 76f29057ecbd694f513ab6083e4ef921882f9944
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T22:37:51Z
create alert updates happen in batch. Not adding calculations on, though
commit 7508e111f27a890c686b814450c57a17265ae77d
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-17T23:57:11Z
Fixing create
commit 646b8d6bfae681dabfd5f9b192633593f54fe5cd
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T02:26:59Z
Moving bulkUpdate to interface. Fixing other bugs
commit bfe179bbe09f24e7be7edf4d87b45aef88cf7805
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T02:57:25Z
refactoring
commit b186a116b5c00af0e104728b0040b0e330eb2881
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T03:43:22Z
refactoring
commit 05cc0cc1368fbbc5708ae09404acfddf1606ee04
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T03:53:00Z
Merge branch 'master' into meta_hiding
commit cb6569b9346a319f55d3df12b91c310d402c257c
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T12:02:29Z
Merge branch 'METRON-1255' into meta_hiding
commit 22896fb841bd6831876352d75572037f147117ba
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-18T13:37:26Z
Renaming method from bulk to batch
commit 22e7462f2a6cae917bffc5c4b39f10b579848e63
Author: merrimanr <me...@gmail.com>
Date: 2017-10-18T19:53:43Z
initial commit
commit 319e130d13ca58bb6ebead96250fed9a7fe5cfae
Author: merrimanr <me...@gmail.com>
Date: 2017-10-18T21:28:04Z
Merge remote-tracking branch 'mirror/master' into metaalert-alert-update
# Conflicts:
# metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java
# metron-platform/metron-elasticsearch/src/test/java/org/apache/metron/elasticsearch/integration/ElasticsearchMetaAlertIntegrationTest.java
commit 7bb7d946fc1b400a3ff0ed1a79a029650609387a
Author: merrimanr <me...@gmail.com>
Date: 2017-10-18T21:43:28Z
fixed a couple findUpdatedDoc statements
commit b926bfd99560dbbfc3fbd6c88a00e2b3ccee6515
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-19T12:56:33Z
remove comment
commit b5290d1b6e0e295c35ffd4e283c23245896a9fbe
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-19T14:06:27Z
Changing group to just hide alerts in a meta alert
commit 395a5a6e546e8539fc191a07f93aa48517d4acc8
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-19T14:11:50Z
Merge branch 'master' into meta_hiding
commit e669a7f98a42dfec8d70621f299f347d3566a110
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-20T19:28:57Z
Merge branch 'METRON-1262' into meta_hiding
commit 860cb846140a91ad028623f14e6c2d545f89905e
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-21T22:28:01Z
Finally get unit tests working. At least it was a unit test problem
commit 68bce64dfd6c1674464d7c9a57e2685d034a0644
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:29:18Z
Fixing / adjusting test and apparently lost the hiding in a merge
commit ff99ad1730a5b650e02d6c02433915666664a35a
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:34:19Z
Improving search test
commit 00203d26f928afd613744af6885c03be6a057de3
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:35:44Z
Adding comment
commit 3061ecdc6c19c223e5dddfef9f7b412fdbcd7fce
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T11:53:02Z
Merge branch 'master' into meta_hiding
commit 1680c75980ee8a94a76cf90cfec570080939e519
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T12:20:41Z
Don't break things when merging
commit 664cc830c7ef137f0abf5dcb910279813b49bfd1
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T12:54:16Z
Grouping test
commit 4f60d52093d39a42df2e6ecccc5c8a77182595a7
Author: justinjleet <ju...@gmail.com>
Date: 2017-10-23T18:34:27Z
Changing ES component to actually clean up after itself during reset
----
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
> @nickwallen suppose you have a metaalert that contains 2 alerts. Then suppose each alert has a different value for the host field. If you grouped on host, which group would you expect the metaalert to appear in?
I would say both. Let me try and explain that.
To me, the "group by" functionality is super-powerful for digging down deep into the alerts data. When I group by hostname and open that accordian, I expect to see all the alerts related to that hostname. That would include a meta-alert that has even 1 contained alert related to that hostname.
Maybe there are other corner cases that I am not considering, but that's what I would expect.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
Thanks @justinleet . This is working well with @iraghumitra UI work in #803.
The metalerts show-up in the left-side "Filters" panel, which provides a decent short-cut to retrieve the metaalerts. For example, here I created 4 metaalerts. If I click on "metaalerts" on that left panel, it filters everything down to metaalerts.
Filtering does nicely retrieve both alerts and metaalerts.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on the issue:
https://github.com/apache/metron/pull/811
@nickwallen what you are looking at is a desired behavior. If the alerts are a part of the meta alert they do not appear in the facets
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
+1 This was more challenging than meets the eye. Thanks for working through this @justinleet . We have some use cases to figure out, but this is a good first step toward metaalerts.
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146925677
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java ---
@@ -295,19 +341,199 @@ protected Document buildCreateDocument(MultiGetResponse multiGetResponse, List<S
/**
* Process an update to a meta alert itself.
* @param update The update Document to be applied
- * @param index The optional index to update to
* @throws IOException If there's a problem running the update
*/
- protected void handleMetaUpdate(Document update, Optional<String> index) throws IOException {
- // We have an update to a meta alert itself
- // If we've updated the alerts field (i.e add/remove), recalculate meta alert scores.
+ protected void handleMetaUpdate(Document update) throws IOException {
--- End diff --
The problem here seems unrelated to your PR. I think the serialized JSON is just too large with these ~500 alerts when it attempts to write them to HBase. I think we impose a limit of 999 currently, but I'm not sure a fixed limit is going to always work. It depends on the size of each alert.
I will create a JIRA to track this separately.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
Another thing I noticed is that there does not seem to be a way to retrieve meta-alerts that you have created.
For example, I created a meta-alert where `host:ip-addr.es`. Assuming we exclude metaalerts from all the nice "group by" functionality, then I need to find it using search. But if I search with `host:ip-addr.es` or even `alert.host.ip-addr.es` it does not return any results.
When using `alert.host` the UI submits the following which returns no results.
```
{
"indices": [
"websphere",
"snort",
"asa",
"bro",
"yaf",
"metaalert"
],
"facetFields": [
"source:type",
"ip_src_addr",
"ip_dst_addr",
"host",
"enrichments:geo:ip_dst_addr:country"
],
"query": "(host:ip\\-addr.es OR alert.host:ip\\-addr.es)",
"from": 0,
"size": 25,
"sort": [
{
"field": "host",
"sortOrder": "asc"
}
]
}
```
I ran this by @justinleet and we found that it is the `sort` field that prevents the meta-alerts from being returned. Removing the sort field like the following query does actually return the meta-alert.
```
{
"indices": [
"websphere",
"snort",
"asa",
"bro",
"yaf",
"metaalert"
],
"facetFields": [
"source:type",
"ip_src_addr",
"ip_dst_addr",
"host",
"enrichments:geo:ip_dst_addr:country"
],
"query": "(host:ip\\-addr.es OR alert.host:ip\\-addr.es)",
"from": 0,
"size": 25
}
```
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146827154
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java ---
@@ -295,19 +341,199 @@ protected Document buildCreateDocument(MultiGetResponse multiGetResponse, List<S
/**
* Process an update to a meta alert itself.
* @param update The update Document to be applied
- * @param index The optional index to update to
* @throws IOException If there's a problem running the update
*/
- protected void handleMetaUpdate(Document update, Optional<String> index) throws IOException {
- // We have an update to a meta alert itself
- // If we've updated the alerts field (i.e add/remove), recalculate meta alert scores.
+ protected void handleMetaUpdate(Document update) throws IOException {
--- End diff --
I'll try to reproduce. I'm not sure what the potential cause could be. Could you see if the alerts you added have the properly populated "metaalerts" field in the original alert? It's possible an issue occurs during that update, but I don't really have any evidence in any direction.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on the issue:
https://github.com/apache/metron/pull/811
+1 from me as well. Great job @justinleet
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146848022
--- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/files/bro_index.template ---
@@ -154,6 +154,10 @@
"alert": {
"type": "nested"
},
+ "metaalerts": {
--- End diff --
If it works without the template changes, then great. Less is more.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/811
@james-sirota I find that extremely confusing as a user of the tool. We need to document that fact and some reasoning behind it at the very least.
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146937137
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java ---
@@ -295,19 +341,199 @@ protected Document buildCreateDocument(MultiGetResponse multiGetResponse, List<S
/**
* Process an update to a meta alert itself.
* @param update The update Document to be applied
- * @param index The optional index to update to
* @throws IOException If there's a problem running the update
*/
- protected void handleMetaUpdate(Document update, Optional<String> index) throws IOException {
- // We have an update to a meta alert itself
- // If we've updated the alerts field (i.e add/remove), recalculate meta alert scores.
+ protected void handleMetaUpdate(Document update) throws IOException {
--- End diff --
https://issues.apache.org/jira/browse/METRON-1279
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on the issue:
https://github.com/apache/metron/pull/811
@nickwallen to avoid scope creep on this PR I created a follow-on PR to figure out how to represent meta alerts in the facet panel. https://issues.apache.org/jira/browse/METRON-1276
I agreee with @merrimanr that meta alerts contains multiple alerts each with with distinct meta data and faceting them with the non-grouped alerts doesn't make sense IMO. We do need to figure out a way to visualize metadata of the meta alert (maybe have a separate facet panel for meta alerts?), but I think this will add scope to this PR and therefore should be resolved outside of this PR.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/811
@nickwallen It's not a perfect solution, but I added the "source:type" field for consistency and to allow for filtering and retrieval of just meta alerts in an obvious way. Should just be "source:type" of "metaalert"
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by merrimanr <gi...@git.apache.org>.
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/811
@nickwallen suppose you have a metaalert that contains 2 alerts. Then suppose each alert has a different value for the host field. If you grouped on host, which group would you expect the metaalert to appear in?
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146715818
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java ---
@@ -295,19 +341,199 @@ protected Document buildCreateDocument(MultiGetResponse multiGetResponse, List<S
/**
* Process an update to a meta alert itself.
* @param update The update Document to be applied
- * @param index The optional index to update to
* @throws IOException If there's a problem running the update
*/
- protected void handleMetaUpdate(Document update, Optional<String> index) throws IOException {
- // We have an update to a meta alert itself
- // If we've updated the alerts field (i.e add/remove), recalculate meta alert scores.
+ protected void handleMetaUpdate(Document update) throws IOException {
--- End diff --
If I have a metaalert with a large number of alerts (say 525 in this example), then I attempt to add 1 or 2 alerts to that metaalert, the REST API responds back with a 500 error.
I don't see any error messages in the logs. I assume that error is coming from somewhere in here.
The UI itself never reports an error or acknowledges. Of course that is a separate problem that we should fix in a separate PR.
But I do find that the metaalert was actually updated, as the number of alerts increased (from 527 to 529 here).
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/811
Travis failure should be resolved once https://github.com/apache/metron/pull/816 is in, and I'll merge in master and push again once it is.
---
[GitHub] metron issue #811: METRON-1272: Hide child alerts from searches and grouping...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/811
Kick Travis
---
[GitHub] metron pull request #811: METRON-1272: Hide child alerts from searches and g...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/811#discussion_r146899435
--- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchMetaAlertDao.java ---
@@ -295,19 +341,199 @@ protected Document buildCreateDocument(MultiGetResponse multiGetResponse, List<S
/**
* Process an update to a meta alert itself.
* @param update The update Document to be applied
- * @param index The optional index to update to
* @throws IOException If there's a problem running the update
*/
- protected void handleMetaUpdate(Document update, Optional<String> index) throws IOException {
- // We have an update to a meta alert itself
- // If we've updated the alerts field (i.e add/remove), recalculate meta alert scores.
+ protected void handleMetaUpdate(Document update) throws IOException {
--- End diff --
Here is the exception that pops out with that 500.
```
"fullMessage":"IOException: class org.apache.metron.indexing.dao.HBaseDao: KeyValue size too large
java.lang.IllegalArgumentException: KeyValue size too large
\tat org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1521)
\tat org.apache.hadoop.hbase.client.BufferedMutatorImpl.validatePut(BufferedMutatorImpl.java:147)
\tat org.apache.hadoop.hbase.client.BufferedMutatorImpl.doMutate(BufferedMutatorImpl.java:134)
\tat org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:105)
\tat org.apache.hadoop.hbase.client.HTable.put(HTable.java:1050)
\tat org.apache.metron.indexing.dao.HBaseDao.batchUpdate(HBaseDao.java:140)
\tat org.apache.metron.indexing.dao.MultiIndexDao.lambda$batchUpdate$3(MultiIndexDao.java:78)
\tat java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
\tat java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
\tat java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
\tat java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
\tat java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:747)
\tat java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:721)
\tat java.util.stream.AbstractTask.compute(AbstractTask.java:316)
\tat java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
\tat java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
\tat java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
\tat java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
\tat java.util.stream.ReduceOps$ReduceOp.evaluateParallel(ReduceOps.java:714)
\tat java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
\tat java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
\tat org.apache.metron.indexing.dao.MultiIndexDao.batchUpdate(MultiIndexDao.java:83)
\tat org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.indexDaoUpdate(ElasticsearchMetaAlertDao.java:374)
\tat org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.handleMetaUpdate(ElasticsearchMetaAlertDao.java:361)
\tat org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.update(ElasticsearchMetaAlertDao.java:248)
\tat org.apache.metron.indexing.dao.IndexDao.patch(IndexDao.java:130)
\tat org.apache.metron.rest.service.impl.UpdateServiceImpl.patch(UpdateServiceImpl.java:44)
\tat org.apache.metron.rest.controller.UpdateController.patch(UpdateController.java:52)
```
---