You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by nickwallen <gi...@git.apache.org> on 2017/11/09 17:52:56 UTC
[GitHub] metron pull request #838: METRON-1310 "Template Delete" Action Deletes Searc...
GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/838
METRON-1310 "Template Delete" Action Deletes Search Indices
The "Elasticsearch Template Delete" action within the Indexing service in Ambari, actually deletes the search indices. Per the name of the action, I would expect it to only delete the template definitions. This can be quite a fun surprise. :)
## Testing
1. Spin up Full Dev or an equivalent Metron environment.
1. Query Elasticsearch for the currently defined indices. Take note of all the indices that you have defined.
```
curl http://node1:9200/_cat/indices?v
```
1. Query Elasticsearch for the currently defined templates. There should be 5 in all; `snort_index`, `bro_index`, `metaalert_index`, `error_index`, and `yaf_index`.
```
curl http://node1:9200/_template | python -mjson.tool
```
1. Open Ambari and go to Metron > Metron Indexing. Then on the hosts page, click on Metron Indexing > Elasticsearch Template Delete.
1. Query Elasticsearch again for the currently defined indices. Make sure none were deleted.
1. Query Elasticsearch again for the currently defined templates. There should be none.
```
curl http://node1:9200/_template | python -mjson.tool
```
1. Open Ambari and go to Metron > Metron Indexing. Then on the hosts page, click on Metron Indexing > Elasticsearch Template Install.
1. Query Elasticsearch again for the currently defined templates. There should be at least 5 in all.
```
curl http://node1:9200/_template | python -mjson.tool
```
## Pull Request Checklist
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
- [ ] Have you written or updated unit tests and or integration tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1310
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/838.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #838
----
commit 6565b351b0a75aff87acce97afd36978bea00e93
Author: Nick Allen <ni...@nickallen.org>
Date: 2017-11-09T17:41:48Z
METRON-1310 "Template Delete" Action Deletes Search Indices
----
---
[GitHub] metron pull request #838: METRON-1310 "Template Delete" Action Deletes Searc...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/838
---
[GitHub] metron issue #838: METRON-1310 "Template Delete" Action Deletes Search Indic...
Posted by justinleet <gi...@git.apache.org>.
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/838
+1 by inspection. Thanks for the fix
---
[GitHub] metron pull request #838: METRON-1310 "Template Delete" Action Deletes Searc...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/838#discussion_r150038381
--- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/params_linux.py ---
@@ -195,7 +195,7 @@
snort_index_path = tmp_dir + "/snort_index.template"
yaf_index_path = tmp_dir + "/yaf_index.template"
error_index_path = tmp_dir + "/error_index.template"
-meta_index_path = tmp_dir + "/meta_index.mapping"
+meta_index_path = tmp_dir + "/meta_index.template"
--- End diff --
I renamed `meta_index.mapping` to `meta_index.template` to be consistent with the other templates.
---
[GitHub] metron pull request #838: METRON-1310 "Template Delete" Action Deletes Searc...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/838#discussion_r150038546
--- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/indexing_master.py ---
@@ -141,38 +141,43 @@ def elasticsearch_template_install(self, env):
File(params.meta_index_path,
mode=0755,
- content=StaticFile('meta_index.mapping')
+ content=StaticFile('meta_index.template')
)
- bro_cmd = ambari_format(
- 'curl -s -XPOST http://{es_http_url}/_template/bro_index -d @{bro_index_path}')
+ bro_cmd = ambari_format('curl -s -XPOST http://{es_http_url}/_template/bro_index -d @{bro_index_path}')
Execute(bro_cmd, logoutput=True)
- snort_cmd = ambari_format(
- 'curl -s -XPOST http://{es_http_url}/_template/snort_index -d @{snort_index_path}')
+
+ snort_cmd = ambari_format('curl -s -XPOST http://{es_http_url}/_template/snort_index -d @{snort_index_path}')
Execute(snort_cmd, logoutput=True)
- yaf_cmd = ambari_format(
- 'curl -s -XPOST http://{es_http_url}/_template/yaf_index -d @{yaf_index_path}')
+
+ yaf_cmd = ambari_format('curl -s -XPOST http://{es_http_url}/_template/yaf_index -d @{yaf_index_path}')
Execute(yaf_cmd, logoutput=True)
- error_cmd = ambari_format(
- 'curl -s -XPOST http://{es_http_url}/_template/error_index -d @{error_index_path}')
- Execute(error_cmd, logoutput=True)
- error_cmd = ambari_format(
- 'curl -s -XPOST http://{es_http_url}/metaalert_index -d @{meta_index_path}')
+
+ error_cmd = ambari_format('curl -s -XPOST http://{es_http_url}/_template/error_index -d @{error_index_path}')
Execute(error_cmd, logoutput=True)
+ meta_cmd = ambari_format('curl -s -XPOST http://{es_http_url}/_template/metaalert_index -d @{meta_index_path}')
+ Execute(meta_cmd, logoutput=True)
+
def elasticsearch_template_delete(self, env):
from params import params
env.set_params(params)
- bro_cmd = ambari_format('curl -s -XDELETE "http://{es_http_url}/bro_index*"')
+ bro_cmd = ambari_format('curl -s -XDELETE "http://{es_http_url}/_template/bro_index"')
--- End diff --
Let's delete the template, rather than the entire index.
---
[GitHub] metron issue #838: METRON-1310 "Template Delete" Action Deletes Search Indic...
Posted by cestella <gi...@git.apache.org>.
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/838
yeesh, good catch @nickwallen +1 by inspection.
---