You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by justinleet <gi...@git.apache.org> on 2018/05/30 14:03:03 UTC
[GitHub] metron issue #1037: METRON-1547: Solr Comment Fields
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/1037
This can be tested both on via REST API and via the UI, although as noted above there is UI instability for right now.
It should work for both ES and Solr (although right now Solr requires `index` to be passed with requests, so make sure to add it in the following steps. `index` will be the same as `sensorType`).
The examples tests are done with curl, but could also be done in Swagger.
## To setup Solr
```
sudo su -
export METRON_HOME=/usr/metron/0.4.3
cd ${METRON_HOME}/bin/
./install_solr.sh
./create_collection.sh bro
./create_collection.sh yaf
./create_collection.sh snort
./create_collection.sh error
./create_collection.sh metaalert
```
Edit the globe config at ${METRON_HOME}/config/zookeeper/global.json to have "source.type.field"
, e.g.
"geo.hdfs.file" : "/apps/metron/geo/default/GeoLite2-City.mmdb.gz",
"source.type.field" : "source.type"
${METRON_HOME}/bin/zk_load_configs.sh -z node1:2181 -c GLOBAL -m PUSH -i ${METRON_HOME}/config/zookeeper/
Ensure the new config is found.
${METRON_HOME}/bin/zk_load_configs.sh -z node1:2181 -c GLOBAL -m DUMP
In Ambari:
Indexing -> Random Access Search Engine -> Solr
Restart Metron Indexing, Metron REST, and Metron Alerts UI
## Testing
Make sure to replace the guid and add index as needed throughout
### Get a GUID
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
"fields": [
"guid"
],
"from": 0,
"indices": [
"bro"
],
"query": "*:*",
"size": 1
}' 'http://node1:8082/api/v1/search/search'
```
#### Sample Response
```
{
"total": 2120,
"results": [
{
"id": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"source": {
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412"
},
"score": 1,
"index": "bro_index_2018.05.15.16"
}
],
"facetCounts": null
}
```
### Create a new comment
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: */*' -d '{
"comment": "My Comment",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro",
"timestamp": 1526401584951,
"username": "test_username"
}' 'http://node1:8082/api/v1/update/add/comment'
```
### Call findOne
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro"
}' 'http://node1:8082/api/v1/search/findOne'
```
#### Response should contain a new comments field
```
"comments": [
{
"comment": "My Comment",
"username": "test_username",
"timestamp": 1526401584951
}
]
```
### Add another comment
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: */*' -d '{
"comment": "My Comment 2",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro",
"timestamp": 1526401584955,
"username": "test_username_2"
}' 'http://node1:8082/api/v1/update/add/comment'
```
### Patch the comment with a new field
This ensures the raw form is properly translated during patch operations (otherwise it can be mangled and not be readable later)
```
curl -u user:password -X PATCH --header 'Content-Type: application/json' --header 'Accept: */*' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"patch": [
{
"op": "add",
"path": "/project",
"value": "metron"
}
],
"sensorType": "bro"
}' 'http://node1:8082/api/v1/update/patch'
```
### Find it again
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro"
}' 'http://node1:8082/api/v1/search/findOne'
```
Response should have both comments and the new field
```
"comments": [
{
"comment": "My Comment",
"username": "test_username",
"timestamp": 1526401584951
},
{
"comment": "My Comment 2",
"username": "test_username_2",
"timestamp": 1526401584955
}
]
```
### Remove comment
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: */*' -d '{
"comment": "My Comment",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro",
"timestamp": 1526401584951,
"username": "test_username"
}' 'http://node1:8082/api/v1/update/remove/comment'
```
### Find it again.
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"sensorType": "bro"
}' 'http://node1:8082/api/v1/search/findOne'
```
The comment should be removed, leaving something like
```
"comments": [
{
"comment": "My Comment 2",
"username": "test_username_2",
"timestamp": 1526401584955
}
]
```
If you repeat the remove, nothing should happen to the alert.
### Remove the remaining comment
```
curl -u user:password -X POST --header 'Content-Type: application/json' --header 'Accept: */*' -d '{
"comment": "My Comment 2",
"guid": "099042a2-ed3f-46df-8d44-2c42e3adf412",
"index": "bro_index_2018.05.15.16",
"sensorType": "bro",
"timestamp": 1526401584955,
"username": "test_username_2"
}' 'http://node1:8082/api/v1/update/remove/comment'
```
### Find the alert again.
It should no longer have comments.
If you repeat the remove, nothing should happen to the alert.
In the UI, similar steps can be performed to add comments to a particular alert, as well as deleting them. The index can be checked to ensure the alert itself has the correct results as above.
---