[GitHub] metron issue #1065: METRON-1620: Fixes for forensic clustering use case exam...

[mmiklavc <gi...@git.apache.org>]

Github user mmiklavc commented on the issue:

    https://github.com/apache/metron/pull/1065
  
    **Testing**
    
    You can run through the full use case, if desired. If you want the TL;DR version to verify the template command, run the command in the README for creating the ES template. Then do the following:
    
    Make sure you have at least 1 other sensor with data, e.g. Bro. In full dev you should be set, otherwise cat data from our unit tests (https://github.com/apache/metron/blob/master/metron-platform/metron-integration-test/src/main/sample/data/bro/raw/BroExampleOutput) into the `bro` Kafka topic and make sure the bro topology is running.
    
    e.g.
    ```
    wget https://github.com/apache/metron/raw/master/metron-platform/metron-integration-test/src/main/sample/data/bro/raw/BroExampleOutput -O ~/sample-bro.json
    cat ~/sample-bro.json | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list $BROKERLIST --topic bro
    ```
    
    Next, check the template was loaded correctly:
    ```
    curl -XGET $ES_HOST'/_template/cowrie_index?pretty=true'
    ```
    
    Then load the following into ES:
    ```
    curl -XPUT $ES_HOST'/cowrie_index_1/cowrie_doc/1' -H 'Content-Type: application/json' -d'
    {
        "eventid" : "cowrie.command.input",
        "adapter:stellaradapter:end:ts" : "1529268179998",
        "threatinteljoinbolt:joiner:ts" : "1529268180010",
        "session" : "4c047bbc016c",
        "threat:triage:rules:0:comment" : "Determine if a host is blacklisted",
        "enrichmentsplitterbolt:splitter:begin:ts" : "1529268179997",
        "enrichmentjoinbolt:joiner:ts" : "1529268180002",
        "threat:triage:rules:0:name" : "Blacklisted Host",
        "src_ip" : "94.51.110.74",
        "source:type" : "cowrie",
        "isError" : 0,
        "original_string" : "{\"src_ip\":\"94.51.110.74\",\"eventid\":\"cowrie.command.input\",\"input\":\"\\/bin\\/busybox XUSRH\",\"system\":\"CowrieTelnetTransport,93,94.51.110.74\",\"isError\":0,\"session\":\"4c047bbc016c\",\"sensor\":\"a927e8b28666\",\"message\":\"CMD: \\/bin\\/busybox XUSRH\",\"timestamp\":\"2017-09-17T04:06:40.419195Z\"}",
        "threatintelsplitterbolt:splitter:end:ts" : "1529268180004",
        "similarity_bin" : "166524",
        "threat:triage:rules:0:score" : 10,
        "timestamp" : 1505621619195,
        "threat:triage:rules:0:reason" : "IP 94.51.110.74 is blacklisted",
        "enrichmentsplitterbolt:splitter:end:ts" : "1529268179997",
        "threat:triage:score" : 10.0,
        "is_alert" : "true",
        "adapter:stellaradapter:begin:ts" : "1529268179998",
        "message" : "CMD: /bin/busybox XUSRH",
        "input" : "/bin/busybox XUSRH",
        "blacklisted" : true,
        "system" : "CowrieTelnetTransport,93,94.51.110.74",
        "threatintelsplitterbolt:splitter:begin:ts" : "1529268180004",
        "guid" : "f4e441d2-74e7-4127-89c4-edcf8227f893",
        "sensor" : "a927e8b28666",
        "tlsh" : "87A002C029850AFE3C890231B18B743C002C10825E5028A6DC8D00C1F213FC6FD31D0C"
    }
    '
    ```
    
    Go to the Alerts UI and enter this in the search:
    ```
    is_alert:true AND similarity_bin:166524
    ```
    
    You should see the alert in the UI.


---