[jira] [Commented] (METRON-1533) Create KAFKA_FIND Stellar Function

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505665#comment-16505665 ] 

ASF GitHub Bot commented on METRON-1533:
----------------------------------------

Github user james-sirota commented on the issue:

    https://github.com/apache/metron/pull/1025
  
    +1 There are minor errors in your test instructions.  CONFIG_PUT("ENRICHMENTS", e, "bro") should be CONFIG_PUT("ENRICHMENT", conf, "bro")


> Create KAFKA_FIND Stellar Function
> ----------------------------------
>
>                 Key: METRON-1533
>                 URL: https://issues.apache.org/jira/browse/METRON-1533
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Nick Allen
>            Assignee: Nick Allen
>            Priority: Minor
>
> When creating enrichments, I often find that I want to validate that the enrichment I just created was successful on the live, incoming stream of telemetry. My workflow looks something like this.
> 1. Create and test the enrichment that I want to create.
> {code:java}
> [Stellar]>>> ip_src_addr := "72.34.49.86"
> 72.34.49.86
> [Stellar]>>> geo := GEO_GET(ip_src_addr)
> {country=US, dmaCode=803, city=Los Angeles, postalCode=90014, latitude=34.0438, location_point=34.0438,-118.2512, locID=5368361, longitude=-118.2512}
> {code}
> 2. That looks good to me. Now let's add that to my Bro telemetry.
> {code:java}
> [Stellar]>>> conf := SHELL_EDIT(conf)
> {
>   "enrichment" : {
>     "fieldMap": {
>       "stellar": {
>         "config": [
>            "geo := GEO_GET(ip_src_addr)"
>         ]
>       }
>     }
>   },
>   "threatIntel": {
>   }
> }
> [Stellar]>>> CONFIG_PUT("ENRICHMENTS", e, "bro")
> {code}
>  
>  3.  It looks like that worked, but did that really work?
> At this point, I would run KAFKA_GET as many times as it takes to retrieve a Bro message. You would just have to get lucky and hope that the enrichment worked and secondly that you would pull down a Bro message (as opposed to a different sensor).
>  
> I would rather have a function that lets me only pull back the messages that I care about. In this case I could either retrieve only Bro messages.
> {code:java}
> KAFKA_FIND('indexing', m -> MAP_GET('source.type', m) == 'bro')
> {code}
> Or I could look for messages that contain geolocation data.
> {code:java}
> KAFKA_FIND('indexing', m -> MAP_EXISTS('geo.city', m))
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)