[GitHub] [metron-bro-plugin-kafka] JonZeolla commented on a change in pull request #46: METRON-2343 Bro Kafka plugin - ability to dynamically modify JSON

[GitBox <gi...@apache.org>]

JonZeolla commented on a change in pull request #46:
URL: https://github.com/apache/metron-bro-plugin-kafka/pull/46#discussion_r427518350



##########
File path: scripts/init.zeek
##########
@@ -53,6 +53,14 @@ export {
                 ["metadata.broker.list"] = "localhost:9092"
         ) &redef;
 
+##  Key value pairs that will be added to outgoing messages at the root level
+##  for example:          ["bro_server"] = "this_server_name"

Review comment:
       Let's use `zeek` in examples instead of the old `bro`

##########
File path: README.md
##########
@@ -224,6 +224,22 @@ event zeek_init() &priority=-10
 }
 ```
 
+### Example 6 - Add static values to each outgoing Kafka message

Review comment:
       The notes are specific to the prior example, so please shift this down in the README

##########
File path: scripts/init.zeek
##########
@@ -53,6 +53,14 @@ export {
                 ["metadata.broker.list"] = "localhost:9092"
         ) &redef;
 
+##  Key value pairs that will be added to outgoing messages at the root level
+##  for example:          ["bro_server"] = "this_server_name"
+##  will results in a  "bro_server":"this_server_name" field added to the outgoing

Review comment:
       Same as above comment

##########
File path: README.md
##########
@@ -224,6 +224,22 @@ event zeek_init() &priority=-10
 }
 ```
 
+### Example 6 - Add static values to each outgoing Kafka message
+It is possible to define name value pairs and have them added to each outgoing Kafka json message when tagged_json is set to true.  Each will be added to the root json object.
+    * the Kafka::additional_message_values table can be configured with each name and value
+    * based on the following configuration, each outgoing message will have "FIRST_STATIC_NAME": "FIRST_STATIC_VALUE", "SECOND_STATIC_NAME": "SECOND_STATIC_VALUE" added.
+```bash

Review comment:
       This is `zeek` not `bash`

##########
File path: scripts/init.zeek
##########
@@ -53,6 +53,14 @@ export {
                 ["metadata.broker.list"] = "localhost:9092"
         ) &redef;
 
+##  Key value pairs that will be added to outgoing messages at the root level
+##  for example:          ["bro_server"] = "this_server_name"
+##  will results in a  "bro_server":"this_server_name" field added to the outgoing
+##  json
+##  note this depends on tag_json being T
+const additional_message_values: table[string] of string = table(

Review comment:
       Can't we just shrink this to one line via `table() &redef;`?

##########
File path: scripts/init.zeek
##########
@@ -53,6 +53,14 @@ export {
                 ["metadata.broker.list"] = "localhost:9092"
         ) &redef;
 
+##  Key value pairs that will be added to outgoing messages at the root level

Review comment:
       Please align indentation with other items

##########
File path: README.md
##########
@@ -224,6 +224,22 @@ event zeek_init() &priority=-10
 }
 ```
 
+### Example 6 - Add static values to each outgoing Kafka message
+It is possible to define name value pairs and have them added to each outgoing Kafka json message when tagged_json is set to true.  Each will be added to the root json object.
+    * the Kafka::additional_message_values table can be configured with each name and value
+    * based on the following configuration, each outgoing message will have "FIRST_STATIC_NAME": "FIRST_STATIC_VALUE", "SECOND_STATIC_NAME": "SECOND_STATIC_VALUE" added.
+```bash
+@load packages
+redef Kafka::logs_to_send = set(HTTP::LOG, DNS::LOG, Conn::LOG, DPD::LOG, FTP::LOG, Files::LOG, Known::CERTS_LOG, SMTP::LOG, SSL::LOG, Weird::LOG, Notice::LOG, DHCP::LOG, SSH::LOG, Software::LOG, RADIUS::LOG, X509::LOG, RFB::LOG, Stats::LOG, CaptureLoss::LOG, SIP::LOG);
+redef Kafka::topic_name = "zeek";
+redef Kafka::tag_json = T;
+redef Kafka::kafka_conf = table(["metadata.broker.list"] = "kafka-1:9092,kafka-2:9092");
+redef Kafka::additional_message_values = table(["FIRST_STATIC_NAME"] = "FIRST_STATIC_VALUE", ["SECOND_STATIC_NAME"] = "SECOND_STATIC_VALUE");

Review comment:
       We should consider a more illustrative example, like:
   ```
   @if ( Cluster::is_enabled() )
   redef Kafka::additional_message_values = table(["CLUSTER_ENABLED"] = "True", ["NODE_ID"] = Cluster::node);
   @else
   redef Kafka::additional_message_values = table(["CLUSTER_ENABLED"] = "False");
   @endif
   ```
   
   Warning: did not test.  May need to pull `node$id` for the NODE_ID value




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org