[jira] [Commented] (METRON-1642) KafkaWriter should be able choose the topic from a field in addition to topology construction time

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530138#comment-16530138 ] 

ASF GitHub Bot commented on METRON-1642:
----------------------------------------

Github user cestella commented on a diff in the pull request:

    https://github.com/apache/metron/pull/1082#discussion_r199552106
  
    --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/kafka/KafkaWriter.java ---
    @@ -197,6 +209,15 @@ public void init(Map stormConf, TopologyContext topologyContext, WriterConfigura
         return producerConfig;
       }
     
    +  public Optional<String> getKafkaTopic(JSONObject message) {
    --- End diff --
    
    They absolutely could create loops in kafka, which wouldn't be ideal.  I don't know how I could prevent it, sadly.  I think this falls in the "with great power comes great responsibility".  Should we call this a spiderman bug (spiderbug?).


> KafkaWriter should be able choose the topic from a field in addition to topology construction time
> --------------------------------------------------------------------------------------------------
>
>                 Key: METRON-1642
>                 URL: https://issues.apache.org/jira/browse/METRON-1642
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Casey Stella
>            Priority: Major
>
> Currently, we choose the kafka topic via the kafka.topic field.  It would be useful to allow people to specify the topic via a field.  This would enable multi-stage (or chain) parsing, among other use-cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)