[jira] [Created] (METRON-1682) Allow the option for intermediate kafka topics to be removed in aggregated sensors

["Justin Leet (JIRA)" <ji...@apache.org>]

Justin Leet created METRON-1682:
-----------------------------------

             Summary: Allow the option for intermediate kafka topics to be removed in aggregated sensors
                 Key: METRON-1682
                 URL: https://issues.apache.org/jira/browse/METRON-1682
             Project: Metron
          Issue Type: Improvement
            Reporter: Justin Leet


Currently in the parser aggregation, the aggregate bolt doesn't do any clever routing for chained parsers.  It just routes messages to a Kafka topic for the sensors, even when they're already bundled into the aggregated bolt.

An option to skip the intermediate Kafka topic should be added.  This should preferably be optional on a per sensor basic, e.g. If parser A produces documents B,C, and D, we may choose to have B and C skip intermediate Kafka and D use Kafka.  This is useful in cases where, for whatever reason, we choose to have parser D be in it's own topology (i.e. parsers A,B,C are in an aggregated topology, and D is in a single topology.  Even though parser A produces all document B,C, and D).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)