[jira] [Created] (METRON-56) Create unified enrichment topology

["Ryan Merriman (JIRA)" <ji...@apache.org>]

Ryan Merriman created METRON-56:
-----------------------------------

             Summary: Create unified enrichment topology
                 Key: METRON-56
                 URL: https://issues.apache.org/jira/browse/METRON-56
             Project: Metron
          Issue Type: Improvement
            Reporter: Ryan Merriman
            Assignee: Ryan Merriman


Currently each sensor requires it's own end-to-end topology.  This improvement will separate parsing and enrichment into different topologies.  There will be separate parsing topologies for each sensor that read from a sensor Kafka topic, parse messages and write back out to a common enrichment Kafka topic.  The enrichment topology will read from the common enrichment topic and enrich messages based on sensor metadata.

This approach offers several benefits:
- Will require fewer Storm workers
- Parser topologies can be added and removed without affecting the enrichment topology
- Configuration will be less verbose and more intuitive



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)