[jira] [Created] (METRON-1010) Reorganize the bro elasticsearch template

["Jon Zeolla (JIRA)" <ji...@apache.org>]

Jon Zeolla created METRON-1010:
----------------------------------

             Summary: Reorganize the bro elasticsearch template
                 Key: METRON-1010
                 URL: https://issues.apache.org/jira/browse/METRON-1010
             Project: Metron
          Issue Type: Improvement
    Affects Versions: Next + 1
            Reporter: Jon Zeolla


Right now, updates to the bro indexing template for ElasticsSearch are somewhat confusing due to field name collisions across distinct bro logs.  I see two possible approaches to make this simpler:

*Option 1* - One template, with duplication, but still one bro index.
We duplicate the field definitions under each log type's section (distinguished by comments) to make it easier to add/remove bro log support to the template, and makes ripping logs out into distinct indexes in the future easier.

Pros:  Doesn't require much refactoring of Metron because all bro logs are still in the same place that they used to be, review of one bro log's indexing details is more intuitive.
Cons:  Changes to a field should be reflected everywhere that field exists in the template.

*Option 2* - Multiple templates, multiple bro indexes.
Configure Metron to send each individual bro log into distinct indexes.  We could continue to use the bro- preface, but we would still need to fix dashboards, saved queries, etc.

Pros:  1:1 mapping of a distinct field to an ES type, so type is always accurate (unlike what we have currently, for details see https://github.com/apache/metron/pull/586/files#diff-262becd0bb95e0520c42c30a857a343eR131).
Cons:  Overall complexity of change.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)