[jira] [Created] (METRON-750) Add new functionality to ParserWriter related classes for batchTimeout flushing

["Matt Foley (JIRA)" <ji...@apache.org>]

Matt Foley created METRON-750:
---------------------------------

             Summary: Add new functionality to ParserWriter related classes for batchTimeout flushing
                 Key: METRON-750
                 URL: https://issues.apache.org/jira/browse/METRON-750
             Project: Metron
          Issue Type: Sub-task
            Reporter: Ajay Yadav
            Assignee: Matt Foley


Add new functionality to BulkMessageWriter and its related classes for batchTimeout flushing.
This primarily affects:
* BulkMessageWriterBolt
* BulkWriterComponent
* IndexingConfigurations
* and newly added utilities in BatchTimeoutHelper

All of the other changes (from METRON-516) are basically boilerplate, just laying "getBatchTimeout()" facade methods alongside existing "getBatchSize()" facade methods. (There's a lot of layers!)

The flush-on-timeout logic is fairly straightforward. It was implemented by a refactoring of BulkWriterComponent and basic Tick Tuple reception in BulkMessageWriterBolt.

The tricky part was figuring out the appropriate setting for 'topology.tick.tuple.freq.secs' if the administrator configures non-default batchTimeouts. It is necessary to enumerate the batchTimeout settings for all configured sensorNames, which is implemented in IndexingConfigurations$getAllConfiguredTimeouts(). Then multiple other factors must be taken into account to determine the allowed and recommended settings, which is implemented in BatchTimeoutHelper. If there are better ways to accomplish these things, please share your ideas.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)