[jira] [Created] (RYA-359) Update PeriodicNotificationApplication to scale beyond one YARN container.

["Jeff Dasch (JIRA)" <ji...@apache.org>]

Jeff Dasch created RYA-359:
------------------------------

             Summary: Update PeriodicNotificationApplication to scale beyond one YARN container.
                 Key: RYA-359
                 URL: https://issues.apache.org/jira/browse/RYA-359
             Project: Rya
          Issue Type: Improvement
            Reporter: Jeff Dasch
            Priority: Minor


In theory, this can be achieved in two different ways:
1) Update code to support concurrent PeriodicNotificationApplications in separate YARN containers that split the workload for a single rya instance.
2) Update code to support the various Runnables executing in separate YARN containers.  This would likely mean replacing the blocking queue being used with serialization and kafka, or possibly adopting the kafka streams library.

The second option has the benefit that it allows for finer grained control of concurrency, though it does introduce an added serialization overhead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)