[jira] [Created] (RYA-246) Separate Fluo query result exporters, Rya PCJ intermingled kafka triggers

["David W. Lotts (JIRA)" <ji...@apache.org>]

David W. Lotts created RYA-246:
----------------------------------

             Summary: Separate Fluo query result exporters, Rya PCJ intermingled kafka triggers
                 Key: RYA-246
                 URL: https://issues.apache.org/jira/browse/RYA-246
             Project: Rya
          Issue Type: Improvement
          Components: dao
         Environment: Fluo, Kafka, PCJ, Accumulo
            Reporter: David W. Lotts


Currently the Fluo query observer framework (FQF) has a great plugin-in-ish way of doing exports.  (See field: {{exporters}} in class: {{org.apache.rya.indexing.pcj.fluo.app.observers.QueryResultObserver}} )  Exports are query results and the action taken when they occur.  Currently there are two.  Despite this awsome framework, there are three interminglings that can become issues in a production system:

1. Create PCJ independently of alerts/triggers
At query register time (with FQF), optionally create PCJ's and optionally create alerts (with Alerting) preferably atomically, so if it fails in the middle it does not leave dangling references and garbage -- currently there is one method that does everything: CreatePCJ.withRyaInteraction()

2. Enable exporters per query -- currently all queries will be exported on all globally enabled exporters

3. Separate export observer queues per exporter -- Currently retrying a failed export observer will re-export all export types, even if some already succeeded, causing duplicate triggers.


Also there is a need to refine the terms for java identifiers (for example: fields, classes) and documentation.  Please document here a decision about what we name different things.  The term PCJ are the index tables used by Rya to optimize it's queries.  But sometimes it is used to describe the whole Fluo framework.  Please define a unique term for the following.  The starred term will be used here, but feel free to rename them:

- Rya query optimization PCJ  (ex: the Blabla exporter)  PCJ*
- Kafka trigger alert notifier   (ex: the Foofoo exporter)  Alerting*
- Fluo query observer framework  (ex: results from the Barbar)  FQF*

Unique identifers:

- FQF uses QueryID
- Alerting uses TopicID 
- PCJ uses PcjId

Note that currently TopicID is assigned the same as QueryID, but that badly presupposes queryID's cannot be grouped onto one Topic.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)