[jira] [Commented] (RYA-246) Separate Fluo query result exporters, Rya PCJ intermingled kafka triggers

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/RYA-246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150798#comment-16150798 ] 

ASF GitHub Bot commented on RYA-246:
------------------------------------

Github user meiercaleb commented on a diff in the pull request:

    https://github.com/apache/incubator-rya/pull/220#discussion_r136615988
  
    --- Diff: extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/CreateFluoPcj.java ---
    @@ -293,18 +296,22 @@ public String withRyaIntegration(
          * @param sparql - sparql query that will registered with Fluo. (not null)
          * @param strategies - ExportStrategies used to specify how final results will be handled
          * @param fluo - A connection to the Fluo application that updates the PCJ index. (not null)
    +<<<<<<< 1660b850151ea8c7dd4be7b338edfc7865411798
     <<<<<<< d47190b1ab429f1a200ac0d9a0ae07b451db5027
          * @param accumulo - Accumulo connector for connecting with Accumulo
          * @param ryaInstance - name of Rya instance to connect to
     =======
     >>>>>>> RYA-246-Query-Export-Strategy
          * @return The Fluo application's Query ID of the query that was created.
    +=======
    +     * @return FluoQuery containing the metadata for the newly registered SPARQL query
    +>>>>>>> RYA-319-Integration of Periodic Query with CLI
    --- End diff --
    
    Oof! Done.


> Separate Fluo query result exporters, Rya PCJ intermingled kafka triggers
> -------------------------------------------------------------------------
>
>                 Key: RYA-246
>                 URL: https://issues.apache.org/jira/browse/RYA-246
>             Project: Rya
>          Issue Type: Improvement
>          Components: dao
>         Environment: Fluo, Kafka, PCJ, Accumulo
>            Reporter: David W. Lotts
>            Assignee: Caleb Meier
>             Fix For: 3.2.11
>
>
> Currently the Fluo query observer framework (FQF) has a great plugin-in-ish way of doing exports.  (See field: {{exporters}} in class: {{org.apache.rya.indexing.pcj.fluo.app.observers.QueryResultObserver}} )  Exports are query results and the action taken when they occur.  Currently there are two.  Despite this awesome framework, there are three interminglings that can become issues in a production system.  Feel free to make sub tasks for each of these three tasks:
> 1. Create PCJ independently of alerts/triggers
> At query register time (with FQF), optionally create PCJ's and optionally create alerts (with Alerting) preferably atomically, so if it fails in the middle it does not leave dangling references and garbage -- currently there is one method that does everything: CreatePCJ.withRyaInteraction()
> 2. Enable exporters per query -- currently all queries will be exported on all globally enabled exporters.  When the query is registered (with FQF) specify the exporters that should be used.
> 3. Separate export observer queues per exporter -- Currently retrying a failed export observer will re-export all export types, even if some already succeeded, causing duplicate triggers.
> Also there is a need to refine the terms for java identifiers (for example: fields, classes) and documentation.  Please document here a decision about what we name different things.  The term PCJ are the index tables used by Rya to optimize it's queries.  But sometimes it is used to describe the whole Fluo framework.  Please define a unique term for the following.  The starred term will be used here, but feel free to rename them:
> - Rya query optimization PCJ  (ex: the Blabla exporter)  PCJ*
> - Kafka trigger alert notifier   (ex: the Foofoo exporter)  Alerting*
> - Fluo query observer framework  (ex: results from the Barbar)  FQF*
> Unique identifers:
> - FQF uses QueryID
> - Alerting uses TopicID 
> - PCJ uses PcjId
> Note that currently TopicID is assigned the same as QueryID, but that badly presupposes queryID's cannot be grouped onto one Topic.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)