[jira] [Updated] (APEXCORE-3) Ability for an operator to populate DAG at launch time

["Thomas Weise (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/APEXCORE-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Weise updated APEXCORE-3:
--------------------------------
    Fix Version/s: 3.3.0

> Ability for an operator to populate DAG at launch time
> ------------------------------------------------------
>
>                 Key: APEXCORE-3
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-3
>             Project: Apache Apex Core
>          Issue Type: New Feature
>            Reporter: Amol Kekre
>            Assignee: Vlad Rozov
>            Priority: Critical
>             Fix For: 3.3.0
>
>
> Apex should have an operator API that lets the operator generate DAG during launch time. This will mean the following
> - Logical DAG will have one operator. This is the operator that will generate a DAG underneath
> - Physical plan will have the DAG generated by the operator
> - Execution plan will mimic physical plan + container location etc.
> For example lets say we have three operators in a DAG (app) A->B->C
> B during launch time generates a DAG B1->B2->B3, then the physical plan will be
> A->B1->B2->B3->C
> This should work irrespective of number of ports, etc. A typical flattening. The operators inside of B (B1, B2, B3) should have properties and attributes just as any. Users should be able to access these at run time and compile time. B itself should support properties and attributes that B1, B2, B3 can inherit from.
> This is a very critical feature as it will open up users to plug-in their own engines and still take up complete operability support from Apex engine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)