[jira] [Updated] (APEXCORE-119) Add Support For A New Type Of (Distributed) Operator

["Thomas Weise (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/APEXCORE-119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Weise updated APEXCORE-119:
----------------------------------
    Labels:   (was: roadmap)

> Add Support For A New Type Of (Distributed) Operator
> ----------------------------------------------------
>
>                 Key: APEXCORE-119
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-119
>             Project: Apache Apex Core
>          Issue Type: Improvement
>            Reporter: Timothy Farkas
>            Assignee: Timothy Farkas
>
> This JIRA Proposes support for a new type of distributed operator. Currently when an operator is partitioned there is no platform supported mechanism through which partitions can talk to each other. A Distributed operator would have an easy to use platform supported mechanism through which operators in a partitioning can exchange information with each other. Eventually Distributed operators would support running plain old single threaded java code transparently across partitions.
> In summary the goals would be to do the following:
> 1 - provide a platform supported fault tolerant mechanism through which operators in a partitioning can talk to each other.
> 2 - provide a platform supported way to run plain old single threaded java code accross all the partitions of a Distributed operator
> The benefits of implementing this would be huge:
> 1 - Using distributed operators we could support large in memory fault tolerant data structures (graphs, maps, arrays) in a fault tolerant way. Like Spark's RDD's but better.
> 2 - Plain old java code could be used to access and manipulate the data structures, without the user having the learn complex API's like with Spark.
> An implementation proposal and presentation are coming soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)