You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ignite.apache.org by "Romain Manni-Bucau (Jira)" <ji...@apache.org> on 2019/09/24 17:00:00 UTC

[jira] [Created] (IGNITE-12228) Implement an Apache Beam runner on top of Ignite compute grid?

Romain Manni-Bucau created IGNITE-12228:
-------------------------------------------

             Summary: Implement an Apache Beam runner on top of Ignite compute grid?
                 Key: IGNITE-12228
                 URL: https://issues.apache.org/jira/browse/IGNITE-12228
             Project: Ignite
          Issue Type: Wish
            Reporter: Romain Manni-Bucau


Apache Ignite provides a compute grid.

It is therefore feasible to use that to execute a DAG pipeline.

Therefore implementing a Apache Beam runner executing a DAG on an ignite cluster would be very valuable for users and would provide a very interesting alternative to Spark, Dataflow etc...which would be very valuable when using data from ignite itself. 

Hoping it can help, here is Hazelcast Jet implementation which is not that far even if Jet is a bit more advanced in term of DAG support: [https://github.com/apache/beam/tree/master/runners/jet-experimental/src/main/java/org/apache/beam/runners/jet]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)