[jira] [Commented] (BEAM-205) An 'in process' runner optimized for efficiency rather than debugging

["Luke Cwik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246065#comment-15246065 ] 

Luke Cwik commented on BEAM-205:
--------------------------------

>From my understanding, both Flink and Spark support running locally.

How is this different from just using a runner that supports running locally?

> An 'in process' runner optimized for efficiency rather than debugging
> ---------------------------------------------------------------------
>
>                 Key: BEAM-205
>                 URL: https://issues.apache.org/jira/browse/BEAM-205
>             Project: Beam
>          Issue Type: New Feature
>          Components: runner-direct
>            Reporter: Mark Shields
>            Assignee: Davor Bonaci
>
> IoT applications often want to distribute processing between the device and the cloud. Device will often massage data, filter, window or batch for communication efficiency, etc.
> During development one could imagine wanting to refactor processing between the device and the cloud. So could imagine having both sides in BEAM would be compelling.
> This bug is to explore an 'in process' runner lite. Ie retain all the streaming support, but disable anything which burns cpu/mem only for the purposes of acid testing pipeline wrt serialiazability, concurrency safety, order assumptions, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)