You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Thomas Groh (JIRA)" <ji...@apache.org> on 2017/10/26 18:21:00 UTC

[jira] [Created] (BEAM-3102) Implement a BeamFnData Service

Thomas Groh created BEAM-3102:
---------------------------------

             Summary: Implement a BeamFnData Service
                 Key: BEAM-3102
                 URL: https://issues.apache.org/jira/browse/BEAM-3102
             Project: Beam
          Issue Type: New Feature
          Components: runner-core
            Reporter: Thomas Groh


The BeamFnData service provides a single API which streams elements to and from an SDK harness. There should be a Java interface for Runners to interact with this API via Java methods.

In general, the two directions should be handled as follows:
Outbound elements (e.g. from the runner's perspective, from a Runner to an SDK harness) should return some handle that consumes elements and can be completed. This is initially configured with the destination that the elements will be consumed for.

Inbound elements (e.g. from the runner's perspective, from an SDK harness to the runner) should receive a consumer of those elements (e.g. write to some durable storage, to a stream that groups elements, etc) and should produce some signal of the producer of those elements having either produced all of the elements it is capable of, or completing exceptionally (and will no longer produce additional elements)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)