[jira] [Commented] (BEAM-7120) Turn `WindowedValue` into `T` for all the interfaces of Data Plane

["Beam JIRA Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17174735#comment-17174735 ] 

Beam JIRA Bot commented on BEAM-7120:
-------------------------------------

This issue is P2 but has been unassigned without any comment for 60 days so it has been labeled "stale-P2". If this issue is still affecting you, we care! Please comment and remove the label. Otherwise, in 14 days the issue will be moved to P3.

Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed explanation of what these priorities mean.


> Turn `WindowedValue<T>` into `T` for all the interfaces of Data Plane
> ---------------------------------------------------------------------
>
>                 Key: BEAM-7120
>                 URL: https://issues.apache.org/jira/browse/BEAM-7120
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-core, sdk-java-harness, sdk-py-harness
>            Reporter: sunjincheng
>            Priority: P2
>              Labels: stale-P2
>
> The main purpose of this JIRA is Turn `WindowedValue<T>` into `T` for all the interfaces of Data Plane.
> The interface definitions for sending and receiving data in the code as follows:
>  - org.apache.beam.runners.fnexecution.data.FnDataService
> {code:java}
> public interface FnDataService {
>   <T> InboundDataClient receive(LogicalEndpoint inputLocation, Coder<WindowedValue<T>> coder, FnDataReceiver<WindowedValue<T>> listener);
>   <T> CloseableFnDataReceiver<WindowedValue<T>> send(
>       LogicalEndpoint outputLocation, Coder<WindowedValue<T>> coder);
> }
> {code}
>  - org.apache.beam.fn.harness.data.BeamFnDataClient
> {code:java}
> public interface BeamFnDataClient {
>   <T> InboundDataClient receive(ApiServiceDescriptor apiServiceDescriptor, LogicalEndpoint inputLocation, Coder<WindowedValue<T>> coder, FnDataReceiver<WindowedValue<T>> receiver);
>   <T> CloseableFnDataReceiver<WindowedValue<T>> send(BeamFnDataGrpcClient Endpoints.ApiServiceDescriptor apiServiceDescriptor, LogicalEndpoint outputLocation, Coder<WindowedValue<T>> coder);
> }
> {code}
> Both `Coder<WindowedValue<T>>` and `FnDataReceiver<WindowedValue<T>>` use `WindowedValue` as the data structure that both sides of Runner and SDK Harness know each other. Control Plane/Data Plane/State Plane/Logging is a highly abstraction, such as Control Plane and Logging, these are common requirements for all multi-language platforms. For example, the Flink community is also discussing how to support Python UDF, as well as how to deal with docker environment. how to data transfer, how to state access, how to logging etc. If Beam can further abstract these service interfaces, i.e., interface definitions are compatible with multiple engines, and finally provided to other projects in the form of class libraries, it definitely will help other platforms that want to support multiple languages. So we can further abstract the interface definition of FnDataService's BeamFnDataClient, take the FnDataService#receive interface as an example, and turn `WindowedValue<T>` into `T` so that other platforms can be extended arbitrarily, as follows:
> {code:java}
> <T> InboundDataClient receive(LogicalEndpoint inputLocation, Coder<T> coder, FnDataReceiver<T>> listener);
> {code}
> For details, see the mail discussion list:
> [https://lists.apache.org/list.html?dev@beam.apache.org:lte=1M:%5BDISCUSS%5D%20Turn%20%60WindowedValue]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)