[GitHub] [beam] kennknowles opened a new issue, #19484: Turn `WindowedValue` into `T` for all the interfaces of Data Plane

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #19484:
URL: https://github.com/apache/beam/issues/19484

   The main purpose of this JIRA is Turn `WindowedValue<T\>` into `T` for all the interfaces of Data Plane.
   
   The interface definitions for sending and receiving data in the code as follows:
    - org.apache.beam.runners.fnexecution.data.FnDataService
   ```
   
   public interface FnDataService {
     <T> InboundDataClient receive(LogicalEndpoint inputLocation, Coder<WindowedValue<T>>
   coder, FnDataReceiver<WindowedValue<T>> listener);
     <T> CloseableFnDataReceiver<WindowedValue<T>>
   send(
         LogicalEndpoint outputLocation, Coder<WindowedValue<T>> coder);
   }
   
   ```
   
   
    - org.apache.beam.fn.harness.data.BeamFnDataClient
   ```
   
   public interface BeamFnDataClient {
     <T> InboundDataClient receive(ApiServiceDescriptor apiServiceDescriptor,
   LogicalEndpoint inputLocation, Coder<WindowedValue<T>> coder, FnDataReceiver<WindowedValue<T>> receiver);
   
    <T> CloseableFnDataReceiver<WindowedValue<T>> send(BeamFnDataGrpcClient Endpoints.ApiServiceDescriptor
   apiServiceDescriptor, LogicalEndpoint outputLocation, Coder<WindowedValue<T>> coder);
   }
   
   ```
   
   
   Both `Coder<WindowedValue<T\>\>` and `FnDataReceiver<WindowedValue<T\>\>` use `WindowedValue` as the data structure that both sides of Runner and SDK Harness know each other. Control Plane/Data Plane/State Plane/Logging is a highly abstraction, such as Control Plane and Logging, these are common requirements for all multi-language platforms. For example, the Flink community is also discussing how to support Python UDF, as well as how to deal with docker environment. how to data transfer, how to state access, how to logging etc. If Beam can further abstract these service interfaces, i.e., interface definitions are compatible with multiple engines, and finally provided to other projects in the form of class libraries, it definitely will help other platforms that want to support multiple languages. So we can further abstract the interface definition of FnDataService's BeamFnDataClient, take the FnDataService#receive interface as an example, and turn `WindowedValue<T\>` into `T` so th
 at other platforms can be extended arbitrarily, as follows:
   ```
   
   <T> InboundDataClient receive(LogicalEndpoint inputLocation, Coder<T> coder, FnDataReceiver<T>> listener);
   
   ```
   
   For details, see the mail discussion list:
   
   [https://lists.apache.org/list.html?dev@beam.apache.org:lte=1M:%5BDISCUSS%5D%20Turn%20%60WindowedValue](https://lists.apache.org/list.html?dev@beam.apache.org:lte=1M:%5BDISCUSS%5D%20Turn%20%60WindowedValue)
   
   Imported from Jira [BEAM-7120](https://issues.apache.org/jira/browse/BEAM-7120). Original Jira may contain additional context.
   Reported by: sunjincheng121.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik closed issue #19484: Turn `WindowedValue` into `T` for all the interfaces of Data Plane

["lukecwik (via GitHub)" <gi...@apache.org>]

lukecwik closed issue #19484: Turn `WindowedValue<T>` into `T` for all the interfaces of Data Plane
URL: https://github.com/apache/beam/issues/19484


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on issue #19484: Turn `WindowedValue` into `T` for all the interfaces of Data Plane

["lukecwik (via GitHub)" <gi...@apache.org>]

lukecwik commented on issue #19484:
URL: https://github.com/apache/beam/issues/19484#issuecomment-1401020378

   This was completed by https://github.com/apache/beam/commit/7e91a7fa227ff15d992accf0825d5383a3ad9ab5


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org