[GitHub] [beam] robertwb commented on a change in pull request #11235: [WIP] Pull bundle descriptors.

[GitBox <gi...@apache.org>]

robertwb commented on a change in pull request #11235: [WIP] Pull bundle descriptors.
URL: https://github.com/apache/beam/pull/11235#discussion_r398805859
 
 

 ##########
 File path: model/fn-execution/src/main/proto/beam_fn_api.proto
 ##########
 @@ -81,6 +81,13 @@ service BeamFnControl {
       returns (
           // A stream of instructions requested of the SDK to be performed.
           stream InstructionRequest) {}
+
+  rpc GetProcessBundleDescriptor(GetProcessBundleDescriptorRequest) returns (
 
 Review comment:
   The current Control must be a stream because (1) it is server-initiated requests on a client-initiated connection and (2) in streaming pipelines, it we want it to be high-volume, low-latency, possibly pipelined. However, using streams in this way vs. "standard" RPC methods comes at a cost (in both code and conceptual complexity). 
   
   As far as ordering, it is preferable to have a protocol that is not sensitive to this kind of thing. (In fact, this eliminates the one need for ordering in our Control stream, where bundles needed to be registered before they were used (and this didn't fit well with pattern of dispatching instructions to threadpool-backed handlers in the SDKs which required extra locking/waiting anyway). Progress and splitting requests were already asynchronous.)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services