You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "lostluck (via GitHub)" <gi...@apache.org> on 2023/04/04 16:22:10 UTC

[GitHub] [beam] lostluck commented on issue #21164: Ensure that the payload is consumed on Elements.Data and Elements.Timers when is_last = true

lostluck commented on issue #21164:
URL: https://github.com/apache/beam/issues/21164#issuecomment-1496261747

   Ah good catch.
   
   Java is correct: Bytes are not dropped for either [data](https://github.com/apache/beam/blob/master/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataInboundObserver.java#L154) or [timers](https://github.com/apache/beam/blob/master/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataInboundObserver.java#L188).
   
   Python remains how it was on first filing: 
   Data is always propagated, timers are only propagated if not last in the In Memory channel:
   https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/worker/data_plane.py#L363
   The GRPC channel just drops both.
   https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/worker/data_plane.py#L516
   
   I'm still unfamiliar with timers in that respect though (ask me again after I add them to Prism). I'd expect the intent behind this issue is to allow runners to avoid extra messages, which does matter for streaming. That benefit is superseded if the SDK supports the "Elements in ProcessBundle" extensions though, which if the Python SDK supports, then this issue may be obselete.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org