You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/04/29 00:29:47 UTC

[GitHub] [beam] TheNeuralBit commented on pull request #17384: [BEAM-14294] Worker changes to support trivial Batched DoFns

TheNeuralBit commented on PR #17384:
URL: https://github.com/apache/beam/pull/17384#issuecomment-1112774127

   @robertwb I think this is largely ready. I added a new `WindowedBatch` implementation, `HomogeneousWindowedBatch` which is effectively a wrapper around `WindowedValue`, where we know the value is a batch. We now enforce that batches are partitioned by `(timestamp, windows, pane_info)` and only create `HomoegeneousWindowedBatch` instances.
   
   The old implementation, `ConcreteWindowedBatch` isn't used, it's probably worth just backing that out and saving it for a future PR.
   
   Other notes: I've added support for {Timestamp, PaneInfo, Window}Param on process_batch (and we validate only these are used at construction time). Other DoFn params that were discussed in https://s.apache.org/batched-dofns are not yet supported.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org