You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Robert Burke (Jira)" <ji...@apache.org> on 2019/09/23 18:30:00 UTC
[jira] [Comment Edited] (BEAM-8272) GroupIntoBatches transform for
Go SDK
[ https://issues.apache.org/jira/browse/BEAM-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936099#comment-16936099 ]
Robert Burke edited comment on BEAM-8272 at 9/23/19 6:29 PM:
-------------------------------------------------------------
Note that the implementation will necessarily be different in the Go SDK. The SDK doesn't yet support the State and Timers API, which both the Java and Python implementations use. Adding state and timers to the Go SDK is a larger task.
Though, this looks like a largely streaming construct, which makes alternative implementations without State and Timers tricky, if not impossible.
It also looks like it requires being able to emit "Iterables" which might be handle-able with slices instead, but otherwise the SDK doesn't yet support user side streams.
was (Author: lostluck):
Note that the implementation will necessarily be different in the Go SDK. The SDK doesn't yet support the State and Timers API, which both the Java and Python implementations use.
> GroupIntoBatches transform for Go SDK
> -------------------------------------
>
> Key: BEAM-8272
> URL: https://issues.apache.org/jira/browse/BEAM-8272
> Project: Beam
> Issue Type: New Feature
> Components: sdk-go
> Reporter: John Patoch
> Priority: Major
>
> Add a PTransform that batches inputs to a desired batch size. Batches will contain only elements of a single key.
> It should offer the same API as its Java counterpart:
> [https://github.com/apache/beam/blob/11a977b8b26eff2274d706541127c19dc93131a2/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java]
>
> And Python counterpart:
> https://github.com/apache/beam/blob/c445fdfdfab4a191aa780210564199f2873f85d8/sdks/python/apache_beam/transforms/util.py#L684
--
This message was sent by Atlassian Jira
(v8.3.4#803005)