You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Luke Cwik (JIRA)" <ji...@apache.org> on 2017/10/05 16:56:00 UTC
[jira] [Comment Edited] (BEAM-2915) Java SDK support for portable
user state
[ https://issues.apache.org/jira/browse/BEAM-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16193187#comment-16193187 ]
Luke Cwik edited comment on BEAM-2915 at 10/5/17 4:55 PM:
----------------------------------------------------------
Support was added within the following PRs to support BagState, ValueState and CombiningState within the Java SDK harness:
* https://github.com/apache/beam/pull/3795
* https://github.com/apache/beam/pull/3788
* https://github.com/apache/beam/pull/3783
* https://github.com/apache/beam/pull/3760
* https://github.com/apache/beam/pull/3736
* https://github.com/apache/beam/pull/3724
* https://github.com/apache/beam/pull/3723
* https://github.com/apache/beam/pull/3638
The remaining features are:
* Add support for closing the BeamFnStateGrpcClientCache which should cancel outgoing calls.
* Handle outputting state which is > 2GiB by using a chunking output stream
* Move to an asynchronous persist model so that we can flush state since we currently cache all changes and only emit at the end of the process bundle call.
* Add map and set support
These items will improve performance:
* Support block level caching and prefetch
* Support readLater by issuing a prefetch
* Optimize squashing accumulators in CombiningState
was (Author: lcwik):
Support was added within the following PRs to support BagState, ValueState and CombiningState:
* https://github.com/apache/beam/pull/3795
* https://github.com/apache/beam/pull/3788
* https://github.com/apache/beam/pull/3783
* https://github.com/apache/beam/pull/3760
* https://github.com/apache/beam/pull/3736
* https://github.com/apache/beam/pull/3724
* https://github.com/apache/beam/pull/3723
* https://github.com/apache/beam/pull/3638
The remaining features are:
* Add support for closing the BeamFnStateGrpcClientCache which should cancel outgoing calls.
* Handle outputting state which is > 2GiB by using a chunking output stream
* Move to an asynchronous persist model so that we can flush state since we currently cache all changes and only emit at the end of the process bundle call.
* Add map and set support
These items will improve performance:
* Support block level caching and prefetch
* Support readLater by issuing a prefetch
* Optimize squashing accumulators in CombiningState
> Java SDK support for portable user state
> ----------------------------------------
>
> Key: BEAM-2915
> URL: https://issues.apache.org/jira/browse/BEAM-2915
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Henning Rohde
> Priority: Minor
> Labels: portability
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)