You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "lostluck (via GitHub)" <gi...@apache.org> on 2023/09/19 22:18:47 UTC

[GitHub] [beam] lostluck opened a new issue, #28543: [Task][prism]: Support State API

lostluck opened a new issue, #28543:
URL: https://github.com/apache/beam/issues/28543

   ### What needs to happen?
   
   The state API is required for supporting Stateful Transforms.
   
   In particular, DoFns that use the State API are stateful transforms.
   
   The simplest way to ensure consistency with Stateful transforms is to have a single bundle process a given key at a time. This requires some additional rework and awareness of keys in the ElementManager around bundle scheduling. Further, it requires ensuring "fusion breaks" are inserted prior to the stateful DoFn, to enable prism to assure correct execution when fusion is implemented. Without fusion, all DoFns already execute with a "checkpointed" state.
   
   This also covers cleaning up hacks for current Side Input support, which leak memory.
   
   Blocker for #28187
   
   ### Issue Priority
   
   Priority: 2 (default / most normal work should be filed as P2)
   
   ### Issue Components
   
   - [ ] Component: Python SDK
   - [ ] Component: Java SDK
   - [X] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Task][prism]: Support State API [beam]

Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck closed issue #28543: [Task][prism]: Support State API 
URL: https://github.com/apache/beam/issues/28543


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org