[jira] [Commented] (BEAM-3736) Add SetUp() and TearDown() for CombineFns

["Ahmet Altay (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341068#comment-17341068 ] 

Ahmet Altay commented on BEAM-3736:
-----------------------------------

Copying the following from changes.md to as an update/summary of the above prs:

"""
Added CombineFn.setup and CombineFn.teardown to Python SDK. These methods let you initialize the CombineFn's state before any of the other methods of the CombineFn is executed and clean that state up later on. If you are using Dataflow, you need to enable Dataflow Runner V2 by passing `--experiments=use_runner_v2` before using this feature. (
"""

> Add SetUp() and TearDown() for CombineFns
> -----------------------------------------
>
>                 Key: BEAM-3736
>                 URL: https://issues.apache.org/jira/browse/BEAM-3736
>             Project: Beam
>          Issue Type: Improvement
>          Components: beam-model, sdk-py-core
>            Reporter: Chuan Yu Foo
>            Priority: P3
>          Time Spent: 13h
>  Remaining Estimate: 0h
>
> I have a CombineFn that has a large amount of state that needs to be loaded once before it can add_input or merge_combiners (for example, the CombineFn might load up a large lookup table used for combining). 
> Right now, to initialise this state, for each of the methods, I check if the state has already been initialised, and if not, I initialise it. It would be nice if CombineFn provided a SetUp() method that is called once to initialise this state (and a corresponding TearDown() method to clean up this state if necessary).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)