You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Charles Chen <cc...@google.com> on 2018/06/06 00:26:00 UTC
Re: Proposal for Beam Python User State and Timer APIs
Thanks everyone for contributing here. We've reached rough consensus on
the approach we should take with this API, and I've summarized this in the
new "Community consensus" sections I added to the doc (
https://s.apache.org/beam-python-user-state-and-timers). I will begin
initial implementation of this API soon.
On Wed, May 23, 2018 at 8:08 PM Thomas Weise <th...@apache.org> wrote:
> Nice proposal; it's exciting to see this about to be added to the SDK as
> it enables a set of more complex use cases.
>
> I also think that some of the content can later be repurposed as user
> documentation.
>
> Thanks,
> Thomas
>
>
> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <cc...@google.com> wrote:
>
>> Thanks everyone for the detailed comments and discussions. It looks like
>> by now, we mostly agree with the requirements and overall direction needed
>> for the API, though there is continuing discussion on specific details. I
>> want to highlight two new sections of the doc, which address some
>> discussions that have come up:
>>
>> - *Existing state and transactionality*: this section addresses how
>> we will address an existing transactionality inconsistency in the existing
>> Java API. (
>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b
>> )
>> - *State for merging windows*: this section addresses how we will
>> deal with non-combinable state in conjunction with merging windows. (
>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy
>> )
>>
>> Let me know any further comments and suggestions.
>>
>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <kl...@google.com> wrote:
>>
>>> Nice. I know that Java users have found it helpful to have this
>>> lower-level way of writing pipelines when the high-level primitives don't
>>> quite have the tight control they are looking for. I hope it will be a big
>>> draw for Python, too.
>>>
>>> (commenting on the doc)
>>>
>>> Kenn
>>>
>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <cc...@google.com> wrote:
>>>
>>>> I want to share a proposal for adding user state and timer support to
>>>> the Beam Python SDK and get the community's thoughts on how such an API
>>>> should look: https://s.apache.org/beam-python-user-state-and-timers
>>>>
>>>> Let me know what you think and please add any comments and suggestions
>>>> you may have.
>>>>
>>>> Best,
>>>> Charles
>>>>
>>>
>
***UNCHECKED*** Re: Proposal for Beam Python User State and Timer
APIs
Posted by Robert Bradshaw <ro...@google.com>.
And its implementation of the Fn API is on it's way too:
https://github.com/apache/beam/pull/6349
https://github.com/apache/beam/pull/6433
On Tue, Sep 18, 2018 at 6:56 PM Charles Chen <cc...@google.com> wrote:
> An update: the reference DirectRunner implementation of (and common
> execution code for) the Python user state and timers API has been merged:
> https://github.com/apache/beam/pull/6304
>
> On Thu, Aug 30, 2018 at 1:48 AM Charles Chen <cc...@google.com> wrote:
>
>> Another update: the reference DirectRunner implementation of the Python
>> user state and timers API is out for review:
>> https://github.com/apache/beam/pull/6304
>>
>> On Mon, Jul 9, 2018 at 2:18 PM Charles Chen <cc...@google.com> wrote:
>>
>>> An update: https://github.com/apache/beam/pull/5691 has been merged. I
>>> hope to send out a reference implementation in the DirectRunner soon. On
>>> the roadmap after that is work on the relevant portability interfaces here
>>> so we can get this working on runners like Beam Python on Flink.
>>>
>>> On Wed, Jun 20, 2018 at 10:00 AM Charles Chen <cc...@google.com> wrote:
>>>
>>>> An update on the implementation: I recently sent out the user-facing
>>>> pipeline construction part of the API implementation out for review:
>>>> https://github.com/apache/beam/pull/5691.
>>>>
>>>> On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <cc...@google.com> wrote:
>>>>
>>>>> Thanks everyone for contributing here. We've reached rough consensus
>>>>> on the approach we should take with this API, and I've summarized this in
>>>>> the new "Community consensus" sections I added to the doc (
>>>>> https://s.apache.org/beam-python-user-state-and-timers). I will
>>>>> begin initial implementation of this API soon.
>>>>>
>>>>> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <th...@apache.org> wrote:
>>>>>
>>>>>> Nice proposal; it's exciting to see this about to be added to the SDK
>>>>>> as it enables a set of more complex use cases.
>>>>>>
>>>>>> I also think that some of the content can later be repurposed as user
>>>>>> documentation.
>>>>>>
>>>>>> Thanks,
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <cc...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks everyone for the detailed comments and discussions. It looks
>>>>>>> like by now, we mostly agree with the requirements and overall direction
>>>>>>> needed for the API, though there is continuing discussion on specific
>>>>>>> details. I want to highlight two new sections of the doc, which address
>>>>>>> some discussions that have come up:
>>>>>>>
>>>>>>> - *Existing state and transactionality*: this section addresses
>>>>>>> how we will address an existing transactionality inconsistency in the
>>>>>>> existing Java API. (
>>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b
>>>>>>> )
>>>>>>> - *State for merging windows*: this section addresses how we
>>>>>>> will deal with non-combinable state in conjunction with merging windows. (
>>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy
>>>>>>> )
>>>>>>>
>>>>>>> Let me know any further comments and suggestions.
>>>>>>>
>>>>>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <kl...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Nice. I know that Java users have found it helpful to have this
>>>>>>>> lower-level way of writing pipelines when the high-level primitives don't
>>>>>>>> quite have the tight control they are looking for. I hope it will be a big
>>>>>>>> draw for Python, too.
>>>>>>>>
>>>>>>>> (commenting on the doc)
>>>>>>>>
>>>>>>>> Kenn
>>>>>>>>
>>>>>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <cc...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I want to share a proposal for adding user state and timer support
>>>>>>>>> to the Beam Python SDK and get the community's thoughts on how such an API
>>>>>>>>> should look:
>>>>>>>>> https://s.apache.org/beam-python-user-state-and-timers
>>>>>>>>>
>>>>>>>>> Let me know what you think and please add any comments and
>>>>>>>>> suggestions you may have.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Charles
>>>>>>>>>
>>>>>>>>
>>>>>>
Re: Proposal for Beam Python User State and Timer APIs
Posted by Pablo Estrada <pa...@google.com>.
Very cool! Thanks Charles!
On Tue, Sep 18, 2018, 9:56 AM Charles Chen <cc...@google.com> wrote:
> An update: the reference DirectRunner implementation of (and common
> execution code for) the Python user state and timers API has been merged:
> https://github.com/apache/beam/pull/6304
>
> On Thu, Aug 30, 2018 at 1:48 AM Charles Chen <cc...@google.com> wrote:
>
>> Another update: the reference DirectRunner implementation of the Python
>> user state and timers API is out for review:
>> https://github.com/apache/beam/pull/6304
>>
>> On Mon, Jul 9, 2018 at 2:18 PM Charles Chen <cc...@google.com> wrote:
>>
>>> An update: https://github.com/apache/beam/pull/5691 has been merged. I
>>> hope to send out a reference implementation in the DirectRunner soon. On
>>> the roadmap after that is work on the relevant portability interfaces here
>>> so we can get this working on runners like Beam Python on Flink.
>>>
>>> On Wed, Jun 20, 2018 at 10:00 AM Charles Chen <cc...@google.com> wrote:
>>>
>>>> An update on the implementation: I recently sent out the user-facing
>>>> pipeline construction part of the API implementation out for review:
>>>> https://github.com/apache/beam/pull/5691.
>>>>
>>>> On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <cc...@google.com> wrote:
>>>>
>>>>> Thanks everyone for contributing here. We've reached rough consensus
>>>>> on the approach we should take with this API, and I've summarized this in
>>>>> the new "Community consensus" sections I added to the doc (
>>>>> https://s.apache.org/beam-python-user-state-and-timers). I will
>>>>> begin initial implementation of this API soon.
>>>>>
>>>>> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <th...@apache.org> wrote:
>>>>>
>>>>>> Nice proposal; it's exciting to see this about to be added to the SDK
>>>>>> as it enables a set of more complex use cases.
>>>>>>
>>>>>> I also think that some of the content can later be repurposed as user
>>>>>> documentation.
>>>>>>
>>>>>> Thanks,
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <cc...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks everyone for the detailed comments and discussions. It looks
>>>>>>> like by now, we mostly agree with the requirements and overall direction
>>>>>>> needed for the API, though there is continuing discussion on specific
>>>>>>> details. I want to highlight two new sections of the doc, which address
>>>>>>> some discussions that have come up:
>>>>>>>
>>>>>>> - *Existing state and transactionality*: this section addresses
>>>>>>> how we will address an existing transactionality inconsistency in the
>>>>>>> existing Java API. (
>>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b
>>>>>>> )
>>>>>>> - *State for merging windows*: this section addresses how we
>>>>>>> will deal with non-combinable state in conjunction with merging windows. (
>>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy
>>>>>>> )
>>>>>>>
>>>>>>> Let me know any further comments and suggestions.
>>>>>>>
>>>>>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <kl...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Nice. I know that Java users have found it helpful to have this
>>>>>>>> lower-level way of writing pipelines when the high-level primitives don't
>>>>>>>> quite have the tight control they are looking for. I hope it will be a big
>>>>>>>> draw for Python, too.
>>>>>>>>
>>>>>>>> (commenting on the doc)
>>>>>>>>
>>>>>>>> Kenn
>>>>>>>>
>>>>>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <cc...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I want to share a proposal for adding user state and timer support
>>>>>>>>> to the Beam Python SDK and get the community's thoughts on how such an API
>>>>>>>>> should look:
>>>>>>>>> https://s.apache.org/beam-python-user-state-and-timers
>>>>>>>>>
>>>>>>>>> Let me know what you think and please add any comments and
>>>>>>>>> suggestions you may have.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Charles
>>>>>>>>>
>>>>>>>>
>>>>>>
Re: Proposal for Beam Python User State and Timer APIs
Posted by Charles Chen <cc...@google.com>.
An update: the reference DirectRunner implementation of (and common
execution code for) the Python user state and timers API has been merged:
https://github.com/apache/beam/pull/6304
On Thu, Aug 30, 2018 at 1:48 AM Charles Chen <cc...@google.com> wrote:
> Another update: the reference DirectRunner implementation of the Python
> user state and timers API is out for review:
> https://github.com/apache/beam/pull/6304
>
> On Mon, Jul 9, 2018 at 2:18 PM Charles Chen <cc...@google.com> wrote:
>
>> An update: https://github.com/apache/beam/pull/5691 has been merged. I
>> hope to send out a reference implementation in the DirectRunner soon. On
>> the roadmap after that is work on the relevant portability interfaces here
>> so we can get this working on runners like Beam Python on Flink.
>>
>> On Wed, Jun 20, 2018 at 10:00 AM Charles Chen <cc...@google.com> wrote:
>>
>>> An update on the implementation: I recently sent out the user-facing
>>> pipeline construction part of the API implementation out for review:
>>> https://github.com/apache/beam/pull/5691.
>>>
>>> On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <cc...@google.com> wrote:
>>>
>>>> Thanks everyone for contributing here. We've reached rough consensus
>>>> on the approach we should take with this API, and I've summarized this in
>>>> the new "Community consensus" sections I added to the doc (
>>>> https://s.apache.org/beam-python-user-state-and-timers). I will begin
>>>> initial implementation of this API soon.
>>>>
>>>> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <th...@apache.org> wrote:
>>>>
>>>>> Nice proposal; it's exciting to see this about to be added to the SDK
>>>>> as it enables a set of more complex use cases.
>>>>>
>>>>> I also think that some of the content can later be repurposed as user
>>>>> documentation.
>>>>>
>>>>> Thanks,
>>>>> Thomas
>>>>>
>>>>>
>>>>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <cc...@google.com> wrote:
>>>>>
>>>>>> Thanks everyone for the detailed comments and discussions. It looks
>>>>>> like by now, we mostly agree with the requirements and overall direction
>>>>>> needed for the API, though there is continuing discussion on specific
>>>>>> details. I want to highlight two new sections of the doc, which address
>>>>>> some discussions that have come up:
>>>>>>
>>>>>> - *Existing state and transactionality*: this section addresses
>>>>>> how we will address an existing transactionality inconsistency in the
>>>>>> existing Java API. (
>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b
>>>>>> )
>>>>>> - *State for merging windows*: this section addresses how we will
>>>>>> deal with non-combinable state in conjunction with merging windows. (
>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy
>>>>>> )
>>>>>>
>>>>>> Let me know any further comments and suggestions.
>>>>>>
>>>>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <kl...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Nice. I know that Java users have found it helpful to have this
>>>>>>> lower-level way of writing pipelines when the high-level primitives don't
>>>>>>> quite have the tight control they are looking for. I hope it will be a big
>>>>>>> draw for Python, too.
>>>>>>>
>>>>>>> (commenting on the doc)
>>>>>>>
>>>>>>> Kenn
>>>>>>>
>>>>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <cc...@google.com> wrote:
>>>>>>>
>>>>>>>> I want to share a proposal for adding user state and timer support
>>>>>>>> to the Beam Python SDK and get the community's thoughts on how such an API
>>>>>>>> should look: https://s.apache.org/beam-python-user-state-and-timers
>>>>>>>>
>>>>>>>> Let me know what you think and please add any comments and
>>>>>>>> suggestions you may have.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Charles
>>>>>>>>
>>>>>>>
>>>>>
Re: Proposal for Beam Python User State and Timer APIs
Posted by Charles Chen <cc...@google.com>.
Another update: the reference DirectRunner implementation of the Python
user state and timers API is out for review:
https://github.com/apache/beam/pull/6304
On Mon, Jul 9, 2018 at 2:18 PM Charles Chen <cc...@google.com> wrote:
> An update: https://github.com/apache/beam/pull/5691 has been merged. I
> hope to send out a reference implementation in the DirectRunner soon. On
> the roadmap after that is work on the relevant portability interfaces here
> so we can get this working on runners like Beam Python on Flink.
>
> On Wed, Jun 20, 2018 at 10:00 AM Charles Chen <cc...@google.com> wrote:
>
>> An update on the implementation: I recently sent out the user-facing
>> pipeline construction part of the API implementation out for review:
>> https://github.com/apache/beam/pull/5691.
>>
>> On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <cc...@google.com> wrote:
>>
>>> Thanks everyone for contributing here. We've reached rough consensus on
>>> the approach we should take with this API, and I've summarized this in the
>>> new "Community consensus" sections I added to the doc (
>>> https://s.apache.org/beam-python-user-state-and-timers). I will begin
>>> initial implementation of this API soon.
>>>
>>> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <th...@apache.org> wrote:
>>>
>>>> Nice proposal; it's exciting to see this about to be added to the SDK
>>>> as it enables a set of more complex use cases.
>>>>
>>>> I also think that some of the content can later be repurposed as user
>>>> documentation.
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>>
>>>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <cc...@google.com> wrote:
>>>>
>>>>> Thanks everyone for the detailed comments and discussions. It looks
>>>>> like by now, we mostly agree with the requirements and overall direction
>>>>> needed for the API, though there is continuing discussion on specific
>>>>> details. I want to highlight two new sections of the doc, which address
>>>>> some discussions that have come up:
>>>>>
>>>>> - *Existing state and transactionality*: this section addresses
>>>>> how we will address an existing transactionality inconsistency in the
>>>>> existing Java API. (
>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b
>>>>> )
>>>>> - *State for merging windows*: this section addresses how we will
>>>>> deal with non-combinable state in conjunction with merging windows. (
>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy
>>>>> )
>>>>>
>>>>> Let me know any further comments and suggestions.
>>>>>
>>>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <kl...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Nice. I know that Java users have found it helpful to have this
>>>>>> lower-level way of writing pipelines when the high-level primitives don't
>>>>>> quite have the tight control they are looking for. I hope it will be a big
>>>>>> draw for Python, too.
>>>>>>
>>>>>> (commenting on the doc)
>>>>>>
>>>>>> Kenn
>>>>>>
>>>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <cc...@google.com> wrote:
>>>>>>
>>>>>>> I want to share a proposal for adding user state and timer support
>>>>>>> to the Beam Python SDK and get the community's thoughts on how such an API
>>>>>>> should look: https://s.apache.org/beam-python-user-state-and-timers
>>>>>>>
>>>>>>> Let me know what you think and please add any comments and
>>>>>>> suggestions you may have.
>>>>>>>
>>>>>>> Best,
>>>>>>> Charles
>>>>>>>
>>>>>>
>>>>
Re: Proposal for Beam Python User State and Timer APIs
Posted by Charles Chen <cc...@google.com>.
An update: https://github.com/apache/beam/pull/5691 has been merged. I
hope to send out a reference implementation in the DirectRunner soon. On
the roadmap after that is work on the relevant portability interfaces here
so we can get this working on runners like Beam Python on Flink.
On Wed, Jun 20, 2018 at 10:00 AM Charles Chen <cc...@google.com> wrote:
> An update on the implementation: I recently sent out the user-facing
> pipeline construction part of the API implementation out for review:
> https://github.com/apache/beam/pull/5691.
>
> On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <cc...@google.com> wrote:
>
>> Thanks everyone for contributing here. We've reached rough consensus on
>> the approach we should take with this API, and I've summarized this in the
>> new "Community consensus" sections I added to the doc (
>> https://s.apache.org/beam-python-user-state-and-timers). I will begin
>> initial implementation of this API soon.
>>
>> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <th...@apache.org> wrote:
>>
>>> Nice proposal; it's exciting to see this about to be added to the SDK as
>>> it enables a set of more complex use cases.
>>>
>>> I also think that some of the content can later be repurposed as user
>>> documentation.
>>>
>>> Thanks,
>>> Thomas
>>>
>>>
>>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <cc...@google.com> wrote:
>>>
>>>> Thanks everyone for the detailed comments and discussions. It looks
>>>> like by now, we mostly agree with the requirements and overall direction
>>>> needed for the API, though there is continuing discussion on specific
>>>> details. I want to highlight two new sections of the doc, which address
>>>> some discussions that have come up:
>>>>
>>>> - *Existing state and transactionality*: this section addresses how
>>>> we will address an existing transactionality inconsistency in the existing
>>>> Java API. (
>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b
>>>> )
>>>> - *State for merging windows*: this section addresses how we will
>>>> deal with non-combinable state in conjunction with merging windows. (
>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy
>>>> )
>>>>
>>>> Let me know any further comments and suggestions.
>>>>
>>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <kl...@google.com> wrote:
>>>>
>>>>> Nice. I know that Java users have found it helpful to have this
>>>>> lower-level way of writing pipelines when the high-level primitives don't
>>>>> quite have the tight control they are looking for. I hope it will be a big
>>>>> draw for Python, too.
>>>>>
>>>>> (commenting on the doc)
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <cc...@google.com> wrote:
>>>>>
>>>>>> I want to share a proposal for adding user state and timer support to
>>>>>> the Beam Python SDK and get the community's thoughts on how such an API
>>>>>> should look: https://s.apache.org/beam-python-user-state-and-timers
>>>>>>
>>>>>> Let me know what you think and please add any comments and
>>>>>> suggestions you may have.
>>>>>>
>>>>>> Best,
>>>>>> Charles
>>>>>>
>>>>>
>>>
Re: Proposal for Beam Python User State and Timer APIs
Posted by Charles Chen <cc...@google.com>.
An update on the implementation: I recently sent out the user-facing
pipeline construction part of the API implementation out for review:
https://github.com/apache/beam/pull/5691.
On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <cc...@google.com> wrote:
> Thanks everyone for contributing here. We've reached rough consensus on
> the approach we should take with this API, and I've summarized this in the
> new "Community consensus" sections I added to the doc (
> https://s.apache.org/beam-python-user-state-and-timers). I will begin
> initial implementation of this API soon.
>
> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <th...@apache.org> wrote:
>
>> Nice proposal; it's exciting to see this about to be added to the SDK as
>> it enables a set of more complex use cases.
>>
>> I also think that some of the content can later be repurposed as user
>> documentation.
>>
>> Thanks,
>> Thomas
>>
>>
>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <cc...@google.com> wrote:
>>
>>> Thanks everyone for the detailed comments and discussions. It looks
>>> like by now, we mostly agree with the requirements and overall direction
>>> needed for the API, though there is continuing discussion on specific
>>> details. I want to highlight two new sections of the doc, which address
>>> some discussions that have come up:
>>>
>>> - *Existing state and transactionality*: this section addresses how
>>> we will address an existing transactionality inconsistency in the existing
>>> Java API. (
>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b
>>> )
>>> - *State for merging windows*: this section addresses how we will
>>> deal with non-combinable state in conjunction with merging windows. (
>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy
>>> )
>>>
>>> Let me know any further comments and suggestions.
>>>
>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <kl...@google.com> wrote:
>>>
>>>> Nice. I know that Java users have found it helpful to have this
>>>> lower-level way of writing pipelines when the high-level primitives don't
>>>> quite have the tight control they are looking for. I hope it will be a big
>>>> draw for Python, too.
>>>>
>>>> (commenting on the doc)
>>>>
>>>> Kenn
>>>>
>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <cc...@google.com> wrote:
>>>>
>>>>> I want to share a proposal for adding user state and timer support to
>>>>> the Beam Python SDK and get the community's thoughts on how such an API
>>>>> should look: https://s.apache.org/beam-python-user-state-and-timers
>>>>>
>>>>> Let me know what you think and please add any comments and suggestions
>>>>> you may have.
>>>>>
>>>>> Best,
>>>>> Charles
>>>>>
>>>>
>>