You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Kenneth Knowles (JIRA)" <ji...@apache.org> on 2017/03/23 04:56:41 UTC
[jira] [Commented] (BEAM-1261) State API should allow state to be
managed in different windows
[ https://issues.apache.org/jira/browse/BEAM-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937725#comment-15937725 ]
Kenneth Knowles commented on BEAM-1261:
---------------------------------------
This is an interesting model idea somewhat related to WindowMappingFn. I think we should first gain familiarity with both of those before tackling this, so I am going to set to unassigned.
> State API should allow state to be managed in different windows
> ---------------------------------------------------------------
>
> Key: BEAM-1261
> URL: https://issues.apache.org/jira/browse/BEAM-1261
> Project: Beam
> Issue Type: New Feature
> Components: beam-model, sdk-java-core
> Reporter: Ben Chambers
> Assignee: Kenneth Knowles
>
> For example, even if the elements are being processed in fixed windows of an hour, it may be desirable for the state to "roll over" between windows (or be available to all windows).
> It will also be necessary to figure out when this state should be deleted (TTL? maximum retention?)
> Another problem is how to deal with out of order data. If data comes in from the 10:00 AM window, should its state changes be visible to the data in the 9:00 AM window?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: [jira] [Commented] (BEAM-1261) State API should allow state to be
managed in different windows
Posted by Robert Bradshaw <ro...@google.com.INVALID>.
I like the idea of being able to use WindowMappingFns to access state
across windows in a manner similar to how side inputs are accessed.
On Wed, Mar 22, 2017 at 9:56 PM, Kenneth Knowles (JIRA) <ji...@apache.org>
wrote:
>
> [ https://issues.apache.org/jira/browse/BEAM-1261?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel&focusedCommentId=15937725#comment-15937725 ]
>
> Kenneth Knowles commented on BEAM-1261:
> ---------------------------------------
>
> This is an interesting model idea somewhat related to WindowMappingFn. I
> think we should first gain familiarity with both of those before tackling
> this, so I am going to set to unassigned.
>
> > State API should allow state to be managed in different windows
> > ---------------------------------------------------------------
> >
> > Key: BEAM-1261
> > URL: https://issues.apache.org/jira/browse/BEAM-1261
> > Project: Beam
> > Issue Type: New Feature
> > Components: beam-model, sdk-java-core
> > Reporter: Ben Chambers
> > Assignee: Kenneth Knowles
> >
> > For example, even if the elements are being processed in fixed windows
> of an hour, it may be desirable for the state to "roll over" between
> windows (or be available to all windows).
> > It will also be necessary to figure out when this state should be
> deleted (TTL? maximum retention?)
> > Another problem is how to deal with out of order data. If data comes in
> from the 10:00 AM window, should its state changes be visible to the data
> in the 9:00 AM window?
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.15#6346)
>