[jira] [Commented] (BEAM-260) Know the getSideInputWindow upper bound so can gc side input state

["Kenneth Knowles (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494237#comment-15494237 ] 

Kenneth Knowles commented on BEAM-260:
--------------------------------------

This came up again on the dev list in [this thread|http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201609.mbox/%3CCA%2B5xAo0JXSufSTSpa1cQ-yUU%3DgKnTK84HjBqWWL8T3NpiRpbOA%40mail.gmail.com%3E] by [~thw]. I have a simple proposed solution at https://s.apache.org/beam-windowmappingfn-1-pager. It is small, but may just be large enough to merit a \[PROPOSAL\] thread on the mailing list or a "BIP" if we ever keep a separate catalog of those.

> Know the getSideInputWindow upper bound so can gc side input state
> ------------------------------------------------------------------
>
>                 Key: BEAM-260
>                 URL: https://issues.apache.org/jira/browse/BEAM-260
>             Project: Beam
>          Issue Type: Bug
>          Components: beam-model
>            Reporter: Mark Shields
>            Assignee: Kenneth Knowles
>
> We currently have no static knowledge about the getSideInputWindow function, and runners are thus forced to hold on to all side input state / elements in case a future element reaches back into an earlier side input element.
> Maybe we need an upper bound on lag from current to result of getSideInputWindow so we can have a progressing gc horizon as we do for  GKB window state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)