[jira] [Commented] (FLINK-6491) Add QueryConfig to specify state retention time for streaming queries

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004626#comment-16004626 ] 

ASF GitHub Bot commented on FLINK-6491:
---------------------------------------

GitHub user sunjincheng121 opened a pull request:

    https://github.com/apache/flink/pull/3863

    [FLINK-6491][talbe]Add QueryConfig to specify state retention time for streaming queries

    In this PR we have the changes as follows:
    1. Add QueryConfig and state clean up for non-windowed aggregates.
    2. Add QueryConfig and state clean up for over-windowed aggregates.
    
    - [x] General
      - The pull request references the related JIRA issue ("[FLINK-6491][talbe]Add QueryConfig to specify state retention time for streaming queries")
      - The pull request addresses only one issue
      - Each commit in the PR has a meaningful commit message (including the JIRA id)
    
    - [ ] Documentation
      - Documentation has been added for new functionality
      - Old documentation affected by the pull request has been updated
      - JavaDoc for public methods has been added
    
    - [x] Tests & Build
      - Functionality added by the pull request is covered by tests
      - `mvn clean verify` has been executed successfully locally or a Travis build has passed


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sunjincheng121/flink FLINK-6491-PR

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/3863.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3863
    
----
commit 2d168a924d5a607343579793b64f621aa15419bd
Author: Fabian Hueske <fh...@apache.org>
Date:   2017-05-08T16:41:37Z

    [FLINK-6491] [table] Add QueryConfig and state clean up for non-windowed aggregates.

commit f8b2ef3d27ef73142679ab50882a46c895074947
Author: sunjincheng121 <su...@gmail.com>
Date:   2017-05-09T06:36:42Z

    [FLINK-6491] [table] Add QueryConfig and state clean up for over-windowed aggregates.

----


> Add QueryConfig to specify state retention time for streaming queries
> ---------------------------------------------------------------------
>
>                 Key: FLINK-6491
>                 URL: https://issues.apache.org/jira/browse/FLINK-6491
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API & SQL
>    Affects Versions: 1.3.0
>            Reporter: Fabian Hueske
>            Assignee: sunjincheng
>            Priority: Critical
>
> By now we have a couple of streaming operators (group-windows, over-windows, non-windowed aggregations) that require operator state. Since state is not automatically cleaned-up by Flink, we need to add a mechanism to configure a state retention time. 
> If configured, a query will retain state for a specified period of state inactivity. If state is not accessed within this period of time, it will be cleared. I propose to add two parameters for this, a min and a max retention time. The min retention time specifies the earliest time and the max retention time the latest time when state is cleared. The reasoning for having two parameters is that we can avoid to register many timers if we have more freedom when to discard state.
> This issue also introduces a QueryConfig object which can be passed to a streaming query, when it is emitted to a TableSink or converted to a DataStream (append or retraction).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)