[jira] [Commented] (FLINK-15393) Change 'ReturnExpiredIfNotCleanedUp' to 'NeverReturnExpired' in JoinRecordStateViews#createTtlConfig

["Shi Feng (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020136#comment-17020136 ] 

Shi Feng commented on FLINK-15393:
----------------------------------

Hi [~hailong wang] , "regular joins use rocksdb filter to clean expired data" ——— Was this feature introduced into flink1.10 by  https://issues.apache.org/jira/browse/FLINK-14898 ? I did not see any clean strategy related to JoinRecordStateViews#createTtlConfig in flink1.9(In my test case, regular join state keep growing because data rows of one join key appear only once ).

> Change  'ReturnExpiredIfNotCleanedUp' to 'NeverReturnExpired' in JoinRecordStateViews#createTtlConfig 
> ------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-15393
>                 URL: https://issues.apache.org/jira/browse/FLINK-15393
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / Runtime
>    Affects Versions: 1.10.0
>            Reporter: hailong wang
>            Priority: Major
>
> In Blink planner, we use ttl state to clean expired data by rocksdb for regular joins. The StateTtlConfig is:
> {code:java}
>  StateTtlConfig
>    .newBuilder(Time.milliseconds(retentionTime))
>    .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
>    .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp)
>    .build();
> {code}
> I think we should use StateTtlConfig.StateVisibility.NeverReturnExpired to prevent to join expired data.
> BTW, Only regular joins use rocksdb filter to clean expired data, should we change timer to rocksdb filter in other sql operator such as TemporalTableJoin?
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)