[jira] [Comment Edited] (FLINK-15938) Idle state not cleaned in StreamingJoinOperator and StreamingSemiAntiJoinOperator

["Andrey Zagrebin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17031663#comment-17031663 ] 

Andrey Zagrebin edited comment on FLINK-15938 at 2/6/20 3:42 PM:
-----------------------------------------------------------------

[~libenchao] 

For 1.9:
 You are right that, by default, the expired state cleanup will happen only when the state is read explicitly by the user code.
 We were conservative here to avoid potential problems with the new background cleanup feature.
 You can activate the background cleanup in 1.9:
{code:java}
return StateTtlConfig
   ...
   .cleanupInBackground()
   .build();{code}
Then, you do not have to read the expired state explicitly, it should be cleaned in background. Read also docs further about the background cleanup.

For 1.10, we decided to activate the background cleanup by default but there was no plan to do it for 1.9. Is it what you wanted to resolve with this issue?


was (Author: azagrebin):
[~libenchao] 

For 1.9:
 You are right that, by default, the expired state cleanup will happen only when the state is read explicitly by the user code.
 We were conservative here to avoid potential problems with the new background cleanup feature.
 You can activate the background cleanup in 1.9:
{code:java}
return StateTtlConfig
   ...
   .cleanupInBackground()
   .build();{code}
Then, you do not have to read the expired state explicitly, it should be cleaned in background. Read also docs further about the background cleanup.

For 1.10, we decided to activate the background cleanup by default but there was no plan to do it for 1.9. Is it what you wanted to resolve with this issue? If so, I suggest to close this issue.

> Idle state not cleaned in StreamingJoinOperator and StreamingSemiAntiJoinOperator
> ---------------------------------------------------------------------------------
>
>                 Key: FLINK-15938
>                 URL: https://issues.apache.org/jira/browse/FLINK-15938
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / Planner
>    Affects Versions: 1.9.2
>            Reporter: Benchao Li
>            Priority: Critical
>
> {code:java}
> return StateTtlConfig
>    .newBuilder(Time.milliseconds(retentionTime))
>    .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
>    .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp)
>    .build();
> {code}
> StateTtl is constructed in above code for `StreamingJoinOperator` and `StreamingSemiAntiJoinOperator`.
> However, as stated in [https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/state/state.html#cleanup-of-expired-state] , the state will be cleaned only when it's read which means the state will not be cleaned enless we read it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)