[jira] [Commented] (FLINK-15506) State TTL Compaction Filter option off by default

["Andrey Zagrebin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17016098#comment-17016098 ] 

Andrey Zagrebin commented on FLINK-15506:
-----------------------------------------

Having the TTL compaction filter always enabled, currently results in that it runs for all Flink states independently from whether they are configured to have TTL or not. This theoretically increases compaction time and wastes some CPU cycles on calling the filter with the disabled TTL check. To test this, we have to try it by running a job with high compaction pressure which is difficult to generate. The common performance benchmark should be at least checked. In general, we do not expect any significant performance degradation as the calling of idle filter should be a negligible internal native C++ call w/o using JNI.

This option can be also quite confusing for users. We can deprecate it but keep to fallback atm. Then see if there are any actual performance degradation reported by users. If none, we can remove it completely later.

> State TTL Compaction Filter option off by default
> -------------------------------------------------
>
>                 Key: FLINK-15506
>                 URL: https://issues.apache.org/jira/browse/FLINK-15506
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / State Backends
>            Reporter: Stephan Ewen
>            Assignee: Andrey Zagrebin
>            Priority: Blocker
>             Fix For: 1.10.0, 1.11.0
>
>
> Compaction filters for state TTL cleanup is on by default in some places, but the config option still is set to {{false}} by default.
> https://github.com/apache/flink/blob/6ff392842f9dc4d3c9c808e7912558d477826379/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBOptions.java#L63



--
This message was sent by Atlassian Jira
(v8.3.4#803005)