You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yanfei Lei (Jira)" <ji...@apache.org> on 2022/07/14 03:10:00 UTC
[jira] [Commented] (FLINK-28539) Enable CompactionDynamicLevelBytes in FLASH_SSD_OPTIMIZED
[ https://issues.apache.org/jira/browse/FLINK-28539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566618#comment-17566618 ]
Yanfei Lei commented on FLINK-28539:
------------------------------------
hi [~usamj] , you can use `setLevelCompactionDynamicLevelBytes` for FLASH_SSD_OPTIMIZED by configuring "RocksDBConfigurableOptions.USE_DYNAMIC_LEVEL_SIZE: true" manually.
{quote}There is also an argument to be made to also disable it for SPINNING_DISK options as it could give increased read/write performance [2]
{quote}
Currently, the default value of PREDEFINED_OPTIONS is [empty in Flink|https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBOptions.java#L77-L90], users can choose whether to open according to their own situation.
> Enable CompactionDynamicLevelBytes in FLASH_SSD_OPTIMIZED
> ---------------------------------------------------------
>
> Key: FLINK-28539
> URL: https://issues.apache.org/jira/browse/FLINK-28539
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Reporter: Usamah Jassat
> Priority: Minor
>
> Investigating the RocksDB predefined options I see that `setLevelCompactionDynamicLevelBytes` is set for SPINNING_DISK options but not FLASH_SSD_OPTIMIZED.
>
> From my research it looks like this change would improve the Space Amplification of RocksDB [1] (which can also lead to a trade-off from read/write amplification [2]). It makes sense to me that this feature should be enabled for SSD's as they tend to have less space compared to their HDD counterparts.
> There is also an argument to be made to also disable it for SPINNING_DISK options as it could give increased read/write performance [2]
> [1] [http://rocksdb.org/blog/2015/07/23/dynamic-level.html]
> [2] [https://github.com/EighteenZi/rocksdb_wiki/blob/master/RocksDB-Tuning-Guide.md#amplification-factors]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)