You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Peter Lee (Jira)" <ji...@apache.org> on 2025/05/23 11:35:00 UTC
[jira] [Updated] (HDDS-13114) MaxCompactionBytes and MaxSubCompactions should be configureable in RDBStore#compactTable
[ https://issues.apache.org/jira/browse/HDDS-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Lee updated HDDS-13114:
-----------------------------
Description:
[https://github.com/apache/ozone/pull/8141#discussion_r2093792426]
In [https://github.com/apache/ozone/pull/8141] RDBStore#compactTable was introduced.
We should set options.setMaxCompactionBytes() when we open the snapshot rocksdb this could be useful so that we ensure one sub compaction doesn't take up a lot of memory. Also look into making ManagedCompactRangeOptions.setMaxSubCompactions() configurable so that we don't use a lot of CPU for this operation. It is ok if the compactions take time.
This can be achieved by adding ManagedRangeCompactionOption into AbstractRDBStore, RDBStore, RDBStoreBuilder. Also the corresponding config keys.
was:
[https://github.com/apache/ozone/pull/8141#discussion_r2093792426]
In https://github.com/apache/ozone/pull/8141 RDBStore#compactTable was introduced.
We should set options.setMaxCompactionBytes() when we open the snapshot rocksdb this could be useful so that we ensure one sub compaction doesn't take up a lot of memory. Also look into making ManagedCompactRangeOptions.setMaxSubCompactions() configurable so that we don't use a lot of CPU for this operation. It is ok if the compactions take time.
This can be achieved by adding ManagedRangeCompactionOption into RDBStore and its builder
> MaxCompactionBytes and MaxSubCompactions should be configureable in RDBStore#compactTable
> -----------------------------------------------------------------------------------------
>
> Key: HDDS-13114
> URL: https://issues.apache.org/jira/browse/HDDS-13114
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Peter Lee
> Assignee: Peter Lee
> Priority: Major
>
> [https://github.com/apache/ozone/pull/8141#discussion_r2093792426]
>
> In [https://github.com/apache/ozone/pull/8141] RDBStore#compactTable was introduced.
> We should set options.setMaxCompactionBytes() when we open the snapshot rocksdb this could be useful so that we ensure one sub compaction doesn't take up a lot of memory. Also look into making ManagedCompactRangeOptions.setMaxSubCompactions() configurable so that we don't use a lot of CPU for this operation. It is ok if the compactions take time.
>
> This can be achieved by adding ManagedRangeCompactionOption into AbstractRDBStore, RDBStore, RDBStoreBuilder. Also the corresponding config keys.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org