You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Wei-Chiu Chuang (Jira)" <ji...@apache.org> on 2025/05/01 00:18:00 UTC

[jira] [Updated] (HDDS-12501) Background thread in RocksDBCheckpointDiffer should convert backup sst files to store only key and get rid of OmKeyInfo

     [ https://issues.apache.org/jira/browse/HDDS-12501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wei-Chiu Chuang updated HDDS-12501:
-----------------------------------
    Parent: HDDS-12940  (was: HDDS-8544)

> Background thread in RocksDBCheckpointDiffer should convert backup sst files to store only key and get rid of OmKeyInfo
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDDS-12501
>                 URL: https://issues.apache.org/jira/browse/HDDS-12501
>             Project: Apache Ozone
>          Issue Type: Sub-task
>            Reporter: Swaminathan Balachandran
>            Assignee: Saketa Chalamchala
>            Priority: Major
>              Labels: pull-request-available
>
> Currently backup sst directories just stores the hard links to the input sst files on compaction. We should eventually rewrite the sst file(Atomic rewrite) and just store the key in the sst file and get rid of OMKeyInfo. This will significantly reduce the size of the directory significantly reducing the storage footprint of the sst backup directory. Currently an average of OMKeyInfo is in the order of  few KBs or even more and the average key would be in the order of 50 bytes at most. 
> While rewriting the sst file it should be kept in mind we read the tombstones in the sst file using the rawSstFileReader and write it back with sstFileWriter



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org