[jira] [Updated] (HDDS-11745) Use single tar to decouple filesystem snapshot layout from OM snapshot transfer

["Wei-Chiu Chuang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wei-Chiu Chuang updated HDDS-11745:
-----------------------------------
    Parent: HDDS-15130  (was: HDDS-13747)

> Use single tar to decouple filesystem snapshot layout from OM snapshot transfer
> -------------------------------------------------------------------------------
>
>                 Key: HDDS-11745
>                 URL: https://issues.apache.org/jira/browse/HDDS-11745
>             Project: Apache Ozone
>          Issue Type: Sub-task
>            Reporter: Ethan Rose
>            Assignee: Sadanand Shenoy
>            Priority: Major
>
> The current OM snapshot creation and transfer logic uses a list of SST files to track hardlink structure at the source and recreate it at the destination. It is also used to incrementally send the snapshots to followers by only sending a few SST files at a time. This adds extra complexity to the OM snapshot flow and prevents us from easily transferring and unpacking OM DB snapshots in other contexts like Recon sync or debug/repair.
> Tar preserves hardlinks on pack and unpack by default, so we can get the same space saving by packing all required files into a single tar, and then unpacking at the destination. The tar can be broken into pieces and streamed to followers to handle cases where it is large. This decouples network transfer from storage format.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org