[jira] [Updated] (HDDS-16319) Support server side key copy that shares blocks instead of rewriting the data

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-16319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HDDS-16319:
----------------------------------
    Labels: pull-request-available  (was: )

> Support server side key copy that shares blocks instead of rewriting the data
> -----------------------------------------------------------------------------
>
>                 Key: HDDS-16319
>                 URL: https://issues.apache.org/jira/browse/HDDS-16319
>             Project: Apache Ozone
>          Issue Type: Improvement
>            Reporter: Chu Cheng Li
>            Priority: Major
>              Labels: pull-request-available
>
> S3 CopyObject in Ozone reads every byte of the source key from the datanodes through the S3 Gateway and writes it back as a new key, so copy latency is proportional to object size. {{ObjectEndpoint.copyObject()}} opens an {{OzoneInputStream}} on the source and streams it into a newly created key; there is no server side copy RPC in OM, and {{ozone sh key cp}} is a client side stream copy for the same reason.
> This proposes an OM {{CopyKey}} request that creates the destination key from the source key's committed block list, so a copy moves no data.
> h3. Why this is not just a new request
> Committed blocks are owned by exactly one key and there is no reference counting in OM or SCM. Deleting or overwriting either key would move the shared blocks to deletedTable, and KeyDeletingService would hand them to SCM, silently breaking the surviving key: a datanode asked to delete an already deleted block succeeds quietly. The one existing protection, {{OMKeyRequest.filterOutBlocksStillInUse}}, only compares against the new version of the same key.
> h3. Proposed design
> * an optional {{sharedBlockGroupId}} on {{KeyInfo}}, zero for every existing key so nothing needs backfilling. A non-zero value is the objectID of the lineage root.
> * a {{sharedBlockGroupTable}} column family holding one sharer count per copy lineage, not one row per block, so copying a key with ten thousand blocks costs a single row.
> * KeyDeletingService consults that count at the single point where deletedTable entries become SCM block deletions. While another sharer is alive the blocks are withheld from the SCM call and only the metadata is purged, reusing the path empty keys already take.
> * the count drop rides the same Ratis transaction as the purge, and is carried by the final batch on purpose: losing it after the rows were purged leaks blocks, which an audit can reclaim, whereas decrementing first and then failing to purge would let a retry decrement twice and release blocks a live key still uses.
> No block release producer has to change, because the tag travels with OmKeyInfo into deletedTable through {{OmUtils.prepareKeyForDelete}}. Every failure direction degrades to a leak rather than to data loss, which is the main reason for preferring this over a per-block reference count that every producer would have to maintain correctly.
> h3. When a copy must still rewrite the data
> Encrypted keys (block data is ciphertext under the source key's own DEK and IV), GDPR enforced keys (erasure destroys the key's own secret), a replication config change (the read path picks the EC or Ratis stream from the key's own config), an hsync active source, and UploadPartCopy, which cannot express a byte range as whole object sharing.
> h3. Measurements
> On a three datanode MiniOzoneCluster, medians over five iterations: CopyKey costs 6.7 to 9.2 ms at 1, 16, 64 and 256 MiB, with no trend across the size range, because it is one OM Ratis transaction regardless of how much data the key holds. The read and rewrite copy grows from 117 ms to 11.1 s over the same range. The ratios overstate a real cluster, whose byte copy beats the 18 to 23 MiB/s a single host manages; projecting from a realistic 100 MiB/s still gives roughly 320x at 256 MiB and 1280x at 1 GiB.
> h3. Relation to HDDS-569
> HDDS-569 proposed the same thing in 2018 and states the copy "in ozone world, this is just a metadata change", but its two attached patches define proto messages only, and the block reclaim problem above was never addressed. This issue carries the design and an implementation.
> h3. Status
> The attached pull request is a proof of concept, scoped to OBJECT_STORE buckets, same bucket, no destination overwrite. Required before it can merge: OMLayoutFeature gating, without which an older OM treats the tag as an unknown proto field and reclaims shared blocks normally. Also outstanding: cross-bucket copy, destination overwrite, FSO, OzoneManagerVersion negotiation, an "ozone repair om" count audit, Recon awareness, and S3 Gateway wiring with fallback.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org