[I] Tracking issue of RFC-8109: Writer::copy_from [opendal]

["Xuanwo (via GitHub)" <gi...@apache.org>]

Xuanwo opened a new issue, #8134:
URL: https://github.com/apache/opendal/issues/8134

   ## Goal
   
   Track the implementation of `Writer::copy_from` accepted in RFC #8109.
   
   The API lets an asynchronous `Writer` assemble one destination from an ordered sequence of local buffers and ranges of existing objects addressed by the same `Operator`. Its portable behavior is read-equivalence: every accepted source range produces the same destination bytes as reading it with the supplied `ReadOptions` and writing those bytes in call order. Native server-side copy is a transparent optimization; the composed streaming path remains the fallback.
   
   ## Required contract
   
   - Add `Writer::copy_from(path, range)` and `Writer::copy_from_options(path, ReadOptions)` without introducing a public copy policy or source type.
   - Allow arbitrary interleaving with `write` under the writer's existing chunk and concurrency budgets.
   - Limit each call to 5 GiB; applications split larger logical ranges into multiple calls.
   - Reject a normalized source path equal to the destination path before writer mutation.
   - Preserve source versions and conditions across metadata lookup, native requests, and streaming fallback without adding snapshot guarantees.
   - Drive each call until all eligible work is scheduled, retaining only the single logical suffix required by multipart minimum part sizes.
   - Allow append writers to use streaming fallback when native assembly is unavailable.
   - Keep `close` and `abort` responsible for the complete destination transaction; do not add a separate progress handle.
   - Preserve the raw result states: `Accepted`, `Unsupported` with no writer mutation, and execution error. Only `Unsupported` permits fallback.
   - Forward native execution through a writer wrapper only when the layer preserves source path, byte content, and route target. Otherwise use the composed read-to-write fallback.
   
   ## Tasks
   
   ### Core API and portable path
   
   - [ ] Add the public asynchronous `Writer` methods and validation.
   - [ ] Add the optional `oio::Write` native operation with one absolute bounded range and the three-state result contract.
   - [ ] Extend `WriteGenerator` to assemble local and remote segments with shared backpressure, retained-suffix handling, and boundary materialization.
   - [ ] Implement composed streaming fallback, including append writers and suffix or unbounded range resolution.
   - [ ] Propagate writer error and abort state correctly after native execution failures.
   
   ### Layers and observability
   
   - [ ] Update writer wrappers to forward or return no-mutation `Unsupported` according to the RFC layer contract.
   - [ ] Cover routing and byte-transforming layers with fallback regression tests.
   - [ ] Report logical, native, boundary-materialized, and streamed bytes through tracing and metrics.
   
   ### Native service execution
   
   - [ ] Implement S3 `UploadPartCopy` within the existing multipart upload transaction.
   - [ ] Implement TOS native part copy within the existing multipart upload transaction.
   - [ ] Implement Azure Blob `PutBlockFromURL` within the existing block transaction.
   
   ### Verification and documentation
   
   - [ ] Add behavior tests for arbitrary `write`/`copy_from` interleaving and destination byte order.
   - [ ] Test range limits, source conditions, unbounded and suffix ranges, self-copy rejection, fallback, retry, close, and abort.
   - [ ] Test multipart boundary materialization and minimum non-final part sizes.
   - [ ] Document the public API, fallback behavior, per-call limit, and observable transfer modes.
   
   ## Completion criteria
   
   The portable path works for explicit bounded ranges on services with read and write support, native implementations preserve the same public result, and unsupported native execution falls back without mutating the writer. The tracking issue remains open until the core contract, regression coverage, and initial S3, TOS, and Azure Blob native paths are complete.
   
   ## Out of scope
   
   - Cross-`Operator` sources.
   - Blocking `Writer` support.
   - Source and destination path equality.
   - A public progress handle or backend-selection policy.
   
   ## References
   
   - RFC PR: #8109
   - [Merged RFC](https://github.com/apache/opendal/blob/main/core/core/src/docs/rfcs/8109_writer_copy_from.md)
   
   ## AI assistance
   
   This tracking issue was drafted with assistance from OpenAI Codex based on the human-reviewed and merged RFC #8109. The RFC remains the source of truth; the checklist only organizes its implementation work.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org