[PR] feat(core,services/s3): support supplying a precomputed checksum on write [opendal]

["wolfv (via GitHub)" <gi...@apache.org>]

wolfv opened a new pull request, #7830:
URL: https://github.com/apache/opendal/pull/7830

   # Which issue does this PR close?
   
   Closes #.
   
   # Rationale for this change
   
   Users often already know an object's checksum ahead of time (for example a SHA-256 they computed when producing the data) and want the storage service to verify the upload against that value, rather than having OpenDAL recompute one from the body. There is currently no way to pass a known checksum through a write operation.
   
   # What changes are included in this PR?
   
   A new, additive public API in `opendal-core`:
   
   - `Checksum` type and `ChecksumAlgorithm` enum (`Checksum::sha256(bytes)`, `crc32c`, `sha1`, `crc64nvme`). The value holds the raw digest bytes; services encode it as their wire format requires.
   - `WriteOptions::checksum` field, `OpWrite::with_checksum` / `OpWrite::checksum` accessors, and a `Operator::write_with(...).checksum(...)` builder method (plus the same on `writer_with`).
   - `Capability::write_with_checksum` to advertise support.
   
   S3 service support:
   
   - On the single-shot `PutObject` path, a supplied checksum is sent as the matching `x-amz-checksum-*` header (base64-encoded from the raw digest bytes) instead of computing one from the body.
   - A whole-object checksum cannot be mapped onto individual multipart parts, so S3 returns `Unsupported` when a checksum is combined with append or multipart writes, rather than silently dropping the integrity check.
   
   Unit tests cover the header mapping, and doctests cover the new public API.
   
   # Are there any user-facing changes?
   
   Yes, all additive (no breaking changes):
   - New public items: `opendal::Checksum`, `opendal::ChecksumAlgorithm`, `WriteOptions::checksum`, `Operator::write_with(...).checksum(...)`, `Capability::write_with_checksum`.
   
   # AI Usage Statement
   
   This PR was prepared with the assistance of Claude Code (Claude Opus 4.8). All changes were reviewed by the author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(core,services/s3): support supplying a precomputed checksum on write [opendal]

["Xuanwo (via GitHub)" <gi...@apache.org>]

Xuanwo commented on PR #7830:
URL: https://github.com/apache/opendal/pull/7830#issuecomment-4842488246

   Maybe it worths to take a look at https://github.com/apache/opendal/pull/6817


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org