You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "antoine-de (via GitHub)" <gi...@apache.org> on 2026/03/20 10:40:43 UTC
[I] bug: service S3: Missing ACL on multipart upload [opendal]
antoine-de opened a new issue, #7254:
URL: https://github.com/apache/opendal/issues/7254
### Describe the bug
After using #7186 in production, I realize that sometimes ACL were missing from the objects.
After digging more, I realized that I forgot to add ACL on multipart upload, so small files were fine, but big files were missing the ACLs.
ACL should also be set for multipart upload
### Steps to Reproduce
minio do not support ACL at file level, so you need to use another s3 provider.
Then with a big file (in python because that is what I use):
```python
import opendal
op = opendal.Operator('s3',
root='<root>',
bucket='<your_bucket>',
access_key_id='<key>',
secret_access_key='<secret>',
endpoint='<endpoint>',
region='<region>',
default_acl='public-read'
)
# with files bigger that 6M I reproduce the bug
with open("a_big_file.jpg", 'rb') as f:
file_bytes=f.read() ...
with op.open('test_file.jpg', 'wb') as s3_file:
s3_file.write(file_bytes)
```
Then you can check the ACL with the aws cli
```
aws s3api --bucket your_bucket --key <your_root>/test_file.jpg
````
output should have something like
```js
"Grants": [
{
"Grantee": {
"Type": "Group",
"URI": "http://acs.amazonaws.com/groups/global/AllUsers"
},
"Permission": "READ"
},
```
but there is not this for the moment
### Expected Behavior
ACL should be set to public-read
### Additional Context
_No response_
### Are you willing to submit a PR to fix this bug?
- [ ] Yes, I would like to submit a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: service S3: Missing ACL on multipart upload [opendal]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #7254:
URL: https://github.com/apache/opendal/issues/7254#issuecomment-4097169631
<!-- Greeting -->
Hi @antoine-de! I'm [Dosu](https://go.dosu.dev/dosubot) and Iām helping the opendal team.
<!-- Answer -->
Thanks for the detailed bug report! You're right - looking at the code, the [`s3_initiate_multipart_upload` function](https://github.com/apache/opendal/blob/27d5677d4526519313cbce00cd28f8fbcc099642/core/src/services/s3/core.rs#L810-L862) doesn't include the `x-amz-acl` header, while the regular PUT object request does get the ACL header after #7186.
The fix would be to add the ACL header to the multipart upload initiation request in the same way it's added to regular uploads. Per AWS S3 documentation, the `x-amz-acl` header should be set during the `CreateMultipartUpload` operation (not during individual part uploads).
The change should be in [`core/src/services/s3/core.rs`](https://github.com/apache/opendal/blob/27d5677d4526519313cbce00cd28f8fbcc099642/core/src/services/s3/core.rs) in the `s3_initiate_multipart_upload` function, adding something like:
```rust
if let Some(acl) = &self.default_acl {
req = req.header(constants::X_AMZ_ACL, acl);
}
```
Would you like to submit a PR for this fix?
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=brand-link).*
---
<sup>How did I do? [Good](https://app.dosu.dev/response-feedback/568fdddc-a381-4336-b6f2-3dd9bfd8f6a1?feedback_type=great_response&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/568fdddc-a381-4336-b6f2-3dd9bfd8f6a1?feedback_type=irrelevant_answer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/568fdddc-a381-4336-b6f2-3dd9bfd8f6a1?feedback_type=incorrect_sources&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/568fdddc-a381-4336-b6f2-3dd9bfd8f6a1?feedback_type=too_verbose&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/568fdddc-a381
-4336-b6f2-3dd9bfd8f6a1?feedback_type=hallucination&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-hallucination) | [Report š](https://app.dosu.dev/response-feedback/568fdddc-a381-4336-b6f2-3dd9bfd8f6a1?feedback_type=bug_report&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-bug_report) | [Other](https://app.dosu.dev/response-feedback/568fdddc-a381-4336-b6f2-3dd9bfd8f6a1?feedback_type=other&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-other)</sup>
[](https://app.dosu.dev/997a2c5a-0c54-497b-989d-44651635dd36/ask?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=chat-badge) [](https://cursor.com/link/prompt?text=Thanks%20for%20the%20detailed%20bug%20report%21%20You%27re%20right%20-%20looking%20at%20the%20code%2C%20the%20%5B%60s3_initiate_multipart_upload%60%20function%5D%28https%3A//github.com/apache/opendal/blob/27d5677d4526519313cbce00cd28f8fbcc099642/core/src/services/s3/core.rs%23L810-L862%29%20doesn%27t%20include%20the%20%60x-amz-acl%60%20header%2C%20while%20the%20regular%20PUT%20object%20request%20does%20get%20the%20ACL%20header%20after%20%237186.%0A%0AThe%20fix%20would%20be%20to%20add%20the%20ACL%20header%20to%20the%20multipart%20upload%20initiation%20request%20in%20the%20same%20way%20it%27s%20added%20to%20regular%20uploads.%20Per%20AWS%20S3%20documentation%2C%20the%20%60x-a
mz-acl%60%20header%20should%20be%20set%20during%20the%20%60CreateMultipartUpload%60%20operation%20%28not%20during%20individual%20part%20uploads%29.%0A%0AThe%20change%20should%20be%20in%20%5B%60core/src/services/s3/core.rs%60%5D%28https%3A//github.com/apache/opendal/blob/27d5677d4526519313cbce00cd28f8fbcc099642/core/src/services/s3/core.rs%29%20in%20the%20%60s3_initiate_multipart_upload%60%20function%2C%20adding%20something%20like%3A%0A%0A%60%60%60rust%0Aif%20let%20Some%28acl%29%20%3D%20%26self.default_acl%20%7B%0A%20%20%20%20req%20%3D%20req.header%28constants%3A%3AX_AMZ_ACL%2C%20acl%29%3B%0A%7D%0A%60%60%60%0A%0AWould%20you%20like%20to%20submit%20a%20PR%20for%20this%20fix%3F) [](https://go.dosu.dev/discord-bot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=join-discord) [](https://twitter.co
m/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/opendal/issues/7254)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: service S3: Missing ACL on multipart upload [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #7254: bug: service S3: Missing ACL on multipart upload
URL: https://github.com/apache/opendal/issues/7254
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org