You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "andrew-pienso (via GitHub)" <gi...@apache.org> on 2025/03/20 21:56:00 UTC
[I] Generic S3 error: Client error with status 411 Length Required [arrow-rs-object-store]
andrew-pienso opened a new issue, #278:
URL: https://github.com/apache/arrow-rs-object-store/issues/278
**Describe the bug**
Using `delta-rs` to write to gcp configured s3 storage yields a 411 error.
Relevant ticket in delta-rs: https://github.com/delta-io/delta-rs/issues/2394
**To Reproduce**
https://github.com/delta-io/delta-rs/issues/2394#issuecomment-2132886429
**Expected behavior**
Should be able to use gcp configured s3 storage.
**Additional context**
I did some digging and it appears this may have been fixed at one point by https://github.com/apache/arrow-rs/pull/4518
but was then regressed by https://github.com/apache/arrow-rs/commit/873176078abe54b4503ef15b8d1a2509095331e9#diff-0feb099a0931110c214fa9b9c8c5528512f38ebf22448772a815baee5973b8c4L354-L357
I also noticed that the aws client doesn't currently appear to pass content length of zero:
https://github.com/apache/arrow-rs/blob/main/object_store/src/aws/client.rs#L615
Yet the gcp client does: https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/client.rs#L441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org