You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "devinrsmith (via GitHub)" <gi...@apache.org> on 2025/11/12 18:01:06 UTC
[I] AWS: DefaultAwsClientFactory s3 / s3Async serviceConfiguration are not equivalent [iceberg]
devinrsmith opened a new issue, #14575:
URL: https://github.com/apache/iceberg/issues/14575
### Apache Iceberg version
1.10.0 (latest release)
### Query engine
None
### Please describe the bug 🐞
The "service configuration" applied to the S3Client is not applied to the S3AsyncClient in `org.apache.iceberg.aws.AwsClientFactories.DefaultAwsClientFactory`. This was noticed because `s3.path-style-access` does not seem to work for the S3AsyncClient.
The configurable bits from `org.apache.iceberg.aws.s3.S3FileIOProperties#applyServiceConfigurations` includes dualstackEnabled, crossRegionAccessEnabled, pathStyleAccessEnabled, useArnRegionEnabled, and accelerateModeEnabled, all which seem to be applicable for `S3AsyncClientBuilder` (and at least some, applicable to `S3CrtAsyncClientBuilder`, but did not dig deeper into that side of things).
Similar to #13131
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] AWS: DefaultAwsClientFactory s3 / s3Async serviceConfiguration are not equivalent [iceberg]
Posted by "RjLi13 (via GitHub)" <gi...@apache.org>.
RjLi13 commented on issue #14575:
URL: https://github.com/apache/iceberg/issues/14575#issuecomment-5210744113
I am encountering the same issue in Iceberg 1.11 for S3-analytics-accelerator library when trying cross region access. @yadavay-amzn would you be repicking up your PR? I think we also need it on aws/src/main/java/org/apache/iceberg/aws/s3/DefaultS3FileIOAwsClientFactory.java as well as CRT side. But it does fix AAL for us (tested internally)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] AWS: DefaultAwsClientFactory s3 / s3Async serviceConfiguration are not equivalent [iceberg]
Posted by "yusufyildiz34 (via GitHub)" <gi...@apache.org>.
yusufyildiz34 commented on issue #14575:
URL: https://github.com/apache/iceberg/issues/14575#issuecomment-6517964123
#16030 by @yadavay-amzn (June) and #17549 by @waterWang (October) were both closed by the stale bot without a review. Unless either of you wants to pick it back up, I'd like to take this on. Following @RjLi13's note, the plan is to apply the service settings in `s3Async()` of the default, `DefaultS3FileIOAwsClientFactory` and assume-role factories, for both the standard and the CRT async client. The CRT builder only supports path-style access, acceleration and cross-region access, so dual-stack and ARN region can't be applied there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] AWS: DefaultAwsClientFactory s3 / s3Async serviceConfiguration are not equivalent [iceberg]
Posted by "yadavay-amzn (via GitHub)" <gi...@apache.org>.
yadavay-amzn commented on issue #14575:
URL: https://github.com/apache/iceberg/issues/14575#issuecomment-4274630372
I would like to work on this. Will submit a PR shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org