You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pdeva (via GitHub)" <gi...@apache.org> on 2026/08/09 06:01:24 UTC
[PR] aws: use dual-stack endpoints for S3 Express [arrow-rs-object-store]
pdeva opened a new pull request, #828:
URL: https://github.com/apache/arrow-rs-object-store/pull/828
S3 Express endpoints currently omit the `dualstack` label. In IPv6-only environments, both session creation and zonal object requests therefore resolve only IPv4 addresses and fail to connect.
This change centralizes S3 Express endpoint construction and uses the AWS dual-stack hostname:
```
https://{bucket}.s3express-{zone}.dualstack.{region}.amazonaws.com
```
The same endpoint feeds both the internal session credential provider and the zonal data client.
Validation:
- `cargo fmt --all -- --check`
- `cargo test -p object_store --features aws test_s3_express_endpoint`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] aws: use dual-stack endpoints for S3 Express [arrow-rs-object-store]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #828:
URL: https://github.com/apache/arrow-rs-object-store/pull/828#issuecomment-5319988343
Thank you for this @pdeva -- can you explain what problem this is solving (as in what isn't working for you?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] aws: use dual-stack endpoints for S3 Express [arrow-rs-object-store]
Posted by "pdeva (via GitHub)" <gi...@apache.org>.
pdeva commented on PR #828:
URL: https://github.com/apache/arrow-rs-object-store/pull/828#issuecomment-5321226858
we want to talk to s3 express using ipv6 . note that you did add support for dualstack for regular s3 buckets recently but forgot about s3 express
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] aws: use dual-stack endpoints for S3 Express [arrow-rs-object-store]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #828:
URL: https://github.com/apache/arrow-rs-object-store/pull/828#discussion_r3816883679
##########
src/aws/builder.rs:
##########
@@ -1739,6 +1803,7 @@ mod tests {
fn s3_test_bucket_endpoint() {
let builder = AmazonS3Builder::new()
.with_endpoint("http://some.host:1234")
+ .with_use_dualstack_endpoint(true)
Review Comment:
I think we should add new test coverage rather than changing existing coverage -- that way it is easier to verify that this PR does not change existing behavior
##########
src/aws/builder.rs:
##########
@@ -1244,15 +1274,15 @@ impl AmazonS3Builder {
)) as _
};
+ let use_dualstack_endpoint = self.use_dualstack_endpoint.get()?;
let (session_provider, zonal_endpoint) = match self.s3_express.get()? {
true => {
let zone = parse_bucket_az(&bucket).ok_or_else(|| {
let bucket = bucket.clone();
Error::ZoneSuffix { bucket }
})?;
- // https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-express-Regions-and-Zones.html
Review Comment:
we lost this comment with the reference to the URL format documentation -- can you please restore it?
##########
src/aws/builder.rs:
##########
@@ -1278,12 +1308,21 @@ impl AmazonS3Builder {
// If `endpoint` is provided it's assumed to be consistent with `virtual_hosted_style_request` or `s3_express`.
// For example, if `virtual_hosted_style_request` is true then `endpoint` should have bucket name included.
let virtual_hosted = self.virtual_hosted_style_request.get()?;
+ let dualstack = if use_dualstack_endpoint {
+ ".dualstack"
+ } else {
+ ""
+ };
let bucket_endpoint = match (&endpoint, zonal_endpoint, virtual_hosted) {
(Some(endpoint), _, true) => endpoint.clone(),
(Some(endpoint), _, false) => format!("{}/{}", endpoint.trim_end_matches("/"), bucket),
(None, Some(endpoint), _) => endpoint,
- (None, None, true) => format!("https://{bucket}.s3.{region}.amazonaws.com"),
- (None, None, false) => format!("https://s3.{region}.amazonaws.com/{bucket}"),
+ (None, None, true) => {
Review Comment:
can you please add a comment here explaining where this format comes from I think it is https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
##########
src/aws/builder.rs:
##########
@@ -961,6 +981,16 @@ impl AmazonS3Builder {
self
}
+ /// Configure whether to use dual-stack endpoints
+ ///
+ /// Defaults to `false`. If enabled for a service or region without a dual-stack endpoint,
Review Comment:
I double checked that the AWS docs do say that dual-stack is only available in some regions
> Dual-stack endpoints are available for some services in some regions.
https://docs.aws.amazon.com/sdkref/latest/guide/feature-endpoints.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] aws: add opt-in dual-stack endpoint generation [arrow-rs-object-store]
Posted by "pdeva (via GitHub)" <gi...@apache.org>.
pdeva commented on code in PR #828:
URL: https://github.com/apache/arrow-rs-object-store/pull/828#discussion_r3817647560
##########
src/aws/builder.rs:
##########
@@ -1739,6 +1803,7 @@ mod tests {
fn s3_test_bucket_endpoint() {
let builder = AmazonS3Builder::new()
.with_endpoint("http://some.host:1234")
+ .with_use_dualstack_endpoint(true)
Review Comment:
Addressed in [`d125913`](https://github.com/apache/arrow-rs-object-store/commit/d1259133d6179ab17ddf857ba26b326da2e1efb1): I restored the existing `test_parse_bucket_az`, `s3_test_bucket_endpoint`, and S3 endpoint-precedence coverage unchanged, and moved the dual-stack override assertions into the new `test_generated_s3_endpoints` matrix.
##########
src/aws/builder.rs:
##########
@@ -1244,15 +1274,15 @@ impl AmazonS3Builder {
)) as _
};
+ let use_dualstack_endpoint = self.use_dualstack_endpoint.get()?;
let (session_provider, zonal_endpoint) = match self.s3_express.get()? {
true => {
let zone = parse_bucket_az(&bucket).ok_or_else(|| {
let bucket = bucket.clone();
Error::ZoneSuffix { bucket }
})?;
- // https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-express-Regions-and-Zones.html
Review Comment:
Restored the original S3 Express Regions and Zones documentation reference immediately above endpoint construction in [`d125913`](https://github.com/apache/arrow-rs-object-store/commit/d1259133d6179ab17ddf857ba26b326da2e1efb1).
##########
src/aws/builder.rs:
##########
@@ -1278,12 +1308,21 @@ impl AmazonS3Builder {
// If `endpoint` is provided it's assumed to be consistent with `virtual_hosted_style_request` or `s3_express`.
// For example, if `virtual_hosted_style_request` is true then `endpoint` should have bucket name included.
let virtual_hosted = self.virtual_hosted_style_request.get()?;
+ let dualstack = if use_dualstack_endpoint {
+ ".dualstack"
+ } else {
+ ""
+ };
let bucket_endpoint = match (&endpoint, zonal_endpoint, virtual_hosted) {
(Some(endpoint), _, true) => endpoint.clone(),
(Some(endpoint), _, false) => format!("{}/{}", endpoint.trim_end_matches("/"), bucket),
(None, Some(endpoint), _) => endpoint,
- (None, None, true) => format!("https://{bucket}.s3.{region}.amazonaws.com"),
- (None, None, false) => format!("https://s3.{region}.amazonaws.com/{bucket}"),
+ (None, None, true) => {
Review Comment:
Added the AWS Virtual Hosting documentation reference above the generated path-style and virtual-hosted endpoint formats in [`d125913`](https://github.com/apache/arrow-rs-object-store/commit/d1259133d6179ab17ddf857ba26b326da2e1efb1).
##########
src/aws/builder.rs:
##########
@@ -961,6 +981,16 @@ impl AmazonS3Builder {
self
}
+ /// Configure whether to use dual-stack endpoints
+ ///
+ /// Defaults to `false`. If enabled for a service or region without a dual-stack endpoint,
Review Comment:
Thanks for confirming. I linked the AWS dual-stack endpoint configuration documentation directly from `with_use_dualstack_endpoint` in [`d125913`](https://github.com/apache/arrow-rs-object-store/commit/d1259133d6179ab17ddf857ba26b326da2e1efb1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] aws: use dual-stack endpoints for S3 Express [arrow-rs-object-store]
Posted by "pdeva (via GitHub)" <gi...@apache.org>.
pdeva commented on PR #828:
URL: https://github.com/apache/arrow-rs-object-store/pull/828#issuecomment-5332807512
@alamb Thanks for the reproducer — you are right. I was testing an S3 Express bucket in a regular Availability Zone (`use1-az4`), where AWS publishes the dual-stack zonal endpoint; `use1-atl2-az1` is a Local Zone, where that hostname is not available.
Fixed in [`2bc6127`](https://github.com/apache/arrow-rs-object-store/commit/2bc6127abe69c290fd11d852fa3d60cf6a1f2883):
- Dual-stack is now opt-in instead of unconditional. The default keeps `https://{bucket}.s3express-{zone}.{region}.amazonaws.com`, so the Local Zone case retains the existing working hostname.
- `AWS_USE_DUALSTACK_ENDPOINT=true`, the `use_dualstack_endpoint` config key, or `with_use_dualstack_endpoint(true)` selects the dual-stack hostname. Our staging configuration already sets the standard AWS environment variable, so it retains the dual-stack behavior.
- The option is applied consistently to generated regular S3 and S3 Express endpoints; explicitly configured endpoint URLs are left unchanged.
- Regression coverage includes the exact `use1-atl2-az1` default case, an opted-in regular AZ, path-style and virtual-hosted S3 endpoints, and explicit endpoint overrides.
Validated with the AWS test suite, the all-provider test/doctest configuration, both `aws` and `aws-base` clippy configurations, rustdoc, and formatting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] aws: use dual-stack endpoints for S3 Express [arrow-rs-object-store]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #828:
URL: https://github.com/apache/arrow-rs-object-store/pull/828#issuecomment-5331872674
I think this breaks hostname creation for s3 express
Here is a reproducer that shows things working on main but broken with this change (the hostname the PR creates is invalid)
[object-store-dual-stack.zip](https://github.com/user-attachments/files/31190545/object-store-dual-stack.zip)
```shell
cargo run
...
head() on s3_express bucket repro--use1-atl2-az1--x-s3 (Atlanta Local Zone, us-east-1)
[main ] error: Generic S3 error: Error performing CreateSession request: Error performing GET https://repro--use1-atl2-az1--x-s3.s3express-use1-atl2-az1.us-east-1.amazonaws.com/?session in 644.927084ms - Server returned non-2xx status code: 403 Forbidden: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><RequestId>019361178c0001a015f431a004001973ef4daba1</RequestId><HostId>2TG3D8a0ILee</HostId></Error>
[main ] root cause: Server returned non-2xx status code: 403 Forbidden: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><RequestId>019361178c0001a015f431a004001973ef4daba1</RequestId><HostId>2TG3D8a0ILee</HostId></Error>
[PR #828] error: Generic S3 error: Error performing CreateSession request: Error performing GET https://repro--use1-atl2-az1--x-s3.s3express-use1-atl2-az1.dualstack.us-east-1.amazonaws.com/?session in 1.438584ms - HTTP error: error sending request
[PR #828] root cause: failed to lookup address information: nodename nor servname provided, or not known
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org