[GitHub] [iceberg] singhpk234 commented on a diff in pull request #5644: AWS: Add s3.dualstack-enabled flag to AwsProperties

[GitBox <gi...@apache.org>]

singhpk234 commented on code in PR #5644:
URL: https://github.com/apache/iceberg/pull/5644#discussion_r956937681


##########
docs/aws.md:
##########
@@ -509,6 +509,24 @@ spark-sql --conf spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCata
 
 For more details on using S3 Acceleration, please refer to [Configuring fast, secure file transfers using Amazon S3 Transfer Acceleration](https://docs.aws.amazon.com/AmazonS3/latest/userguide/transfer-acceleration.html).
 
+### S3 Dual-stack
+
+[S3 Dual-stack](https://docs.aws.amazon.com/AmazonS3/latest/userguide/dual-stack-endpoints.html) allows a client to access an S3 bucket through a dual-stack endpoint. 
+When clients make a request to a dual-stack endpoint, the bucket URL resolves to an IPv6 address if possible, otherwise fallback to IPv4.
+
+To use S3 Dual-stack, we need to set `s3.dualstack-enabled` catalog property to `true` to enable `S3FileIO` to make dual-stack S3 calls.
+
+For example, to use S3 Dual-stack with Spark 3.0, you can start the Spark SQL shell with:
+```
+spark-sql --conf spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCatalog \
+    --conf spark.sql.catalog.my_catalog.warehouse=s3://my-bucket2/my/key/prefix \
+    --conf spark.sql.catalog.my_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog \
+    --conf spark.sql.catalog.my_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO \
+    --conf spark.sql.catalog.my_catalog.s3.s3.dualstack-enabled=true

Review Comment:
   This should be : 
   ```suggestion
       --conf spark.sql.catalog.my_catalog.s3.dualstack-enabled=true
   ```



##########
aws/src/main/java/org/apache/iceberg/aws/AwsClientFactories.java:
##########
@@ -97,6 +97,7 @@ static class DefaultAwsClientFactory implements AwsClientFactory {
     private Boolean s3AccelerationEnabled;
     private String dynamoDbEndpoint;
     private String httpClientType;
+    private Boolean s3DualStackEnabled;

Review Comment:
   [minor] is box type required here ? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org