[jira] [Commented] (HADOOP-18579) Warn when no region is configured

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17648598#comment-17648598 ] 

ASF GitHub Bot commented on HADOOP-18579:
-----------------------------------------

ahmarsuhail opened a new pull request, #5230:
URL: https://github.com/apache/hadoop/pull/5230

   ### Description of PR
   
   [Jira](https://issues.apache.org/jira/browse/HADOOP-18579)
   
   Warn when no region has been configured `fs.s3a.endpoint.region`. This is to prepare for the SDK V2 upgrade, in which requests to S3 will fail if the client is configured with an incorrect region. 
   
   ### How was this patch tested?
   
   Did not set `fs.s3a.endpoint.region`, and checked that warning is logged when running an integration test. Set the property, and checked that warning is not logged. 
   
   Also tested in `eu-west-1` by running `mvn -Dparallel-tests -DtestsThreadCount=16 clean verify`.
   
   
   




> Warn when no region is configured
> ---------------------------------
>
>                 Key: HADOOP-18579
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18579
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.3.9
>            Reporter: Ahmar Suhail
>            Priority: Minor
>
> The AWS Java SDK V1 allows for cross region access. This means that even if you instantiate the S3 client with US_EAST_1 (or any region different to your actual bucket's region), the SDK will figure out the region. 
>  
> With the upgrade to SDK V2, this is no longer supported and the region should be set explicitly. Requests with the incorrect region will fail. To prepare for this change, S3A should warn when a region is not set via fs.s3a.endpoint.region. 
>  
> We should warn even if fs.s3a.endpoint is set and region can be parsed from this. This is because it is recommended to let the SDK V2 figure out the endpoint to use from the region, and so S3A should discourage from setting the endpoint unless absolutely required (eg for third party stores). 
>  
> Ideally rename fs.s3a.endpoint.region to fs.s3a.region, but not sure if this is ok to do. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org