[GitHub] [hadoop] mukund-thakur commented on a change in pull request #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

[GitBox <gi...@apache.org>]

mukund-thakur commented on a change in pull request #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()
URL: https://github.com/apache/hadoop/pull/1838#discussion_r377517531
 
 

 ##########
 File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
 ##########
 @@ -452,6 +450,33 @@ public void initialize(URI name, Configuration originalConf)
 
   }
 
+  /**
+   * Test bucket existence in S3.
+   * When value of {@link Constants#S3A_BUCKET_PROBE is set to 0 by client,
+   * bucket existence check is not done to improve performance of
+   * S3AFileSystem initialisation. When set to 1 or 2, bucket existence check
+   * will be performed which is potentially slow.
+   * @throws IOException
+   */
+  private void doBucketProbing() throws IOException {
 
 Review comment:
   verifyBucketExists() and verifyBucketExistsV2() are the methods which are getting called from the doBucketProbing() method and I see they are already using invoked which has a retry policy set to TRY_ONCE_THEN_FAIL. Do we need to put explicit retry in this method? 
   Also both these methods are annotated with RetryTranslated and the documentation of Retried say that if RetryTranslated is used, the called shouldn't perform another layer of retries.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org