[jira] [Commented] (HADOOP-16711) With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init()

["Steve Loughran (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16987995#comment-16987995 ] 

Steve Loughran commented on HADOOP-16711:
-----------------------------------------

We do a lot of stuff in init time; making this all lazy/async eval is not trivial. It will also delay the time at which problem surface until much later.

I am not saying that it can't be done but it would take work -you don't just have to make the operation async, every public API will need to be extended to possibly block for the operation to complete, and to propagate any failure. We could *maybe* hide this in checkOpen calls, but still...

> With S3Guard + authmode, consider skipping "verifyBuckets" check in S3A fs init()
> ---------------------------------------------------------------------------------
>
>                 Key: HADOOP-16711
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16711
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Rajesh Balamohan
>            Priority: Minor
>              Labels: performance
>         Attachments: HADOOP-16711.prelim.1.patch
>
>
> When authoritative mode is enabled with s3guard, it would be good to skip verifyBuckets call during S3A filesystem init(). This would save call to S3 during init method.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org