[jira] [Updated] (HADOOP-15460) S3A FS to add ""fs.s3a.create.performance" to the builder file creation option set

["Steve Loughran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran updated HADOOP-15460:
------------------------------------
    Summary: S3A FS to add  ""fs.s3a.create.performance" to the builder file creation option set  (was: S3A FS to add  "s3a:no-existence-checks" to the builder file creation option set)

> S3A FS to add  ""fs.s3a.create.performance" to the builder file creation option set
> -----------------------------------------------------------------------------------
>
>                 Key: HADOOP-15460
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15460
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.1.0
>            Reporter: Steve Loughran
>            Priority: Major
>
> As promised to [~StephanEwen]: add and s3a-specific option to the builder-API to create files for all existence checks to be skipped.
> This
> # eliminates a few hundred milliseconds
> # avoids any caching of negative HEAD/GET responses in the S3 load balancers.
> Callers will be expected to know what what they are doing.
> FWIW, we are doing some PUT calls in the committer which bypass this stuff, for the same reason. If you've just created a directory, you know there's nothing underneath, so no need to check.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org