[jira] [Created] (HADOOP-15460) S3A FS to add "s3a:no-existence-checks" to the builder file creation option set

["Steve Loughran (JIRA)" <ji...@apache.org>]

Steve Loughran created HADOOP-15460:
---------------------------------------

             Summary: S3A FS to add  "s3a:no-existence-checks" to the builder file creation option set
                 Key: HADOOP-15460
                 URL: https://issues.apache.org/jira/browse/HADOOP-15460
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.1.0
            Reporter: Steve Loughran


As promised to [~StephanEwen]: add and s3a-specific option to the builder-API to create files for all existence checks to be skipped.

This
# eliminates a few hundred milliseconds
# avoids any caching of negative HEAD/GET responses in the S3 load balancers.

Callers will be expected to know what what they are doing.

FWIW, we are doing some PUT calls in the committer which bypass this stuff, for the same reason. If you've just created a directory, you know there's nothing underneath, so no need to check.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org