[jira] [Resolved] (HADOOP-16017) Add some S3A-specific create file options

["Steve Loughran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved HADOOP-16017.
-------------------------------------
    Fix Version/s: 3.3.5
         Assignee: Steve Loughran
       Resolution: Duplicate

> Add some S3A-specific create file options
> -----------------------------------------
>
>                 Key: HADOOP-16017
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16017
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>             Fix For: 3.3.5
>
>
> Add some options in createFile() for S3A specifically. 
> I think we need something for "put-only" where none of the existence checks nor cleanup checks are made when s3guard == off. That is: no HEAD/LIST first, no DELETE parents after. Only the PUT is done.
> This is 
> * faster
> * lower cost
> * avoids 404's being cached in load balancers and so create inconsistency
> It does rely on the caller knowing what they are doing else you end up in a mess, but since the s3a committers use WriteOperationsHelper for this exact operation, we should open it up to others who also know what they are doing. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org