[jira] [Created] (HADOOP-17215) ABFS: Excessive Create overwrites leading to unnecessary extra transactions

["Sneha Vijayarajan (Jira)" <ji...@apache.org>]

Sneha Vijayarajan created HADOOP-17215:
------------------------------------------

             Summary: ABFS: Excessive Create overwrites leading to unnecessary extra transactions
                 Key: HADOOP-17215
                 URL: https://issues.apache.org/jira/browse/HADOOP-17215
             Project: Hadoop Common
          Issue Type: Sub-task
            Reporter: Sneha Vijayarajan
            Assignee: Sneha Vijayarajan


Filesystem Create APIs that do not accept an argument for overwrite flag end up defaulting it to true. 

We are observing that request count of creates with overwrite=true is more and primarily because of the default setting of the flag is true of the called Create API. When a create with overwrite ends up timing out, we have observed that it could lead to race conditions between the first create and retried one running almost parallel.

To avoid this scenario for create with overwrite=true request, ABFS driver will always attempt to create without overwrite. If the create fails due to fileAlreadyPresent, it will resend the request with overwrite=true. 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org