[jira] [Commented] (HADOOP-14583) wasb throws an exception if you try to create a file and there's no parent directory

["Esfandiar Manii (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128044#comment-16128044 ] 

Esfandiar Manii commented on HADOOP-14583:
------------------------------------------

This is not related to if parent directory does not exist. The issue is concurrency. If there are multiple threads trying to create and delete the same file over and over, you will be having a scenario where:
ThreadA -> Creates File
ThreadB -> Looks up the file and it exists
ThreadA -> Removes file
ThreadB -> Looks up for metadata before creation and throws exception

The logic to make the metadata lookup safe was missed from the create function where most of the other ones have this logic. I updated the code and added a test to ensure this wont happen under over 100 threads.

Please take a look over the patch and let me know if you have comments.


> wasb throws an exception if you try to create a file and there's no parent directory
> ------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14583
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14583
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Priority: Minor
>         Attachments: HADOOP-14583-001.patch
>
>
> It's a known defect of the Hadoop FS API (and one we don't explicitly test for enough), but you can create a file on a path which doesn't exist. In that situation, the create() logic is expectd to create the entries.
> Wasb appears to raise an exception if you try to call {{create(filepath)}} without calling {{mkdirs(filepath.getParent()}} first. That's the semantics expected of {{createNonRecursive()}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org