[jira] [Updated] (HADOOP-13221) s3a create() doesn't check for a parent path being a file

["Sean Mackrory (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Mackrory updated HADOOP-13221:
-----------------------------------
    Attachment: HADOOP-13321-test.patch

Attaching a test for this originally written as part of HADOOP-14457. I agree with all your feedback on that portion of the patch, but given that it's less urgent than the rest of the patch and impacts other filesystems, I'd suggest we just address it after HADOOP-13345 is merged. Perhaps we can make the S3Guard-specific part asynchronous at that point as well.

Also, I did some testing of other FSs this morning:

The following already pass the test:

* TestAzureNativeContractCreate (WASB)
* TestHDFSContractCreate
* TestLocalFSContractCreate
* TestRawlocalContractCreate

The following tests fail:

* TestAdlContractCreateLive: It fails with an ACL error. All other tests in the ContractCreate class pass, so I suspect this is actually the same bug, but manifesting in ACLs instead of storage metadata explicitly.
* ITestS3NContractCreate, ITestS3AContractCreate (hence this JIRA getting opened in the first place).

The following tests were skipped, and I'll need to do some setup to run them unless someone else can do so expediently

* TestFTPContractCreate
* TestSwiftContractCreate
* TestAliyunOSSContractCreate

> s3a create() doesn't check for a parent path being a file
> ---------------------------------------------------------
>
>                 Key: HADOOP-13221
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13221
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.7.2
>            Reporter: Steve Loughran
>            Assignee: Rajesh Balamohan
>         Attachments: HADOOP-13321-test.patch
>
>
> Seen in a code review. Notable that if true, this got by all the FS contract tests —showing we missed a couple.
> {{S3AFilesystem.create()}} does not examine its parent paths to verify that there does not exist one which is a file. It looks for the destination path if overwrite=false (see HADOOP-13188 for issues there), but it doesn't check the parent for not being a file, or the parent of that path.
> It must go up the tree, verifying that either a path does not exist, or that the path is a directory. The scan can stop at the first entry which is is a directory, thus the operation is O(empty-directories) and not O(directories).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org