[jira] [Commented] (HADOOP-13221) s3a create() doesn't check for a parent path being a file

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108924#comment-16108924 ] 

Steve Loughran commented on HADOOP-13221:
-----------------------------------------

good catch of the Adl problem; shows how the same assumption "who would create a file under a file" reoccurs in independent codepaths. We all bring the same assumption about what a sensible FS would do, and code for it.

For S3N, I propose a skip(). For S3A, well, that's the tough one, isn't it? I'd go with a direct parent check, but the cost of checking an entire path is potentially very expensive. And we have never, ever, seen this problem arise in the field..it's a corner case which production code doesn't attempt. I think that's partially because production code tends to call mkdirs() before creating files, and mkdirs does contain the check

> s3a create() doesn't check for a parent path being a file
> ---------------------------------------------------------
>
>                 Key: HADOOP-13221
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13221
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.7.2
>            Reporter: Steve Loughran
>            Assignee: Rajesh Balamohan
>         Attachments: HADOOP-13321-test.patch
>
>
> Seen in a code review. Notable that if true, this got by all the FS contract tests —showing we missed a couple.
> {{S3AFilesystem.create()}} does not examine its parent paths to verify that there does not exist one which is a file. It looks for the destination path if overwrite=false (see HADOOP-13188 for issues there), but it doesn't check the parent for not being a file, or the parent of that path.
> It must go up the tree, verifying that either a path does not exist, or that the path is a directory. The scan can stop at the first entry which is is a directory, thus the operation is O(empty-directories) and not O(directories).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org