You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/07/26 18:15:00 UTC
[jira] [Commented] (HADOOP-13221) s3a create() doesn't check for an
ancestor path being a file
[ https://issues.apache.org/jira/browse/HADOOP-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558709#comment-16558709 ]
Steve Loughran commented on HADOOP-13221:
-----------------------------------------
revisiting this.
With S3Guard, we can add this the check. Without it: its too expensive. So if this goes in, we'd actually run the test if s3guard==true
> s3a create() doesn't check for an ancestor path being a file
> ------------------------------------------------------------
>
> Key: HADOOP-13221
> URL: https://issues.apache.org/jira/browse/HADOOP-13221
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.7.2
> Reporter: Steve Loughran
> Assignee: Sean Mackrory
> Priority: Major
> Attachments: HADOOP-13321-test.patch
>
>
> Seen in a code review. Notable that if true, this got by all the FS contract tests —showing we missed a couple.
> {{S3AFilesystem.create()}} does not examine its parent paths to verify that there does not exist one which is a file. It looks for the destination path if overwrite=false (see HADOOP-13188 for issues there), but it doesn't check the parent for not being a file, or the parent of that path.
> It must go up the tree, verifying that either a path does not exist, or that the path is a directory. The scan can stop at the first entry which is is a directory, thus the operation is O(empty-directories) and not O(directories).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org