[jira] [Commented] (HADOOP-14124) S3AFileSystem silently deletes "fake" directories when writing a file.

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988565#comment-15988565 ] 

Steve Loughran commented on HADOOP-14124:
-----------------------------------------

revisiting this. S3A FS should recreate a fake directory {{/job/task}}, when {{job/task/file}} is deleted, and there were no other entries under {{job/task}}. Look at the method {{S3AFileSystem.createFakeDirectoryIfNecessary()}}

> S3AFileSystem silently deletes "fake" directories when writing a file.
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-14124
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14124
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, fs/s3
>    Affects Versions: 2.6.0
>            Reporter: Joel Baranick
>              Labels: filesystem, s3
>
> I realize that you guys probably have a good reason for {{S3AFileSystem}} to cleanup "fake" folders when a file is written to S3.  That said, that fact that it silently does this feels like a separation of concerns issue.  It also leads to weird behavior issues where calls to {{AmazonS3Client.getObjectMetadata}} for folders work before calling {{S3AFileSystem.create}} but not after.  Also, there seems to be no mention in the javadoc that the {{deleteUnnecessaryFakeDirectories}} method is automatically invoked. Lastly, it seems like the goal of {{FileSystem}} should be to ensure that code built on top of it is portable to different implementations.  This behavior is an example of a case where this can break down.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org