[jira] [Commented] (HADOOP-16380) S3A non-recursive deletion of root to return false

["Gabor Bota (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886293#comment-16886293 ] 

Gabor Bota commented on HADOOP-16380:
-------------------------------------

The following should be fixed to wrap the findings up fully:
* HADOOP-16383. Pass ITtlTimeProvider instance in initialize method in MetadataStore interface
* HADOOP-16433. Filter expired entries and tombstones when listing with MetadataStore#listChildren

> S3A non-recursive deletion of root to return false
> --------------------------------------------------
>
>                 Key: HADOOP-16380
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16380
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>    Affects Versions: 3.2.0, 3.0.3, 3.3.0, 3.1.2
>            Reporter: Steve Loughran
>            Assignee: Gabor Bota
>            Priority: Major
>
> If S3AFileSystem does an S3 LIST restricted to a single object to see if a directory is empty, and the single entry found has a tombstone marker (either from an inconsistent DDB Table or from an eventually consistent LIST) then it will consider the directory empty, _even if there is 1+ entry which is not deleted_
> We need to make sure the calculation of whether a directory is empty or not is resilient to this, efficiently. 
> It surfaces  as an issue two places
> * delete(path) (where it may make things worse)
> * rename(src, dest), where a check is made for dest != an empty directory.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org