You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/07/23 13:54:00 UTC
[jira] [Resolved] (HADOOP-16380) S3A tombstones can confuse empty
directory status
[ https://issues.apache.org/jira/browse/HADOOP-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran resolved HADOOP-16380.
-------------------------------------
Resolution: Fixed
Fixed -thanks for help in testing this
> S3A tombstones can confuse empty directory status
> -------------------------------------------------
>
> Key: HADOOP-16380
> URL: https://issues.apache.org/jira/browse/HADOOP-16380
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Fix For: 3.3.0
>
>
> If S3AFileSystem does an S3 LIST restricted to a single object to see if a directory is empty, and the single entry found has a tombstone marker (either from an inconsistent DDB Table or from an eventually consistent LIST) then it will consider the directory empty, _even if there is 1+ entry which is not deleted_
> We need to make sure the calculation of whether a directory is empty or not is resilient to this, efficiently.
> It surfaces as an issue two places
> * delete(path) (where it may make things worse)
> * rename(src, dest), where a check is made for dest != an empty directory.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org