You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/07/12 13:57:00 UTC

[jira] [Created] (HADOOP-16429) DynamoDBMetaStore deleteSubtree to delete leaf nodes first

Steve Loughran created HADOOP-16429:
---------------------------------------

             Summary: DynamoDBMetaStore deleteSubtree to delete leaf nodes first
                 Key: HADOOP-16429
                 URL: https://issues.apache.org/jira/browse/HADOOP-16429
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.3.0
            Reporter: Steve Loughran


in {{deleteSubtree(path)}}, the DynamoDB metastore walks down the tree, returning elements to delete. But it will delete parent entries before children, so if an operation fails partway through, there will be orphans

Better: DescendantsIterator to return all the leaf nodes before their parents so the deletion is done bottom up

Also: push the deletions off into their own async queue/pool so that they don't become the bottleneck on the process



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org