You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2016/12/23 00:13:58 UTC

[jira] [Created] (HADOOP-13936) S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation

Rajesh Balamohan created HADOOP-13936:
-----------------------------------------

             Summary: S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation
                 Key: HADOOP-13936
                 URL: https://issues.apache.org/jira/browse/HADOOP-13936
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
            Reporter: Rajesh Balamohan
            Priority: Minor


As a part of {{S3AFileSystem.delete}} operation {{innerDelete}} is invoked, which deletes keys from S3 in batches (default is 1000). But DynamoDB is updated only at the end of this operation. This can cause issues when deleting large number of keys. 

E.g, it is possible to get exception after deleting 1000 keys and in such cases dynamoDB would not be updated. This can cause DynamoDB to go out of sync. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org