You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Gian Merlino (JIRA)" <ji...@apache.org> on 2014/06/23 19:19:25 UTC

[jira] [Created] (HADOOP-10737) S3n silent failure on copy, data loss on rename

Gian Merlino created HADOOP-10737:
-------------------------------------

             Summary: S3n silent failure on copy, data loss on rename
                 Key: HADOOP-10737
                 URL: https://issues.apache.org/jira/browse/HADOOP-10737
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs/s3
    Affects Versions: 2.4.0
            Reporter: Gian Merlino


Jets3tNativeFileSystemStore.copy(String, String) handles its exceptions with handleServiceException(String, ServiceException), which behaves like:

1) Throw FileNotFoundException if the exception's error code is NoSuchKey
2) Otherwise, throw IOException if the exception's cause is an IOException
3) Otherwise, LOG.debug a message and throw nothing

So S3 exceptions other than NoSuchKey (like RequestTimeout, ServiceUnavailable) are suppressed. This makes "copy" fail while still returning as if it succeeded. Furthermore since NativeS3FileSystem's "rename" is implemented as a copy followed by a delete, this means "rename" can delete the source key even though the copy has failed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)