You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2021/10/26 16:05:00 UTC

[jira] [Created] (HADOOP-17979) Interface EtagFromFileStatus to allow FileStatus subclasses to provide etags

Steve Loughran created HADOOP-17979:
---------------------------------------

             Summary: Interface EtagFromFileStatus to allow FileStatus subclasses to provide etags
                 Key: HADOOP-17979
                 URL: https://issues.apache.org/jira/browse/HADOOP-17979
             Project: Hadoop Common
          Issue Type: New Feature
          Components: fs, fs/azure, fs/s3
    Affects Versions: 3.3.1
            Reporter: Steve Loughran
            Assignee: Steve Loughran


Various objects stores provide etags in their FileStatus implementations

Make these values accessible
* new interface {{EtagFromFileStatus}} to be implemented when provided
* filesystem.md to declare requirements of etags (constant between LIST and HEAD)...
* path capabilities for (a) etag and (b) etags consistent across rename

Add implementation for abfs, later s3a (and google gcs)

This is initially to handle recovery from certain failures in job commit against abfs, but it would allow a cloud-ready version of distcp to track etags of uploaded files, so diff properly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org