[jira] [Updated] (HADOOP-13736) Change PathMetadata to hold S3AFileStatus instead of FileStatus.

["Lei (Eddy) Xu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lei (Eddy) Xu updated HADOOP-13736:
-----------------------------------
    Attachment: HADOOP-13736.wip-01.patch

Upload a working in progress patch to make {{PathMetadata}}, {{DirListingMetadata}} and {{MetadataStore}} as template, and only implements them using {{S3AFileStatus}} in {{s3guard}}.  So it leaves the room to share the metadata store code with other projects, while it still enforces that the metadata store implementation in s3guard can directly write in such way:

{code}
public class DynamoDBMetadataStore implements MetadataStore<S3AFileStatus> {
....
{code}

[~liuml07] and [~fabbri] could you please take a look whether it makes sense to your implementations respectively?  I attached a {{DummyMS}} and its contract tests in this patch to show the basic usage. 

[~stevel@apache.org] Thanks for the inputs. HDFS-6984 would be nice to have here, so that we can just store protobuf bytes in dynamo. However the difficulty still exists when it comes to deserialize these bytes, it needs to know the concret FileStatus class.


> Change PathMetadata to hold S3AFileStatus instead of FileStatus.
> ----------------------------------------------------------------
>
>                 Key: HADOOP-13736
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13736
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>         Attachments: HADOOP-13736.wip-01.patch
>
>
> {{S3AFileStatus}} is implemented differently with {{FileStatus}}, for instance {{S3AFileStatus#isEmptyDirectory()}} is not implemented in {{FileStatus()}}. And {{access_time}}, {{block_replication}}, {{owner}}, {{group}} and a few other fields are not meaningful in {{S3AFileStatus}}.  
> So in the scope of {{S3guard}}, it should use {{S3AFileStatus}} in  instead of {{FileStatus}} in {{PathMetadaa}} to avoid casting the types back and forth in S3A. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org