You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Kai Zheng (JIRA)" <ji...@apache.org> on 2017/12/04 02:57:00 UTC
[jira] [Comment Edited] (HADOOP-15038) Abstract MetadataStore in
S3Guard into a common module.
[ https://issues.apache.org/jira/browse/HADOOP-15038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276234#comment-16276234 ]
Kai Zheng edited comment on HADOOP-15038 at 12/4/17 2:56 AM:
-------------------------------------------------------------
Thanks for the discussion, folks.
The idea to extract MetadataStore into a Hadoop common module sounds good to me, but when do it, we should be careful not to introduce non-trivial dependencies.
[~stevel@apache.org] it looks to me the cloudup package would be a good fit for the new module. Could you introduce the new module when do the work of HADOOP-14766? If nice, how to name the module (under hadoop-common-project)? IMO hadoop-cloud-common might be better, sure hadoop-cloud-core is also good to me.
[~fabbri] do you have any concerns if the main work of MetadataStore is extracted and put into hadoop-common-project/hadoop-cloud-common?
Not sure if [~chris.douglas] would have more thoughts on this.
Thanks!
was (Author: drankye):
Thanks for the discussion, folks.
The idea to extract MetadataStore into a Hadoop common module sounds good to me, but when do it, we should be careful not to introduce non-trivial dependencies.
[~stevel@apache.org] it looks to me the cloudup package would be a good fit for the new module. Could you introduce the new module when do the work of HADOOP-14766? If nice, how to name the module (under hadoop-common-project)? IMO hadoop-cloud-common might be better, sure hadoop-cloud-core is also good to me.
[~ajfabbri] do you have any concerns if the main work of MetadataStore is extracted and put into hadoop-common-project/hadoop-cloud-common?
Not sure if [~chris.douglas] would have more thoughts on this.
Thanks!
> Abstract MetadataStore in S3Guard into a common module.
> -------------------------------------------------------
>
> Key: HADOOP-15038
> URL: https://issues.apache.org/jira/browse/HADOOP-15038
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs
> Affects Versions: 3.0.0-beta1
> Reporter: Genmao Yu
>
> Open this JIRA to discuss if we should move {{MetadataStore}} in {{S3Guard}} into a common module.
> Based on this work, other filesystem or object store can implement their own metastore for optimization (known issues like consistency problem and metadata operation performance). [~stevel@apache.org] and other guys have done many base and great works in {{S3Guard}}. It is very helpful to start work. I did some perf test in HADOOP-14098, and started related work for Aliyun OSS. Indeed there are still works to do for {{S3Guard}}, like metadata cache inconsistent with S3 and so on. It also will be a problem for other object store. However, we can do these works in parallel.
> [~stevel@apache.org] [~fabbri] [~drankye] Any suggestion is appreciated.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org