You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Mingliang Liu (JIRA)" <ji...@apache.org> on 2016/11/08 02:19:58 UTC
[jira] [Comment Edited] (HADOOP-13650) S3Guard: Provide command
line tools to manipulate metadata store.
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646188#comment-15646188 ]
Mingliang Liu edited comment on HADOOP-13650 at 11/8/16 2:19 AM:
-----------------------------------------------------------------
{{S3AFileSystem#initialize(name, conf)}}, name may have the user:pass login info for creating the credentials of AWS client. Also the S3AFS is used to get the bucket name and region. This may bring extra work if we only support initializing {{DDBMetadataStore}} via Configuration parameter only. Let's figure out this later; by now let's rely on the S3AFS to initialize the DDBMetadataStore.
# {{s3a s3guard init -m dynamodb:// s3a://bucket/path}} will use the bucket name as the table name
# {{s3a s3guard init -m dynamodb://mytable s3a://bucket/path}}; we will set the config key "fs.s3a.s3guard.ddb.table" in the configuration before initializing the S3AFS. The DDBMetadataStore will prefer this config if provided to the default bucket name.
Either way, the fully qualified S3A uri/name is needed before we can initialize the MetadataStore from configuration.
Thanks,
was (Author: liuml07):
{{S3AFileSystem#initialize(name, conf)}}, name may have the user:pass login info for creating the credentials of AWS client. Also the S3AFS is used to get the bucket name and region. This may bring extra work if we only support initializing {{DDBMetadataStore}} via Configuration parameter only. Let's figure out this later; by now let's rely on the S3AFS to initialize the DDBMetadataStore.
> S3Guard: Provide command line tools to manipulate metadata store.
> -----------------------------------------------------------------
>
> Key: HADOOP-13650
> URL: https://issues.apache.org/jira/browse/HADOOP-13650
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-13650-HADOOP-13345.000.patch, HADOOP-13650-HADOOP-13345.001.patch
>
>
> Similar systems like EMRFS has the CLI tools to manipulate the metadata store, i.e., create or delete metadata store, or {{import}}, {{sync}} the file metadata between metadata store and S3.
> http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emrfs-cli-reference.html
> S3Guard should offer similar functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org