You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Aaron Fabbri (JIRA)" <ji...@apache.org> on 2016/12/07 01:56:58 UTC
[jira] [Comment Edited] (HADOOP-13449) S3Guard: Implement
DynamoDBMetadataStore.
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15727382#comment-15727382 ]
Aaron Fabbri edited comment on HADOOP-13449 at 12/7/16 1:55 AM:
----------------------------------------------------------------
If it is MetadataStoreTestBase line 154 "assertEmptyDirs() shown here:
{code}
ms.put(new PathMetadata(makeFileStatus("/da1/db1/fc1", 100)));
assertEmptyDirs("/da1", "/da2", "/da3");
assertDirectorySize("/da1/db1", 1);
{code}
I think we can change that to be
{code}
assertEmptyDirs("/da2", "/da3")
{code}
Why? Because it is not unreasonable to allow a MetadataStore to infer the existence of /da1/db1 from a call to put(/da1/db1/fc1).
In fact, as we see here, it can be a helpful implementation technique when we don't have a cheap way to prefix scan everything in the MetadataStore.
was (Author: fabbri):
If it is MetadataStoreTestBase line 154 "assertEmptyDirs() shown here:
{code}
ms.put(new PathMetadata(makeFileStatus("/da1/db1/fc1", 100)));
assertEmptyDirs("/da1", "/da2", "/da3");
assertDirectorySize("/da1/db1", 1);
{code}
I think we can change that to be
{code}
assertEmptyDirs("/da2", "/da3)
{code}
Why? Because it is not unreasonable to allow a MetadataStore to infer the existence of /da1/db1 from a call to put(/da1/db1/fc1).
In fact, as we see here, it can be a helpful implementation technique when we don't have a cheap way to prefix scan everything in the MetadataStore.
> S3Guard: Implement DynamoDBMetadataStore.
> -----------------------------------------
>
> Key: HADOOP-13449
> URL: https://issues.apache.org/jira/browse/HADOOP-13449
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Mingliang Liu
> Attachments: HADOOP-13449-HADOOP-13345.000.patch, HADOOP-13449-HADOOP-13345.001.patch, HADOOP-13449-HADOOP-13345.002.patch, HADOOP-13449-HADOOP-13345.003.patch, HADOOP-13449-HADOOP-13345.004.patch, HADOOP-13449-HADOOP-13345.005.patch, HADOOP-13449-HADOOP-13345.006.patch, HADOOP-13449-HADOOP-13345.007.patch, HADOOP-13449-HADOOP-13345.008.patch, HADOOP-13449-HADOOP-13345.009.patch, HADOOP-13449-HADOOP-13345.010.patch, HADOOP-13449-HADOOP-13345.011.patch
>
>
> Provide an implementation of the metadata store backed by DynamoDB.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org