You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/04/28 19:52:00 UTC
[jira] [Commented] (HADOOP-15428) s3guard bucket-info -unguarded
will guard bucket if FS is set to do this automatically
[ https://issues.apache.org/jira/browse/HADOOP-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457775#comment-16457775 ]
Steve Loughran commented on HADOOP-15428:
-----------------------------------------
{code}
> hadoop s3guard bucket-info -unguarded s3a://hwdev-steve-ireland-new/
18/04/28 19:49:05 INFO s3guard.DynamoDBMetadataStore: Creating non-existent DynamoDB table hwdev-steve-ireland-new in region eu-west-1
Filesystem s3a://hwdev-steve-ireland-new
Location: eu-west-1
Filesystem s3a://hwdev-steve-ireland-new is using S3Guard with store DynamoDBMetadataStore{region=eu-west-1, tableName=hwdev-steve-ireland-new}
Authoritative S3Guard: fs.s3a.metadatastore.authoritative=false
Metadata Store Diagnostics:
ARN=arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new
description=S3Guard metadata store in DynamoDB
name=hwdev-steve-ireland-new
read-capacity=100
region=eu-west-1
retryPolicy=ExponentialBackoffRetry(maxRetries=9, sleepTime=100 MILLISECONDS)
size=0
status=ACTIVE
table={AttributeDefinitions: [{AttributeName: child,AttributeType: S}, {AttributeName: parent,AttributeType: S}],TableName: hwdev-steve-ireland-new,KeySchema: [{AttributeName: parent,KeyType: HASH}, {AttributeName: child,KeyType: RANGE}],TableStatus: ACTIVE,CreationDateTime: Sat Apr 28 19:49:05 UTC 2018,ProvisionedThroughput: {NumberOfDecreasesToday: 0,ReadCapacityUnits: 100,WriteCapacityUnits: 100},TableSizeBytes: 0,ItemCount: 0,TableArn: arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new,TableId: 7e9bb285-6147-4fc1-8703-fdd3428f8bf5,}
write-capacity=100
The "magic" committer is not supported
S3A Client
Endpoint: fs.s3a.endpoint=(unset)
Encryption: fs.s3a.server-side-encryption-algorithm=none
Input seek policy: fs.s3a.experimental.input.fadvise=normal
18/04/28 19:49:18 INFO util.ExitUtil: Exiting with status 46: 46: S3Guard is enabled for s3a://hwdev-steve-ireland-new
{code}
> s3guard bucket-info -unguarded will guard bucket if FS is set to do this automatically
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-15428
> URL: https://issues.apache.org/jira/browse/HADOOP-15428
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.1.0
> Reporter: Steve Loughran
> Priority: Major
>
> If you call hadoop s3guard bucket-info on a bucket where the fs is set to create a s3guard table on demand, then the DDB table is automatically created. As a result
> the {{bucket-info -unguarded}} option cannot be used, and the call has significant side effects (i.e. it can run up bills)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org