You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/08/28 14:55:16 UTC
[GitHub] [hadoop] steveloughran commented on pull request #2254: HADOOP-17227. Marker Tool tuning
steveloughran commented on pull request #2254:
URL: https://github.com/apache/hadoop/pull/2254#issuecomment-682660636
bucket info gets improved too, based on working on the CLI.
```
bin/hadoop s3guard bucket-info s3a://stevel-london/
2020-08-28 15:45:02,624 [main] INFO impl.DirectoryPolicyImpl (DirectoryPolicyImpl.java:getDirectoryPolicy(193)) - Directory markers will be kept on authoritative paths
Filesystem s3a://stevel-london
Location: eu-west-2
Filesystem s3a://stevel-london is using S3Guard with store DynamoDBMetadataStore{region=eu-west-2, tableName=stevel-london, tableArn=arn:aws:dynamodb:eu-west-2:152813717728:table/stevel-london}
Authoritative Metadata Store: fs.s3a.metadatastore.authoritative=false
Authoritative Path: fs.s3a.authoritative.path=/tables
Qualified Authoritative Paths:
s3a://stevel-london/tables/
Metadata time to live: (set in fs.s3a.metadatastore.metadata.ttl) = 00:15:00.000
Metadata Store Diagnostics:
ARN=arn:aws:dynamodb:eu-west-2:152813717728:table/stevel-london
billing-mode=per-request
description=S3Guard metadata store in DynamoDB
name=stevel-london
persist.authoritative.bit=true
read-capacity=0
region=eu-west-2
retryPolicy=ExponentialBackoffRetry(maxRetries=9, sleepTime=100 MILLISECONDS)
size=38538
sse=DISABLED
status=ACTIVE
table={AttributeDefinitions: [{AttributeName: child,AttributeType: S}, {AttributeName: parent,AttributeType: S}],TableName: stevel-london,KeySchema: [{AttributeName: parent,KeyType: HASH}, {AttributeName: child,KeyType: RANGE}],TableStatus: ACTIVE,CreationDateTime: Mon Mar 16 20:21:32 GMT 2020,ProvisionedThroughput: {NumberOfDecreasesToday: 0,ReadCapacityUnits: 0,WriteCapacityUnits: 0},TableSizeBytes: 38538,ItemCount: 300,TableArn: arn:aws:dynamodb:eu-west-2:152813717728:table/stevel-london,TableId: 422878eb-823c-4071-826a-3746b7c8fd18,BillingModeSummary: {BillingMode: PAY_PER_REQUEST,LastUpdateToPayPerRequestDateTime: Mon Mar 16 20:21:32 GMT 2020},}
write-capacity=0
S3A Client
Signing Algorithm: fs.s3a.signing-algorithm=(unset)
Endpoint: fs.s3a.endpoint=s3.eu-west-2.amazonaws.com
Encryption: fs.s3a.server-side-encryption-algorithm=none
Input seek policy: fs.s3a.experimental.input.fadvise=normal
Change Detection Source: fs.s3a.change.detection.source=etag
Change Detection Mode: fs.s3a.change.detection.mode=server
S3A Committers
The "magic" committer is supported in the filesystem
S3A Committer factory class: mapreduce.outputcommitter.factory.scheme.s3a=org.apache.hadoop.fs.s3a.commit.S3ACommitterFactory
S3A Committer name: fs.s3a.committer.name=directory
Cluster filesystem staging directory: fs.s3a.committer.staging.tmp.path=tmp/staging
Local filesystem buffer directory: fs.s3a.buffer.dir=/tmp/hadoop-stevel/s3a
File conflict resolution: fs.s3a.committer.staging.conflict-mode=append
Security
Delegation token support is disabled
Security
The directory marker policy is "authoritative"
Available Policies: delete, keep, authoritative
Authoritative paths: fs.s3a.authoritative.path=/tables
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org