[jira] [Commented] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609058#comment-16609058 ] 

Steve Loughran commented on HADOOP-14927:
-----------------------------------------

Reviewing this. 

h3.Stack #1, failure in DDB {{initMetadataStore}}

It's failing as the table name hasn't been passed in {{ tableName = conf.getTrimmed(S3GUARD_DDB_TABLE_NAME_KEY);}}

which is happening because at the time {{S3GuardTool.initMetadataStore()}} is invoked, the field {{filesystem}} is null (it wasn't there/inited), and the codepath assumes that when this is the case then the store name is preserved in {{S3GUARD_DDB_TABLE_NAME_KEY}}. But that will only happen if (a) we're unsetting any per-bucket options and (b) a metastore URI was passed in as a command.

If you have set a S3Guard region, then parseDynamoDBRegion() will succeed without attempting to init the FS (no need!), so it gets down to this initMetadataStore without either a created FS or a metastore URI, hence the failure.


h3. Stack #2, FNFE

The second of [~gabor.bota] stack traces is *almost* what we want, though the error string is different. The test is looking for  "s3a://this-bucket-does-not-exist-00000000000"; but the error message {{Bucket this-bucket-does-not-exist-00000000000 does not exist}}

And its kicking off early in the {{initS3AFileSystem}} phase inside {{parseDynamoDBRegion}}. So it is being raised as a failure. The test is expecting that, but the string coming back is wrong so it's being rethrown.

h3. Actions

Ether the test consider all of gabor's failure modes legit, or the codepath in destroy changed to explicitly try to create the FS if one is on the command line, even when the region is specified in a config option.


> ITestS3GuardTool failures in testDestroyNoBucket()
> --------------------------------------------------
>
>                 Key: HADOOP-14927
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14927
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1, 3.0.0-alpha3, 3.1.0
>            Reporter: Aaron Fabbri
>            Assignee: Aaron Fabbri
>            Priority: Minor
>         Attachments: HADOOP-14927.001.patch
>
>
> Hit this when testing for the Hadoop 3.0.0-beta1 RC0.
> {noformat}
> hadoop-3.0.0-beta1-src/hadoop-tools/hadoop-aws$ mvn clean verify -Dit.test="ITestS3GuardTool*" -Dtest=none -Ds3guard -Ddynamo
> ...
> Failed tests: 
>   ITestS3GuardToolDynamoDB>AbstractS3GuardToolTestBase.testDestroyNoBucket:228 Expected an exception, got 0
>   ITestS3GuardToolLocal>AbstractS3GuardToolTestBase.testDestroyNoBucket:228 Expected an exception, got 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org