You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2020/04/24 03:59:00 UTC
[jira] [Assigned] (IMPALA-9677) FE Analysis tests using fake S3
bucket fail with AnalysisException
[ https://issues.apache.org/jira/browse/IMPALA-9677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe McDonnell reassigned IMPALA-9677:
-------------------------------------
Assignee: Joe McDonnell
> FE Analysis tests using fake S3 bucket fail with AnalysisException
> ------------------------------------------------------------------
>
> Key: IMPALA-9677
> URL: https://issues.apache.org/jira/browse/IMPALA-9677
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: broken-build
>
> It looks like Hadoop added extra validation for initializing the S3AFilesystem. [HADOOP-16711|https://issues.apache.org/jira/browse/HADOOP-16711] now defaults to checking for the existence of the bucket and permissions on it. The level of validation is controlled by fs.s3a.bucket.probe. We use a bogus S3 bucket in our analysis tests with the expectation that all we require is a valid URI. Instead, we get this error:
> {noformat}
> Error during analysis:
> org.apache.impala.common.AnalysisException: bucket: doesBucketExistV2 on bucket: com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 10988D12BA86A604; S3 Extended Request ID: qbfw3pMHGGiPTZyLiAOIAvMlJLpy5HOgkHSN4I1n1pBhX2H3BYSwDXdt+gIi3+xiA+E2w5Itv5o=), S3 Extended Request ID: qbfw3pMHGGiPTZyLiAOIAvMlJLpy5HOgkHSN4I1n1pBhX2H3BYSwDXdt+gIi3+xiA+E2w5Itv5o=:InvalidAccessKeyId
> sql:
> create table tbl (i int) location 's3a://bucket/test-warehouse/new_table'{noformat}
> Developers frequently don't have S3 credentials set up, so this will always fail.
> Currently, these tests fails:
> {noformat}
> org.apache.impala.analysis.AnalyzeDDLTest.TestAlterTableAddDropPartition
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateDataSource
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateDb
> org.apache.impala.analysis.AnalyzeDDLTest.TestAlterTableSet
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTableLike
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTable
> org.apache.impala.planner.ExplainTest.testScanNodeFsScheme
> org.apache.impala.planner.PlannerTest.testScanNodeFsScheme{noformat}
> It reproduces consistently on jenkins.impala.io: [https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/69/]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org