You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Joe McDonnell (Jira)" <ji...@apache.org> on 2020/04/20 16:24:00 UTC
[jira] [Created] (IMPALA-9677) FE Analysis tests using fake S3
bucket fail with AnalysisException
Joe McDonnell created IMPALA-9677:
-------------------------------------
Summary: FE Analysis tests using fake S3 bucket fail with AnalysisException
Key: IMPALA-9677
URL: https://issues.apache.org/jira/browse/IMPALA-9677
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 4.0
Reporter: Joe McDonnell
It looks like Hadoop added extra validation for initializing the S3AFilesystem. [HADOOP-16711|https://issues.apache.org/jira/browse/HADOOP-16711] now defaults to checking for the existence of the bucket and permissions on it. The level of validation is controlled by fs.s3a.bucket.probe. We use a bogus S3 bucket in our analysis tests with the expectation that all we require is a valid URI. Instead, we get this error:
{noformat}
Error during analysis:
org.apache.impala.common.AnalysisException: bucket: doesBucketExistV2 on bucket: com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 10988D12BA86A604; S3 Extended Request ID: qbfw3pMHGGiPTZyLiAOIAvMlJLpy5HOgkHSN4I1n1pBhX2H3BYSwDXdt+gIi3+xiA+E2w5Itv5o=), S3 Extended Request ID: qbfw3pMHGGiPTZyLiAOIAvMlJLpy5HOgkHSN4I1n1pBhX2H3BYSwDXdt+gIi3+xiA+E2w5Itv5o=:InvalidAccessKeyId
sql:
create table tbl (i int) location 's3a://bucket/test-warehouse/new_table'{noformat}
Developers frequently don't have S3 credentials set up, so this will always fail.
Currently, these tests fails:
{noformat}
org.apache.impala.analysis.AnalyzeDDLTest.TestAlterTableAddDropPartition
org.apache.impala.analysis.AnalyzeDDLTest.TestCreateDataSource
org.apache.impala.analysis.AnalyzeDDLTest.TestCreateDb
org.apache.impala.analysis.AnalyzeDDLTest.TestAlterTableSet
org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTableLike
org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTable
org.apache.impala.planner.ExplainTest.testScanNodeFsScheme
org.apache.impala.planner.PlannerTest.testScanNodeFsScheme{noformat}
It reproduces consistently on jenkins.impala.io: [https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/69/]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org