[jira] [Commented] (IMPALA-8754) S3 with S3Guard tests encounter "ResourceNotFoundException" from DynamoDB

["Steve Loughran (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914206#comment-16914206 ] 

Steve Loughran commented on IMPALA-8754:
----------------------------------------

DDB table wasn't found 
# the table doesn't exist
# the table does exist, but it is in a different region
S3Guard infers the region of the table to be that of the bucket; if you are reading data from buckets in other regions, the inference will be wrong.

There's some option to fix the table region; {{fs.s3a.s3guard.ddb.region}}.
you need to set this to the region where the table is

> S3 with S3Guard tests encounter "ResourceNotFoundException" from DynamoDB
> -------------------------------------------------------------------------
>
>                 Key: IMPALA-8754
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8754
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>    Affects Versions: Impala 3.3.0
>            Reporter: Joe McDonnell
>            Assignee: Joe McDonnell
>            Priority: Critical
>              Labels: broken-build, flaky
>
> When running tests on s3 with s3guard, various tests can encounter the following error coming from the DynamoDB:
> {noformat}
> E    Query aborted:Disk I/O error on impala-ec2-centos74-m5-4xlarge-ondemand-02c8.vpc.cloudera.com:22002: Failed to open HDFS file s3a://impala-test-uswest2-1/test-warehouse/tpcds.store_sales_parquet/ss_sold_date_sk=2451718/6843d8a91fc5ae1d-88b2af4b00000004_156969840_data.0.parq
> E   Error(2): No such file or directory
> E   Root cause: ResourceNotFoundException: Requested resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX){noformat}
> Tests that have seen this (this is flaky):
>  * TestTpcdsQuery.test_tpcds_count
>  * TestHdfsFdCaching.test_caching_disabled_by_param
>  * TestMtDop.test_compute_stats
>  * TestScanRangeLengths.test_scan_ranges



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org