You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2022/04/14 16:03:00 UTC
[jira] [Resolved] (HADOOP-18201) Remove base and bucket overrides for endpoint in ITestS3ARequesterPays.java
[ https://issues.apache.org/jira/browse/HADOOP-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran resolved HADOOP-18201.
-------------------------------------
Fix Version/s: 3.3.4
Resolution: Fixed
> Remove base and bucket overrides for endpoint in ITestS3ARequesterPays.java
> ---------------------------------------------------------------------------
>
> Key: HADOOP-18201
> URL: https://issues.apache.org/jira/browse/HADOOP-18201
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.3.3
> Reporter: Mehakmeet Singh
> Assignee: Daniel Carl Jones
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.4
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> If a user has set the endpoint for their bucket in a test environment, we should ignore that in a test that won't use the bucket that the endpoint is set for. In this case, we are using "s3a://usgs-landsat/" which is in the region us-west-2, and would fail if the user has explicitly set the endpoint to something else.
> Example (I have set the endpoint to ap-south-1):
> {code:java}
> [ERROR] testRequesterPaysDisabledFails(org.apache.hadoop.fs.s3a.ITestS3ARequesterPays) Time elapsed: 9.323 s <<< ERROR!
> org.apache.hadoop.fs.s3a.AWSRedirectException: getFileStatus on s3a://usgs-landsat/collection02/catalog.json: com.amazonaws.services.s3.model.AmazonS3Exception: The bucket is in this region: us-west-2. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: Z09V8PMEEN5PHDRZ; S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=; Proxy: null), S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=:301 Moved Permanently: The bucket is in this region: us-west-2. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: Z09V8PMEEN5PHDRZ; S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=; Proxy: null)
> at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:233)
> at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:171)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3440)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3346)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.extractOrFetchSimpleFileStatus(S3AFileSystem.java:4890)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$executeOpen$6(S3AFileSystem.java:1437)
> at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:543)
> at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:524)
> at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:445)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.executeOpen(S3AFileSystem.java:1435)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:1409)
>
> <......>{code}
> CC: [~stevel@apache.org] [~mthakur] [~dannycjones]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org