You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/04/13 14:42:00 UTC
[jira] [Work logged] (HADOOP-18201) Remove base and bucket overrides for endpoint in ITestS3ARequesterPays.java
[ https://issues.apache.org/jira/browse/HADOOP-18201?focusedWorklogId=756440&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-756440 ]
ASF GitHub Bot logged work on HADOOP-18201:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Apr/22 14:41
Start Date: 13/Apr/22 14:41
Worklog Time Spent: 10m
Work Description: dannycjones opened a new pull request, #4169:
URL: https://github.com/apache/hadoop/pull/4169
### Description of PR
Requester pays was added in #3962. The new tests remove overrides for requester pays enablement but do not account for developers changing the S3 endpoint.
To address this, we remove the override for endpoint.
Addresses HADOOP-18201.
### How was this patch tested?
Patch will be tested against both `eu-west-1` and `af-south-1`.
Issue Time Tracking
-------------------
Worklog Id: (was: 756440)
Remaining Estimate: 0h
Time Spent: 10m
> Remove base and bucket overrides for endpoint in ITestS3ARequesterPays.java
> ---------------------------------------------------------------------------
>
> Key: HADOOP-18201
> URL: https://issues.apache.org/jira/browse/HADOOP-18201
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.3.3
> Reporter: Mehakmeet Singh
> Assignee: Daniel Carl Jones
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If a user has set the endpoint for their bucket in a test environment, we should ignore that in a test that won't use the bucket that the endpoint is set for. In this case, we are using "s3a://usgs-landsat/" which is in the region us-west-2, and would fail if the user has explicitly set the endpoint to something else.
> Example (I have set the endpoint to ap-south-1):
> {code:java}
> [ERROR] testRequesterPaysDisabledFails(org.apache.hadoop.fs.s3a.ITestS3ARequesterPays) Time elapsed: 9.323 s <<< ERROR!
> org.apache.hadoop.fs.s3a.AWSRedirectException: getFileStatus on s3a://usgs-landsat/collection02/catalog.json: com.amazonaws.services.s3.model.AmazonS3Exception: The bucket is in this region: us-west-2. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: Z09V8PMEEN5PHDRZ; S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=; Proxy: null), S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=:301 Moved Permanently: The bucket is in this region: us-west-2. Please use this region to retry the request (Service: Amazon S3; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: Z09V8PMEEN5PHDRZ; S3 Extended Request ID: B7KDQntCuVmLJAyXvuY4UNXjdUrgn3xd26n8u7ThueNNxvKas6g3RsXo7oxBcvHrpcous2L+Lbk=; Proxy: null)
> at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:233)
> at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:171)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3440)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3346)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.extractOrFetchSimpleFileStatus(S3AFileSystem.java:4890)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$executeOpen$6(S3AFileSystem.java:1437)
> at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:543)
> at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:524)
> at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:445)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.executeOpen(S3AFileSystem.java:1435)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.open(S3AFileSystem.java:1409)
>
> <......>{code}
> CC: [~stevel@apache.org] [~mthakur] [~dannycjones]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org