You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/04/14 14:16:44 UTC
[GitHub] [hadoop] steveloughran commented on issue #1820: HADOOP-16830. Add
public IOStatistics API + S3A implementation
steveloughran commented on issue #1820: HADOOP-16830. Add public IOStatistics API + S3A implementation
URL: https://github.com/apache/hadoop/pull/1820#issuecomment-613469465
* tweak of the EmptyIOStatistics.
We do need tests of the dynamic stuff next
* S3A stats -findbugs changed to turn off complaints about volatile
the change to the builder API needed to wire up AWS stats collection is broken regarding region signing calculation:
```
[ERROR] testSelectNothing(org.apache.hadoop.fs.s3a.select.ITestS3SelectCLI) Time elapsed: 2.059 s <<< ERROR!
org.apache.hadoop.fs.s3a.AWSBadRequestException: getFileStatus on s3a://landsat-pds/scene_list.gz: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 9BDEA365F9EE00C9; S3 Extended Request ID: oeSi4G7orAk2b3ogGQp9ydQf6a0ktIuQPFKKQ+tox6OngBliI6NNPcZTIPLI37U21ffivz/HDw8=), S3 Extended Request ID: oeSi4G7orAk2b3ogGQp9ydQf6a0ktIuQPFKKQ+tox6OngBliI6NNPcZTIPLI37U21ffivz/HDw8=:400 Bad Request: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 9BDEA365F9EE00C9; S3 Extended Request ID: oeSi4G7orAk2b3ogGQp9ydQf6a0ktIuQPFKKQ+tox6OngBliI6NNPcZTIPLI37U21ffivz/HDw8=)
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:241)
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:168)
at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3014)
at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2932)
at org.apache.hadoop.fs.s3a.S3AFileSystem.extractOrFetchSimpleFileStatus(S3AFileSystem.java:4636)
at org.apache.hadoop.fs.s3a.S3AFileSystem.select(S3AFileSystem.java:4562)
at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$null$24(S3AFileSystem.java:4719)
at org.apache.hadoop.util.LambdaUtils.eval(LambdaUtils.java:52)
at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$openFileWithOptions$25(S3AFileSystem.java:4718)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
```
need to understand more; may want to do an SDK updated too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org