You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2016/06/27 21:10:18 UTC
[jira] [Updated] (HADOOP-13324) s3a doesn't authenticate with S3
frankfurt (or other V4 auth only endpoints)
[ https://issues.apache.org/jira/browse/HADOOP-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-13324:
------------------------------------
Attachment: HADOOP-13324-branch-2-001.patch
Docs in detail, including how to interpret some of the ways endpoint problems can surface (400 or 301).
Tests did need tuning to work here, because if you switch your {{fs.s3a.endpoint=eu-central}} then the scale tests against the s3a://landsat bucket fail. Added logic in the test runner to switch the test FS back to us-west when the default CSV file is used, and an option to set a different endpoint otherwise (or we could just leave it as fs.s3a.endpoint by default, assuming if you change the CSV file then you'd better use one at the current endpoint).
All documented in the testing section, which I moved around slightly. I think the aws doc page is now way too big and could/should be split up. Separate issue there, perhaps
Testing: S3 frankfurt. The s3n:// and s3:// tests fail, and were skipped
> s3a doesn't authenticate with S3 frankfurt (or other V4 auth only endpoints)
> ----------------------------------------------------------------------------
>
> Key: HADOOP-13324
> URL: https://issues.apache.org/jira/browse/HADOOP-13324
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-13324-branch-2-001.patch
>
>
> S3A doesn't auth with S3 frankfurt. This installation only supports v4 API.
> There are some JVM options which should set this, but even they don't appear to be enough. It appears that we have to allow the s3a client to change the endpoint with which it authenticates from a generic "AWS S3" to a frankfurt-specific one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org