You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2014/04/16 15:24:16 UTC
[jira] [Commented] (HADOOP-10511) s3n:// incorrectly handles URLs
with secret keys that contain a slash
[ https://issues.apache.org/jira/browse/HADOOP-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13971398#comment-13971398 ]
ASF GitHub Bot commented on HADOOP-10511:
-----------------------------------------
GitHub user darabos opened a pull request:
https://github.com/apache/hadoop-common/pull/14
Fix secret key handling in s3n:// (HADOOP-10511)
https://issues.apache.org/jira/browse/HADOOP-10511
I forgot to mention in the commit why I'm adding "example.com" as the default bucket. "s3n:///" is not a valid path, and the URI constructor throws a URISyntaxException for it. I think it is not a great idea anyway to use an invalid path in the tests. I understand the intention of not using something that may be a real bucket, because it could result in accidental damage to production data. But "example.com" is perfect for this, since it cannot exist.
Let me know if you have other questions about the change. Thanks!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/darabos/hadoop-common HADOOP-10511
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hadoop-common/pull/14.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14
----
commit 670926439f7f2abf098fb576f47266564ae50984
Author: Daniel Darabos <da...@gmail.com>
Date: 2014-04-16T13:00:22Z
Unit test demonstrating HADOOP-10511. It fails with "expected:<s3n://key:pass/word@example.com/test> but was:<s3n://key:pass/test>".
commit d7935231d6a9094678dfe9f917ac4733aeaf864b
Author: Daniel Darabos <da...@gmail.com>
Date: 2014-04-16T13:14:57Z
Fix HADOOP-10511. URI.create() takes an encoded string. But URI.getAuthority() returns a decoded string. Instead of using URI.create() we need to use the URI constructor which takes a set of URI components.
----
> s3n:// incorrectly handles URLs with secret keys that contain a slash
> ---------------------------------------------------------------------
>
> Key: HADOOP-10511
> URL: https://issues.apache.org/jira/browse/HADOOP-10511
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Reporter: Daniel Darabos
>
> This is similar to HADOOP-3733, but happens on s3n:// instead of s3://.
> Essentially if I have a path like "s3n://key:pass%2Fword@example.com/test", it will under certain circumstances be replaced with "s3n://key:pass/test" which then causes "Invalid hostname in URI" exceptions.
> I have a unit test and a fix for this. I'll make a pull request in a moment.
--
This message was sent by Atlassian JIRA
(v6.2#6252)