You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2016/12/04 11:22:58 UTC
[jira] [Commented] (HADOOP-13863) Hadoop - Azure: Add a new SAS key
mode for WASB.
[ https://issues.apache.org/jira/browse/HADOOP-13863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15719783#comment-15719783 ]
Steve Loughran commented on HADOOP-13863:
-----------------------------------------
# any link for more info on SAS keys?
# this new storage interface: is it going to be a think layer of extra auth, or a copy & paste of the existing code? Because C&P is something to absolutely avoid...if it can't be done without some changes for better extensibility in the NativeAzureStorage classes, I'd prefer adding those extension points.
# testing. The local one works with today's keys, so can be run on anyone's desktop? If so, that's essential. It does mean that the other codepath, the httpclient stuff, is going to have to be tested in the release process by volunteers with access to that feature
S3 has a similar problem there and its IAM credential auth on EC2 machines. We trust AWS to test their HTTP client, but even there, in HADOOP-13727, we had to do some patching to deal with how IAM/AWS throttled clients —the kind of problem which we didn't see during our integration tests, even running in-EC2. It'd be good to make sure that the Azure client avoids the same problem
> Hadoop - Azure: Add a new SAS key mode for WASB.
> ------------------------------------------------
>
> Key: HADOOP-13863
> URL: https://issues.apache.org/jira/browse/HADOOP-13863
> Project: Hadoop Common
> Issue Type: Improvement
> Components: azure, fs/azure
> Affects Versions: 2.8.0
> Reporter: Dushyanth
> Assignee: Dushyanth
> Attachments: WASB-SAS Key Mode-Design Proposal.pdf
>
>
> Current implementation of WASB, only supports Azure storage keys and SAS key being provided via org.apache.hadoop.conf.Configuration, which results in these secrets residing in the same address space as the WASB process and providing complete access to the Azure storage account and its containers. Added to the fact that WASB does not inherently support ACL's, WASB is its current implementation cannot be securely used for environments like secure hadoop cluster. This JIRA is created to add a new mode in WASB, which operates on Azure Storage SAS keys, which can provide fine grained timed access to containers and blobs, providing a segway into supporting WASB for secure hadoop cluster.
> More details about the issue and the proposal are provided in the design proposal document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org