You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Larry McCay (JIRA)" <ji...@apache.org> on 2018/02/15 13:48:00 UTC
[jira] [Comment Edited] (HADOOP-14507) extend per-bucket secret key
config with explicit getPassword() on fs.s3a.$bucket.secret,key
[ https://issues.apache.org/jira/browse/HADOOP-14507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365538#comment-16365538 ]
Larry McCay edited comment on HADOOP-14507 at 2/15/18 1:47 PM:
---------------------------------------------------------------
Hi [~stevel@apache.org] - this looks good.
The per bucket layering looks right to me.
Question: Why use the credential providers for the encryption key as well rather than the key provider API?
The key provider API would assure that the algorithm and key length, etc are proper and also provide key rolling/versioning, and allow for management in one of the available KMS servers as well - rather than just jceks.
Migrating to key provider later may make sense if you want to get this in now. It would however require backward compatible code to remain in this layer for existing jceks stores. Just another layer, I guess.
was (Author: lmccay):
Hi [~stevel@apache.org] - this looks good.
The per bucket layering looks right to me.
Question: Why use the credential providers for the encryption key as well rather than the key provider API?
The key provider API would assure that the algorithm and key length, etc are proper and also provide key rolling/versioning, and allow for management in one of the available KMS servers as well - rather than just jceks.
> extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret,key
> --------------------------------------------------------------------------------------------
>
> Key: HADOOP-14507
> URL: https://issues.apache.org/jira/browse/HADOOP-14507
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.1
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Critical
> Attachments: HADOOP-14507-001.patch, HADOOP-14507-002.patch, HADOOP-14507-003.patch, HADOOP-14507-004.patch, HADOOP-14507-005.patch, HADOOP-14507-006.patch, HADOOP-14507-006.patch, HADOOP-14507-007.patch
>
>
> Per-bucket jceks support turns out to be complex as you have to manage multiple jecks files & configure the client to ask for the right one. This is because we're calling {{Configuration.getPassword{"fs,s3a.secret.key"}}.
> If before that, we do a check for the explict id, key, session key in the properties {{fs.s3a.$bucket.secret}} ( & c), we could have a single JCEKs file with all the secrets for different bucket. You would only need to explicitly point the base config to the secrets file, and the right credentials would be picked up, if set
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org