You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Kirby Zhou <ki...@gmail.com> on 2022/04/15 12:07:47 UTC
Re: Review Request 73912: RANGER-3682 Unify the ways that rangerkeystore to encapsulate zonekey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73912/
-----------------------------------------------------------
(Updated 四月 15, 2022, 12:07 p.m.)
Review request for ranger, Bhavik Bavishi, Dhaval Shah, and Mateen Mansoori.
Changes
-------
fix bug of DBToAzureKeyVault.java
replace base64 codec to java.util.base64
Bugs: RANGER-3682
https://issues.apache.org/jira/browse/RANGER-3682
Repository: ranger
Description
-------
Unify the ways that rangerkeystore to encapsulate zonekey
Now we have 2 styles of MasterKeyProvider:
1. RangerMasterKey, RangerHSM, RangerSafenetKeySecure
2. RangerAzureKeyVaultKeyGenerator, RangerGoogleCloudHSMProvider, RangerTencentKMSProvider
Style 1 can get out master key string from provider, Style 2 can not.
In old, I add a flag KeyVaultEnabled to distinguish them. KeyVaultEnabled=false means style1, true means style2
RangerKeyStore with style1 use SecretKeyEntry with SealedObject to store a key and do encryption / decryption by itself.
RangerKeyStore with style2 use SecretKeyByteEntry to store a key and let MK provider to encryption / decryption.
These are ugly and hard to maintain. I refactor it by removing SecretKeyEntry, and let providers of style1 do encryption / decryption.
Add a common base class of RangerMasterKey, RangerHSM andd RangerSafenetKeySecure, named AbstractRangerMasterKey. It provides the common logic of encryptZoneKey and decryptZoneKey.
And, there is no unified method to initialize a master key provider. Duplicate code is distributed in RangerKeyStoreProvider and a bunch of CLI classes.
I made a new RangerKMSMKIFactory class to unify it.
Diffs (updated)
-----
kms/src/main/java/org/apache/hadoop/crypto/key/AbstractRangerMasterKey.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java 39de0a503
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java a1a6f348b
kms/src/main/java/org/apache/hadoop/crypto/key/MigrateDBMKeyToGCP.java d3b717a8a
kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1935a0185
kms/src/main/java/org/apache/hadoop/crypto/key/RangerGoogleCloudHSMProvider.java a61cabb1b
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 90ef729b2
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java b09cd5bad
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKIFactory.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 8dc129069
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java cb5739f61
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java c37e98ee5
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java eb8a90a71
kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsDBMasterkeyCorrect.java 632e728f4
kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsHSMMasterkeyCorrect.java e5ebeb783
kms/src/main/java/org/apache/ranger/kms/biz/RangerKMSStartUp.java aae722b39
kms/src/test/java/org/apache/hadoop/crypto/key/kms/TestRangerKeyStore.java bcdf2e337
kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/RangerMasterKeyTest.java f420322ca
Diff: https://reviews.apache.org/r/73912/diff/3/
Changes: https://reviews.apache.org/r/73912/diff/2-3/
Testing
-------
Tested by fresh install and update.
Thanks,
Kirby Zhou
Re: Review Request 73912: RANGER-3682 Unify the ways that rangerkeystore to encapsulate zonekey
Posted by Abhishek Kumar <ab...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73912/#review225129
-----------------------------------------------------------
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKIFactory.java
Lines 157 (patched)
<https://reviews.apache.org/r/73912/#comment313910>
Since the exception message is the same, this can be written as:
catch (ClassNotFoundException | InstantiationException | InvocationTargetException | IllegalAccessException e){
throw new NoSuchProviderException(e.getMessage());
}
- Abhishek Kumar
On Jan. 17, 2023, 9:34 a.m., Kirby Zhou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73912/
> -----------------------------------------------------------
>
> (Updated Jan. 17, 2023, 9:34 a.m.)
>
>
> Review request for ranger, Bhavik Bavishi, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Madhan Neethiraj, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3682
> https://issues.apache.org/jira/browse/RANGER-3682
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Unify the ways that rangerkeystore to encapsulate zonekey
>
> Now we have 2 styles of MasterKeyProvider:
> 1. RangerMasterKey, RangerHSM, RangerSafenetKeySecure
> 2. RangerAzureKeyVaultKeyGenerator, RangerGoogleCloudHSMProvider, RangerTencentKMSProvider
>
> Style 1 can get out master key string from provider, Style 2 can not.
> In old, I add a flag KeyVaultEnabled to distinguish them. KeyVaultEnabled=false means style1, true means style2
> RangerKeyStore with style1 use SecretKeyEntry with SealedObject to store a key and do encryption / decryption by itself.
> RangerKeyStore with style2 use SecretKeyByteEntry to store a key and let MK provider to encryption / decryption.
> These are ugly and hard to maintain. I refactor it by removing SecretKeyEntry, and let providers of style1 do encryption / decryption.
> Add a common base class of RangerMasterKey, RangerHSM andd RangerSafenetKeySecure, named AbstractRangerMasterKey. It provides the common logic of encryptZoneKey and decryptZoneKey.
> And, there is no unified method to initialize a master key provider. Duplicate code is distributed in RangerKeyStoreProvider and a bunch of CLI classes.
> I made a new RangerKMSMKIFactory class to unify it.
>
>
> Diffs
> -----
>
> kms/src/main/java/org/apache/hadoop/crypto/key/AbstractRangerMasterKey.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java 39de0a503
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java a1a6f348b
> kms/src/main/java/org/apache/hadoop/crypto/key/MigrateDBMKeyToGCP.java d3b717a8a
> kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1935a0185
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerGoogleCloudHSMProvider.java a61cabb1b
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 90ef729b2
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java b09cd5bad
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKIFactory.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 429d1ce45
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java eb8a90a71
> kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsDBMasterkeyCorrect.java 632e728f4
> kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsHSMMasterkeyCorrect.java e5ebeb783
> kms/src/main/java/org/apache/ranger/kms/biz/RangerKMSStartUp.java 8b0f74eac
> kms/src/test/java/org/apache/hadoop/crypto/key/kms/TestRangerKeyStore.java bcdf2e337
> kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/RangerMasterKeyTest.java f420322ca
>
>
> Diff: https://reviews.apache.org/r/73912/diff/4/
>
>
> Testing
> -------
>
> Tested by fresh install and update.
>
>
> Thanks,
>
> Kirby Zhou
>
>
Re: Review Request 73912: RANGER-3682 Unify the ways that rangerkeystore to encapsulate zonekey
Posted by Kirby Zhou <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73912/
-----------------------------------------------------------
(Updated 一月 29, 2023, 1:11 p.m.)
Review request for ranger, Bhavik Bavishi, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Madhan Neethiraj, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Changes
-------
improve exception catch
Bugs: RANGER-3682
https://issues.apache.org/jira/browse/RANGER-3682
Repository: ranger
Description
-------
Unify the ways that rangerkeystore to encapsulate zonekey
Now we have 2 styles of MasterKeyProvider:
1. RangerMasterKey, RangerHSM, RangerSafenetKeySecure
2. RangerAzureKeyVaultKeyGenerator, RangerGoogleCloudHSMProvider, RangerTencentKMSProvider
Style 1 can get out master key string from provider, Style 2 can not.
In old, I add a flag KeyVaultEnabled to distinguish them. KeyVaultEnabled=false means style1, true means style2
RangerKeyStore with style1 use SecretKeyEntry with SealedObject to store a key and do encryption / decryption by itself.
RangerKeyStore with style2 use SecretKeyByteEntry to store a key and let MK provider to encryption / decryption.
These are ugly and hard to maintain. I refactor it by removing SecretKeyEntry, and let providers of style1 do encryption / decryption.
Add a common base class of RangerMasterKey, RangerHSM andd RangerSafenetKeySecure, named AbstractRangerMasterKey. It provides the common logic of encryptZoneKey and decryptZoneKey.
And, there is no unified method to initialize a master key provider. Duplicate code is distributed in RangerKeyStoreProvider and a bunch of CLI classes.
I made a new RangerKMSMKIFactory class to unify it.
Diffs (updated)
-----
kms/src/main/java/org/apache/hadoop/crypto/key/AbstractRangerMasterKey.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java 39de0a503
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java a1a6f348b
kms/src/main/java/org/apache/hadoop/crypto/key/MigrateDBMKeyToGCP.java d3b717a8a
kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1935a0185
kms/src/main/java/org/apache/hadoop/crypto/key/RangerGoogleCloudHSMProvider.java a61cabb1b
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 90ef729b2
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java b09cd5bad
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKIFactory.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 429d1ce45
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java eb8a90a71
kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsDBMasterkeyCorrect.java 632e728f4
kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsHSMMasterkeyCorrect.java e5ebeb783
kms/src/main/java/org/apache/ranger/kms/biz/RangerKMSStartUp.java 8b0f74eac
kms/src/test/java/org/apache/hadoop/crypto/key/kms/TestRangerKeyStore.java bcdf2e337
kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/RangerMasterKeyTest.java f420322ca
Diff: https://reviews.apache.org/r/73912/diff/5/
Changes: https://reviews.apache.org/r/73912/diff/4-5/
Testing
-------
Tested by fresh install and update.
Thanks,
Kirby Zhou
Re: Review Request 73912: RANGER-3682 Unify the ways that rangerkeystore to encapsulate zonekey
Posted by Kirby Zhou <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73912/
-----------------------------------------------------------
(Updated 一月 17, 2023, 9:34 a.m.)
Review request for ranger, Bhavik Bavishi, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Madhan Neethiraj, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Changes
-------
Rebase to HEAD
Bugs: RANGER-3682
https://issues.apache.org/jira/browse/RANGER-3682
Repository: ranger
Description
-------
Unify the ways that rangerkeystore to encapsulate zonekey
Now we have 2 styles of MasterKeyProvider:
1. RangerMasterKey, RangerHSM, RangerSafenetKeySecure
2. RangerAzureKeyVaultKeyGenerator, RangerGoogleCloudHSMProvider, RangerTencentKMSProvider
Style 1 can get out master key string from provider, Style 2 can not.
In old, I add a flag KeyVaultEnabled to distinguish them. KeyVaultEnabled=false means style1, true means style2
RangerKeyStore with style1 use SecretKeyEntry with SealedObject to store a key and do encryption / decryption by itself.
RangerKeyStore with style2 use SecretKeyByteEntry to store a key and let MK provider to encryption / decryption.
These are ugly and hard to maintain. I refactor it by removing SecretKeyEntry, and let providers of style1 do encryption / decryption.
Add a common base class of RangerMasterKey, RangerHSM andd RangerSafenetKeySecure, named AbstractRangerMasterKey. It provides the common logic of encryptZoneKey and decryptZoneKey.
And, there is no unified method to initialize a master key provider. Duplicate code is distributed in RangerKeyStoreProvider and a bunch of CLI classes.
I made a new RangerKMSMKIFactory class to unify it.
Diffs (updated)
-----
kms/src/main/java/org/apache/hadoop/crypto/key/AbstractRangerMasterKey.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java 39de0a503
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java a1a6f348b
kms/src/main/java/org/apache/hadoop/crypto/key/MigrateDBMKeyToGCP.java d3b717a8a
kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1935a0185
kms/src/main/java/org/apache/hadoop/crypto/key/RangerGoogleCloudHSMProvider.java a61cabb1b
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 90ef729b2
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java b09cd5bad
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKIFactory.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 429d1ce45
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java eb8a90a71
kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsDBMasterkeyCorrect.java 632e728f4
kms/src/main/java/org/apache/hadoop/crypto/key/VerifyIsHSMMasterkeyCorrect.java e5ebeb783
kms/src/main/java/org/apache/ranger/kms/biz/RangerKMSStartUp.java 8b0f74eac
kms/src/test/java/org/apache/hadoop/crypto/key/kms/TestRangerKeyStore.java bcdf2e337
kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/RangerMasterKeyTest.java f420322ca
Diff: https://reviews.apache.org/r/73912/diff/4/
Changes: https://reviews.apache.org/r/73912/diff/3-4/
Testing
-------
Tested by fresh install and update.
Thanks,
Kirby Zhou