You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Ankita Sinha <an...@freestoneinfotech.com> on 2016/03/14 05:37:19 UTC
Review Request 44757: Add support for Hardware Security Modules (HSM)
to Ranger
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44757/
-----------------------------------------------------------
Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
Bugs: RANGER-868
https://issues.apache.org/jira/browse/RANGER-868
Repository: ranger
Description
-------
** Problem Statement **
1. Ranger KMS needs to have a option of saving Master Key in HSM.
2. Ranger KMS need to support HSM HA.
3. Ranger KMS needs to have functionality of migrating Master Key to HSM from Ranger KMS DB and vice versa.
** Proposed Solution **
1. To give option to Store Ranger KMS Master Key to either DB/HSM.
2. Create a new Provider in Ranger KMS to support HSM.
3. Develop Migration script for migrating Ranger KMS Master Key from HSM to Ranger KMS DB and vice versa.
Diffs
-----
kms/config/kms-webapp/dbks-site.xml edaff93
kms/scripts/DBMK2HSM.sh PRE-CREATION
kms/scripts/HSMMK2DB.sh PRE-CREATION
kms/scripts/install.properties cf5dd92
kms/scripts/setup.sh 0a825c7
kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 23547a7
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 75a34b2
src/main/assembly/kms.xml e267687
Diff: https://reviews.apache.org/r/44757/diff/
Testing
-------
** Testing Done **
1. Tested Ranger KMS with HSM enabled as well as disabled.
2. Tested Ranger KMS with HSM in secure environment.
3. Tested Ranger KMS in HSM HA mode.
4. Tested migration script for migrating Master Key from Ranger KMS DB to HSM.
5. Tested migration script for migrating Master Key from HSM to Ranger KMS DB.
6. Tested for all the Key operations (create, delete, rollover and list) through UI, CURL and hadoop command.
7. Tested for Zone operations related operation.
8. Tested for Copying file from one Zone to another.
Thanks,
Ankita Sinha
Re: Review Request 44757: Add support for Hardware Security Modules
(HSM) to Ranger
Posted by Velmurugan Periasamy <vp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44757/#review128702
-----------------------------------------------------------
Ship it!
Ship It!
- Velmurugan Periasamy
On April 13, 2016, 2:16 p.m., Ankita Sinha wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44757/
> -----------------------------------------------------------
>
> (Updated April 13, 2016, 2:16 p.m.)
>
>
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-868
> https://issues.apache.org/jira/browse/RANGER-868
>
>
> Repository: ranger
>
>
> Description
> -------
>
> ** Problem Statement **
> 1. Ranger KMS needs to have a option of saving Master Key in HSM.
> 2. Ranger KMS need to support HSM HA.
> 3. Ranger KMS needs to have functionality of migrating Master Key to HSM from Ranger KMS DB and vice versa.
>
> ** Proposed Solution **
> 1. To give option to Store Ranger KMS Master Key to either DB/HSM.
> 2. Create a new Provider in Ranger KMS to support HSM.
> 3. Develop Migration script for migrating Ranger KMS Master Key from HSM to Ranger KMS DB and vice versa.
>
>
> Diffs
> -----
>
> kms/config/kms-webapp/dbks-site.xml edaff93
> kms/scripts/DBMK2HSM.sh PRE-CREATION
> kms/scripts/HSMMK2DB.sh PRE-CREATION
> kms/scripts/install.properties d30b28c
> kms/scripts/setup.sh 64abcc7
> kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java a9e43fc
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 75a34b2
> src/main/assembly/kms.xml e267687
>
> Diff: https://reviews.apache.org/r/44757/diff/
>
>
> Testing
> -------
>
> ** Testing Done **
> 1. Tested Ranger KMS with HSM enabled as well as disabled.
> 2. Tested Ranger KMS with HSM in secure environment.
> 3. Tested Ranger KMS in HSM HA mode.
> 4. Tested migration script for migrating Master Key from Ranger KMS DB to HSM.
> 5. Tested migration script for migrating Master Key from HSM to Ranger KMS DB.
> 6. Tested for all the Key operations (create, delete, rollover and list) through UI, CURL and hadoop command.
> 7. Tested for Zone operations related operation.
> 8. Tested for Copying file from one Zone to another.
>
>
> Thanks,
>
> Ankita Sinha
>
>
Re: Review Request 44757: Add support for Hardware Security Modules
(HSM) to Ranger
Posted by Ankita Sinha <an...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44757/
-----------------------------------------------------------
(Updated April 13, 2016, 2:16 p.m.)
Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
Changes
-------
Updated the Patch to get applied on latest Master Branch
Bugs: RANGER-868
https://issues.apache.org/jira/browse/RANGER-868
Repository: ranger
Description
-------
** Problem Statement **
1. Ranger KMS needs to have a option of saving Master Key in HSM.
2. Ranger KMS need to support HSM HA.
3. Ranger KMS needs to have functionality of migrating Master Key to HSM from Ranger KMS DB and vice versa.
** Proposed Solution **
1. To give option to Store Ranger KMS Master Key to either DB/HSM.
2. Create a new Provider in Ranger KMS to support HSM.
3. Develop Migration script for migrating Ranger KMS Master Key from HSM to Ranger KMS DB and vice versa.
Diffs (updated)
-----
kms/config/kms-webapp/dbks-site.xml edaff93
kms/scripts/DBMK2HSM.sh PRE-CREATION
kms/scripts/HSMMK2DB.sh PRE-CREATION
kms/scripts/install.properties d30b28c
kms/scripts/setup.sh 64abcc7
kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java PRE-CREATION
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java a9e43fc
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 75a34b2
src/main/assembly/kms.xml e267687
Diff: https://reviews.apache.org/r/44757/diff/
Testing
-------
** Testing Done **
1. Tested Ranger KMS with HSM enabled as well as disabled.
2. Tested Ranger KMS with HSM in secure environment.
3. Tested Ranger KMS in HSM HA mode.
4. Tested migration script for migrating Master Key from Ranger KMS DB to HSM.
5. Tested migration script for migrating Master Key from HSM to Ranger KMS DB.
6. Tested for all the Key operations (create, delete, rollover and list) through UI, CURL and hadoop command.
7. Tested for Zone operations related operation.
8. Tested for Copying file from one Zone to another.
Thanks,
Ankita Sinha
Re: Review Request 44757: Add support for Hardware Security Modules
(HSM) to Ranger
Posted by Velmurugan Periasamy <vp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44757/#review128635
-----------------------------------------------------------
kms/scripts/setup.sh (line 91)
<https://reviews.apache.org/r/44757/#comment192116>
Not able to apply this patch. Can you please check?
Checking patch kms/scripts/setup.sh...
error: while searching for:
sqlanywhere_core_file=$(get_prop 'sqlanywhere_core_file' $PROPFILE)
cred_keystore_filename=$(eval echo "$(get_prop 'cred_keystore_filename' $PROPFILE)")
KMS_BLACKLIST_DECRYPT_EEK=$(get_prop 'KMS_BLACKLIST_DECRYPT_EEK' $PROPFILE)
DB_HOST="${db_host}"
error: patch failed: kms/scripts/setup.sh:83
error: kms/scripts/setup.sh: patch does not apply
- Velmurugan Periasamy
On March 14, 2016, 4:37 a.m., Ankita Sinha wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44757/
> -----------------------------------------------------------
>
> (Updated March 14, 2016, 4:37 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-868
> https://issues.apache.org/jira/browse/RANGER-868
>
>
> Repository: ranger
>
>
> Description
> -------
>
> ** Problem Statement **
> 1. Ranger KMS needs to have a option of saving Master Key in HSM.
> 2. Ranger KMS need to support HSM HA.
> 3. Ranger KMS needs to have functionality of migrating Master Key to HSM from Ranger KMS DB and vice versa.
>
> ** Proposed Solution **
> 1. To give option to Store Ranger KMS Master Key to either DB/HSM.
> 2. Create a new Provider in Ranger KMS to support HSM.
> 3. Develop Migration script for migrating Ranger KMS Master Key from HSM to Ranger KMS DB and vice versa.
>
>
> Diffs
> -----
>
> kms/config/kms-webapp/dbks-site.xml edaff93
> kms/scripts/DBMK2HSM.sh PRE-CREATION
> kms/scripts/HSMMK2DB.sh PRE-CREATION
> kms/scripts/install.properties cf5dd92
> kms/scripts/setup.sh 0a825c7
> kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKMSMKI.java PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 23547a7
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 75a34b2
> src/main/assembly/kms.xml e267687
>
> Diff: https://reviews.apache.org/r/44757/diff/
>
>
> Testing
> -------
>
> ** Testing Done **
> 1. Tested Ranger KMS with HSM enabled as well as disabled.
> 2. Tested Ranger KMS with HSM in secure environment.
> 3. Tested Ranger KMS in HSM HA mode.
> 4. Tested migration script for migrating Master Key from Ranger KMS DB to HSM.
> 5. Tested migration script for migrating Master Key from HSM to Ranger KMS DB.
> 6. Tested for all the Key operations (create, delete, rollover and list) through UI, CURL and hadoop command.
> 7. Tested for Zone operations related operation.
> 8. Tested for Copying file from one Zone to another.
>
>
> Thanks,
>
> Ankita Sinha
>
>