You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Kirby Zhou <ki...@gmail.com> on 2022/01/19 02:59:37 UTC
Review Request 73807: Support Ranger KMS integration with TencentKMS
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/
-----------------------------------------------------------
Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Bugs: RANGER-3580
https://issues.apache.org/jira/browse/RANGER-3580
Repository: ranger
Description
-------
Ranger KMS integration with TencentKMS
- This task is to integrate the RANGER KMS Service with TencentKMS.
- To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
```
# Do you use Tencent Cloud KMS?
TENCENT_KMS_ENABLED=true
# MasterKeyID on Tencent Cloud
TENCENT_MASTERKEY_ID=YourKeyID
# Login ID
TENCENT_CLIENT_ID=YourClientLoginId
# Login password
TENCENT_CLIENT_SECRET=YourClientLoginSecret
# Tencent Cloud area, see Tencent Cloud SDK for details.
TENCENT_CLIENT_REGION=ap-beijing
```
Run the setup.sh, It will add the below configs in dbks-site.xml
```
<!--Tencent KMS START-->
<property>
<name>ranger.kms.tencentkms.enabled</name>
<value>false</value>
<description>Flag for Tencent KMS</description>
</property>
<property>
<name>ranger.kms.tencent.client.id</name>
<value></value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret</name>
<value></value>
<description>Tencent Client Secret</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret.alias</name>
<value>ranger.ks.tencent.client.secret</value>
<description>Tencent Client Secret Alias</description>
</property>
<property>
<name>ranger.kms.tencent.client.region</name>
<value>ap-beijing</value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.masterkey.id</name>
<value></value>
<description>Tencent master key name</description>
</property>
<!--Tencent KMS END-->
```
Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
Start the kms service, On start Master Key from TencentKMS should be used.
Diffs
-----
distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
kms/pom.xml 908e8841a4a8c0627622d35486ffe2dac7bbdc61
kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java 19335893a4d875743fcf71b12124b1c40a4ee6e1
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 12d485a36423d1c25a3b59cc8436e0a40863a78f
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java bd85c0d364ad06691a0f5d8f19b5124189261db3
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java d21d32320bbd32e4efbe83c701d11c94dc804765
kms/src/main/java/org/apache/hadoop/crypto/key/TencentKMSZoneKeyGenerator.java PRE-CREATION
kms/src/main/resources/log4j.properties 5cd037a49c670a3a0eec9e85fdcafeeeabff1405
pom.xml c663937eba452321b4e2400cc6d3f528f74596de
Diff: https://reviews.apache.org/r/73807/diff/1/
Testing
-------
Thanks,
Kirby Zhou
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Kirby Zhou <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/
-----------------------------------------------------------
(Updated 一月 20, 2022, 3:29 a.m.)
Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Changes
-------
simplify code and fix typo
Bugs: RANGER-3580
https://issues.apache.org/jira/browse/RANGER-3580
Repository: ranger
Description
-------
Ranger KMS integration with TencentKMS
- This task is to integrate the RANGER KMS Service with TencentKMS.
- To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
```
# Do you use Tencent Cloud KMS?
TENCENT_KMS_ENABLED=true
# MasterKeyID on Tencent Cloud
TENCENT_MASTERKEY_ID=YourKeyID
# Login ID
TENCENT_CLIENT_ID=YourClientLoginId
# Login password
TENCENT_CLIENT_SECRET=YourClientLoginSecret
# Tencent Cloud area, see Tencent Cloud SDK for details.
TENCENT_CLIENT_REGION=ap-beijing
```
Run the setup.sh, It will add the below configs in dbks-site.xml
```
<!--Tencent KMS START-->
<property>
<name>ranger.kms.tencentkms.enabled</name>
<value>false</value>
<description>Flag for Tencent KMS</description>
</property>
<property>
<name>ranger.kms.tencent.client.id</name>
<value></value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret</name>
<value></value>
<description>Tencent Client Secret</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret.alias</name>
<value>ranger.ks.tencent.client.secret</value>
<description>Tencent Client Secret Alias</description>
</property>
<property>
<name>ranger.kms.tencent.client.region</name>
<value>ap-beijing</value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.masterkey.id</name>
<value></value>
<description>Tencent master key name</description>
</property>
<!--Tencent KMS END-->
```
Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
Start the kms service, On start Master Key from TencentKMS should be used.
Diffs (updated)
-----
distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
kms/pom.xml 8350403c76cd3f5a6d80e263f54b766dcf6e62e4
kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
kms/src/main/java/org/apache/hadoop/crypto/key/RangerGoogleCloudHSMProvider.java 666a8c38faa157a79c3c3e3dd00050978b7681da
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
pom.xml 13f9bfdc5a88ffdf8d3502605831059fbb9ad4cc
Diff: https://reviews.apache.org/r/73807/diff/4/
Changes: https://reviews.apache.org/r/73807/diff/3-4/
Testing
-------
+ mvn clean compile test verify
+ Fresh setup
File Attachments (updated)
----------------
0001-add-TencentKMS-as-MasterKeyProvider.patch
https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
Thanks,
Kirby Zhou
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Kirby Zhou <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/
-----------------------------------------------------------
(Updated 一月 19, 2022, 10:40 a.m.)
Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Changes
-------
switch-from-log4j-to-slf4j
Bugs: RANGER-3580
https://issues.apache.org/jira/browse/RANGER-3580
Repository: ranger
Description
-------
Ranger KMS integration with TencentKMS
- This task is to integrate the RANGER KMS Service with TencentKMS.
- To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
```
# Do you use Tencent Cloud KMS?
TENCENT_KMS_ENABLED=true
# MasterKeyID on Tencent Cloud
TENCENT_MASTERKEY_ID=YourKeyID
# Login ID
TENCENT_CLIENT_ID=YourClientLoginId
# Login password
TENCENT_CLIENT_SECRET=YourClientLoginSecret
# Tencent Cloud area, see Tencent Cloud SDK for details.
TENCENT_CLIENT_REGION=ap-beijing
```
Run the setup.sh, It will add the below configs in dbks-site.xml
```
<!--Tencent KMS START-->
<property>
<name>ranger.kms.tencentkms.enabled</name>
<value>false</value>
<description>Flag for Tencent KMS</description>
</property>
<property>
<name>ranger.kms.tencent.client.id</name>
<value></value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret</name>
<value></value>
<description>Tencent Client Secret</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret.alias</name>
<value>ranger.ks.tencent.client.secret</value>
<description>Tencent Client Secret Alias</description>
</property>
<property>
<name>ranger.kms.tencent.client.region</name>
<value>ap-beijing</value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.masterkey.id</name>
<value></value>
<description>Tencent master key name</description>
</property>
<!--Tencent KMS END-->
```
Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
Start the kms service, On start Master Key from TencentKMS should be used.
Diffs
-----
distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
kms/pom.xml 8350403c76cd3f5a6d80e263f54b766dcf6e62e4
kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
pom.xml 13f9bfdc5a88ffdf8d3502605831059fbb9ad4cc
Diff: https://reviews.apache.org/r/73807/diff/3/
Testing
-------
+ mvn clean compile test verify
+ Fresh setup
File Attachments (updated)
----------------
0001-add-TencentKMS-as-MasterKeyProvider.patch
https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
0002-switch-from-log4j-to-slf4j.patch
https://reviews.apache.org/media/uploaded/files/2022/01/19/2f5c0202-c13f-420e-926a-61728f8b041e__0002-switch-from-log4j-to-slf4j.patch
Thanks,
Kirby Zhou
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by bhavik patel <bh...@gmail.com>.
> On Jan. 19, 2022, 7:56 a.m., bhavik patel wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java
> > Lines 26 (patched)
> > <https://reviews.apache.org/r/73807/diff/3/?file=2265348#file2265348line26>
> >
> > please use slf4j logger factory
>
> Kirby Zhou wrote:
> I fixed it with 0002-xxxx.patch
>
> Should I merge the 2 patches into 1?
Yes please merge.
Here at review board we have to upload final single patch.
> On Jan. 19, 2022, 7:56 a.m., bhavik patel wrote:
> > pom.xml
> > Lines 242 (patched)
> > <https://reviews.apache.org/r/73807/diff/3/?file=2265349#file2265349line242>
> >
> > Make sure this Doesn’t pull the any extra and vulnerable jar files
>
> Kirby Zhou wrote:
> % mvn dependency:tree -Dverbose -f tencentcloud-sdk-java-3.1.322.pom
> ...
> [INFO] com.tencentcloudapi:tencentcloud-sdk-java:jar:3.1.322
> [INFO] +- commons-logging:commons-logging:jar:1.2:compile
> [INFO] +- com.squareup.okio:okio:jar:1.12.0:compile
> [INFO] +- com.squareup.okhttp:okhttp:jar:2.7.5:compile
> [INFO] | - (com.squareup.okio:okio:jar:1.6.0:compile - omitted for conflict with 1.12.0)
> [INFO] +- com.google.code.gson:gson:jar:2.2.4:compile
> [INFO] +- javax.xml.bind:jaxb-api:jar:2.3.0:compile
> [INFO] +- com.squareup.okhttp:logging-interceptor:jar:2.7.5:compile
> [INFO] | - (com.squareup.okhttp:okhttp:jar:2.7.5:compile - omitted for duplicate)
> [INFO] - org.ini4j:ini4j:jar:0.5.4:compile
> ...
>
> Is it ok?
Yes. I just want to make sure we are not fetching any vulnerable dependencies.
- bhavik
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/#review223972
-----------------------------------------------------------
On Jan. 20, 2022, 3:29 a.m., Kirby Zhou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73807/
> -----------------------------------------------------------
>
> (Updated Jan. 20, 2022, 3:29 a.m.)
>
>
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3580
> https://issues.apache.org/jira/browse/RANGER-3580
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger KMS integration with TencentKMS
> - This task is to integrate the RANGER KMS Service with TencentKMS.
> - To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
>
> ```
> # Do you use Tencent Cloud KMS?
> TENCENT_KMS_ENABLED=true
> # MasterKeyID on Tencent Cloud
> TENCENT_MASTERKEY_ID=YourKeyID
> # Login ID
> TENCENT_CLIENT_ID=YourClientLoginId
> # Login password
> TENCENT_CLIENT_SECRET=YourClientLoginSecret
> # Tencent Cloud area, see Tencent Cloud SDK for details.
> TENCENT_CLIENT_REGION=ap-beijing
> ```
>
> Run the setup.sh, It will add the below configs in dbks-site.xml
> ```
> <!--Tencent KMS START-->
> <property>
> <name>ranger.kms.tencentkms.enabled</name>
> <value>false</value>
> <description>Flag for Tencent KMS</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.id</name>
> <value></value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret</name>
> <value></value>
> <description>Tencent Client Secret</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret.alias</name>
> <value>ranger.ks.tencent.client.secret</value>
> <description>Tencent Client Secret Alias</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.region</name>
> <value>ap-beijing</value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.masterkey.id</name>
> <value></value>
> <description>Tencent master key name</description>
> </property>
> <!--Tencent KMS END-->
> ```
>
> Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
> Start the kms service, On start Master Key from TencentKMS should be used.
>
>
> Diffs
> -----
>
> distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
> kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
> kms/pom.xml 8350403c76cd3f5a6d80e263f54b766dcf6e62e4
> kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
> kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
> kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerGoogleCloudHSMProvider.java 666a8c38faa157a79c3c3e3dd00050978b7681da
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
> pom.xml 13f9bfdc5a88ffdf8d3502605831059fbb9ad4cc
>
>
> Diff: https://reviews.apache.org/r/73807/diff/4/
>
>
> Testing
> -------
>
> + mvn clean compile test verify
> + Fresh setup
>
>
> File Attachments
> ----------------
>
> 0001-add-TencentKMS-as-MasterKeyProvider.patch
> https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
>
>
> Thanks,
>
> Kirby Zhou
>
>
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Kirby Zhou <ki...@gmail.com>.
> On 一月 19, 2022, 7:56 a.m., bhavik patel wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java
> > Line 36 (original), 37 (patched)
> > <https://reviews.apache.org/r/73807/diff/3/?file=2265347#file2265347line37>
> >
> > is it better option to upadate the class name to RangerAzure*?
> > @dhavalshah any thought on this?
>
> Kirby Zhou wrote:
> KeyVault looks like a gerneral name, so I think should add “Azure" here.
@dhavalshah ?
Any idea?
- Kirby
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/#review223972
-----------------------------------------------------------
On 一月 20, 2022, 3:29 a.m., Kirby Zhou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73807/
> -----------------------------------------------------------
>
> (Updated 一月 20, 2022, 3:29 a.m.)
>
>
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3580
> https://issues.apache.org/jira/browse/RANGER-3580
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger KMS integration with TencentKMS
> - This task is to integrate the RANGER KMS Service with TencentKMS.
> - To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
>
> ```
> # Do you use Tencent Cloud KMS?
> TENCENT_KMS_ENABLED=true
> # MasterKeyID on Tencent Cloud
> TENCENT_MASTERKEY_ID=YourKeyID
> # Login ID
> TENCENT_CLIENT_ID=YourClientLoginId
> # Login password
> TENCENT_CLIENT_SECRET=YourClientLoginSecret
> # Tencent Cloud area, see Tencent Cloud SDK for details.
> TENCENT_CLIENT_REGION=ap-beijing
> ```
>
> Run the setup.sh, It will add the below configs in dbks-site.xml
> ```
> <!--Tencent KMS START-->
> <property>
> <name>ranger.kms.tencentkms.enabled</name>
> <value>false</value>
> <description>Flag for Tencent KMS</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.id</name>
> <value></value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret</name>
> <value></value>
> <description>Tencent Client Secret</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret.alias</name>
> <value>ranger.ks.tencent.client.secret</value>
> <description>Tencent Client Secret Alias</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.region</name>
> <value>ap-beijing</value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.masterkey.id</name>
> <value></value>
> <description>Tencent master key name</description>
> </property>
> <!--Tencent KMS END-->
> ```
>
> Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
> Start the kms service, On start Master Key from TencentKMS should be used.
>
>
> Diffs
> -----
>
> distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
> kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
> kms/pom.xml 8350403c76cd3f5a6d80e263f54b766dcf6e62e4
> kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
> kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
> kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerGoogleCloudHSMProvider.java 666a8c38faa157a79c3c3e3dd00050978b7681da
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
> pom.xml 13f9bfdc5a88ffdf8d3502605831059fbb9ad4cc
>
>
> Diff: https://reviews.apache.org/r/73807/diff/4/
>
>
> Testing
> -------
>
> + mvn clean compile test verify
> + Fresh setup
>
>
> File Attachments
> ----------------
>
> 0001-add-TencentKMS-as-MasterKeyProvider.patch
> https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
>
>
> Thanks,
>
> Kirby Zhou
>
>
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Kirby Zhou <ki...@gmail.com>.
> On 一月 19, 2022, 7:56 a.m., bhavik patel wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java
> > Line 36 (original), 37 (patched)
> > <https://reviews.apache.org/r/73807/diff/3/?file=2265347#file2265347line37>
> >
> > is it better option to upadate the class name to RangerAzure*?
> > @dhavalshah any thought on this?
KeyVault looks like a gerneral name, so I think should add “Azure" here.
> On 一月 19, 2022, 7:56 a.m., bhavik patel wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java
> > Lines 26 (patched)
> > <https://reviews.apache.org/r/73807/diff/3/?file=2265348#file2265348line26>
> >
> > please use slf4j logger factory
I fixed it with 0002-xxxx.patch
Should I merge the 2 patches into 1?
> On 一月 19, 2022, 7:56 a.m., bhavik patel wrote:
> > pom.xml
> > Lines 242 (patched)
> > <https://reviews.apache.org/r/73807/diff/3/?file=2265349#file2265349line242>
> >
> > Make sure this Doesn’t pull the any extra and vulnerable jar files
% mvn dependency:tree -Dverbose -f tencentcloud-sdk-java-3.1.322.pom
...
[INFO] com.tencentcloudapi:tencentcloud-sdk-java:jar:3.1.322
[INFO] +- commons-logging:commons-logging:jar:1.2:compile
[INFO] +- com.squareup.okio:okio:jar:1.12.0:compile
[INFO] +- com.squareup.okhttp:okhttp:jar:2.7.5:compile
[INFO] | - (com.squareup.okio:okio:jar:1.6.0:compile - omitted for conflict with 1.12.0)
[INFO] +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] +- javax.xml.bind:jaxb-api:jar:2.3.0:compile
[INFO] +- com.squareup.okhttp:logging-interceptor:jar:2.7.5:compile
[INFO] | - (com.squareup.okhttp:okhttp:jar:2.7.5:compile - omitted for duplicate)
[INFO] - org.ini4j:ini4j:jar:0.5.4:compile
...
Is it ok?
- Kirby
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/#review223972
-----------------------------------------------------------
On 一月 19, 2022, 10:40 a.m., Kirby Zhou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73807/
> -----------------------------------------------------------
>
> (Updated 一月 19, 2022, 10:40 a.m.)
>
>
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3580
> https://issues.apache.org/jira/browse/RANGER-3580
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger KMS integration with TencentKMS
> - This task is to integrate the RANGER KMS Service with TencentKMS.
> - To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
>
> ```
> # Do you use Tencent Cloud KMS?
> TENCENT_KMS_ENABLED=true
> # MasterKeyID on Tencent Cloud
> TENCENT_MASTERKEY_ID=YourKeyID
> # Login ID
> TENCENT_CLIENT_ID=YourClientLoginId
> # Login password
> TENCENT_CLIENT_SECRET=YourClientLoginSecret
> # Tencent Cloud area, see Tencent Cloud SDK for details.
> TENCENT_CLIENT_REGION=ap-beijing
> ```
>
> Run the setup.sh, It will add the below configs in dbks-site.xml
> ```
> <!--Tencent KMS START-->
> <property>
> <name>ranger.kms.tencentkms.enabled</name>
> <value>false</value>
> <description>Flag for Tencent KMS</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.id</name>
> <value></value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret</name>
> <value></value>
> <description>Tencent Client Secret</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret.alias</name>
> <value>ranger.ks.tencent.client.secret</value>
> <description>Tencent Client Secret Alias</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.region</name>
> <value>ap-beijing</value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.masterkey.id</name>
> <value></value>
> <description>Tencent master key name</description>
> </property>
> <!--Tencent KMS END-->
> ```
>
> Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
> Start the kms service, On start Master Key from TencentKMS should be used.
>
>
> Diffs
> -----
>
> distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
> kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
> kms/pom.xml 8350403c76cd3f5a6d80e263f54b766dcf6e62e4
> kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
> kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
> kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
> pom.xml 13f9bfdc5a88ffdf8d3502605831059fbb9ad4cc
>
>
> Diff: https://reviews.apache.org/r/73807/diff/3/
>
>
> Testing
> -------
>
> + mvn clean compile test verify
> + Fresh setup
>
>
> File Attachments
> ----------------
>
> 0001-add-TencentKMS-as-MasterKeyProvider.patch
> https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
> 0002-switch-from-log4j-to-slf4j.patch
> https://reviews.apache.org/media/uploaded/files/2022/01/19/2f5c0202-c13f-420e-926a-61728f8b041e__0002-switch-from-log4j-to-slf4j.patch
>
>
> Thanks,
>
> Kirby Zhou
>
>
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Dhaval Shah <dh...@gmail.com>.
> On Jan. 19, 2022, 7:56 a.m., bhavik patel wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java
> > Line 36 (original), 37 (patched)
> > <https://reviews.apache.org/r/73807/diff/3/?file=2265347#file2265347line37>
> >
> > is it better option to upadate the class name to RangerAzure*?
> > @dhavalshah any thought on this?
>
> Kirby Zhou wrote:
> KeyVault looks like a gerneral name, so I think should add “Azure" here.
>
> Kirby Zhou wrote:
> @dhavalshah ?
> Any idea?
Its fine.
- Dhaval
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/#review223972
-----------------------------------------------------------
On Jan. 29, 2022, 9:53 a.m., Kirby Zhou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73807/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2022, 9:53 a.m.)
>
>
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3580
> https://issues.apache.org/jira/browse/RANGER-3580
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger KMS integration with TencentKMS
> - This task is to integrate the RANGER KMS Service with TencentKMS.
> - To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
>
> ```
> # Do you use Tencent Cloud KMS?
> TENCENT_KMS_ENABLED=true
> # MasterKeyID on Tencent Cloud
> TENCENT_MASTERKEY_ID=YourKeyID
> # Login ID
> TENCENT_CLIENT_ID=YourClientLoginId
> # Login password
> TENCENT_CLIENT_SECRET=YourClientLoginSecret
> # Tencent Cloud area, see Tencent Cloud SDK for details.
> TENCENT_CLIENT_REGION=ap-beijing
> ```
>
> Run the setup.sh, It will add the below configs in dbks-site.xml
> ```
> <!--Tencent KMS START-->
> <property>
> <name>ranger.kms.tencentkms.enabled</name>
> <value>false</value>
> <description>Flag for Tencent KMS</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.id</name>
> <value></value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret</name>
> <value></value>
> <description>Tencent Client Secret</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret.alias</name>
> <value>ranger.ks.tencent.client.secret</value>
> <description>Tencent Client Secret Alias</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.region</name>
> <value>ap-beijing</value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.masterkey.id</name>
> <value></value>
> <description>Tencent master key name</description>
> </property>
> <!--Tencent KMS END-->
> ```
>
> Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
> Start the kms service, On start Master Key from TencentKMS should be used.
>
>
> Diffs
> -----
>
> distro/src/main/assembly/kms.xml 33d754b491c166cc781338b6530b8a0b9623529e
> kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
> kms/pom.xml 35fbe8a9d94eb156e3510b5b611e5ce329ad8af5
> kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
> kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
> kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
> pom.xml 5982ace7d0acf16e63809aef7914819a0f908df6
>
>
> Diff: https://reviews.apache.org/r/73807/diff/5/
>
>
> Testing
> -------
>
> + mvn clean compile test verify
> + Fresh setup
>
>
> File Attachments
> ----------------
>
> 0001-add-TencentKMS-as-MasterKeyProvider.patch
> https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
>
>
> Thanks,
>
> Kirby Zhou
>
>
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by bhavik patel <bh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/#review223972
-----------------------------------------------------------
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java
Line 36 (original), 37 (patched)
<https://reviews.apache.org/r/73807/#comment312994>
is it better option to upadate the class name to RangerAzure*?
@dhavalshah any thought on this?
kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java
Lines 26 (patched)
<https://reviews.apache.org/r/73807/#comment312993>
please use slf4j logger factory
pom.xml
Lines 242 (patched)
<https://reviews.apache.org/r/73807/#comment312995>
Make sure this Doesn’t pull the any extra and vulnerable jar files
- bhavik patel
On Jan. 19, 2022, 3:55 a.m., Kirby Zhou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73807/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2022, 3:55 a.m.)
>
>
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-3580
> https://issues.apache.org/jira/browse/RANGER-3580
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger KMS integration with TencentKMS
> - This task is to integrate the RANGER KMS Service with TencentKMS.
> - To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
>
> ```
> # Do you use Tencent Cloud KMS?
> TENCENT_KMS_ENABLED=true
> # MasterKeyID on Tencent Cloud
> TENCENT_MASTERKEY_ID=YourKeyID
> # Login ID
> TENCENT_CLIENT_ID=YourClientLoginId
> # Login password
> TENCENT_CLIENT_SECRET=YourClientLoginSecret
> # Tencent Cloud area, see Tencent Cloud SDK for details.
> TENCENT_CLIENT_REGION=ap-beijing
> ```
>
> Run the setup.sh, It will add the below configs in dbks-site.xml
> ```
> <!--Tencent KMS START-->
> <property>
> <name>ranger.kms.tencentkms.enabled</name>
> <value>false</value>
> <description>Flag for Tencent KMS</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.id</name>
> <value></value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret</name>
> <value></value>
> <description>Tencent Client Secret</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.secret.alias</name>
> <value>ranger.ks.tencent.client.secret</value>
> <description>Tencent Client Secret Alias</description>
> </property>
> <property>
> <name>ranger.kms.tencent.client.region</name>
> <value>ap-beijing</value>
> <description>Tencent Client Id</description>
> </property>
> <property>
> <name>ranger.kms.tencent.masterkey.id</name>
> <value></value>
> <description>Tencent master key name</description>
> </property>
> <!--Tencent KMS END-->
> ```
>
> Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
> Start the kms service, On start Master Key from TencentKMS should be used.
>
>
> Diffs
> -----
>
> distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
> kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
> kms/pom.xml 8350403c76cd3f5a6d80e263f54b766dcf6e62e4
> kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
> kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
> kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
> pom.xml 13f9bfdc5a88ffdf8d3502605831059fbb9ad4cc
>
>
> Diff: https://reviews.apache.org/r/73807/diff/3/
>
>
> Testing
> -------
>
> + mvn clean compile test verify
> + Fresh setup
>
>
> File Attachments
> ----------------
>
> 0001-add-TencentKMS-as-MasterKeyProvider.patch
> https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
>
>
> Thanks,
>
> Kirby Zhou
>
>
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Kirby Zhou <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/
-----------------------------------------------------------
(Updated 一月 19, 2022, 3:55 a.m.)
Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Changes
-------
merge with master-HEAD
Bugs: RANGER-3580
https://issues.apache.org/jira/browse/RANGER-3580
Repository: ranger
Description
-------
Ranger KMS integration with TencentKMS
- This task is to integrate the RANGER KMS Service with TencentKMS.
- To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
```
# Do you use Tencent Cloud KMS?
TENCENT_KMS_ENABLED=true
# MasterKeyID on Tencent Cloud
TENCENT_MASTERKEY_ID=YourKeyID
# Login ID
TENCENT_CLIENT_ID=YourClientLoginId
# Login password
TENCENT_CLIENT_SECRET=YourClientLoginSecret
# Tencent Cloud area, see Tencent Cloud SDK for details.
TENCENT_CLIENT_REGION=ap-beijing
```
Run the setup.sh, It will add the below configs in dbks-site.xml
```
<!--Tencent KMS START-->
<property>
<name>ranger.kms.tencentkms.enabled</name>
<value>false</value>
<description>Flag for Tencent KMS</description>
</property>
<property>
<name>ranger.kms.tencent.client.id</name>
<value></value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret</name>
<value></value>
<description>Tencent Client Secret</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret.alias</name>
<value>ranger.ks.tencent.client.secret</value>
<description>Tencent Client Secret Alias</description>
</property>
<property>
<name>ranger.kms.tencent.client.region</name>
<value>ap-beijing</value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.masterkey.id</name>
<value></value>
<description>Tencent master key name</description>
</property>
<!--Tencent KMS END-->
```
Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
Start the kms service, On start Master Key from TencentKMS should be used.
Diffs (updated)
-----
distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
kms/pom.xml 8350403c76cd3f5a6d80e263f54b766dcf6e62e4
kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 5234dc7422793b3b88dcc4574fafcf34556fa33f
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java c661268c3c25362e428884a3bb34d88d827e7f31
kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
pom.xml 13f9bfdc5a88ffdf8d3502605831059fbb9ad4cc
Diff: https://reviews.apache.org/r/73807/diff/3/
Changes: https://reviews.apache.org/r/73807/diff/2-3/
Testing
-------
+ mvn clean compile test verify
+ Fresh setup
File Attachments
----------------
0001-add-TencentKMS-as-MasterKeyProvider.patch
https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
Thanks,
Kirby Zhou
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Kirby Zhou <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/
-----------------------------------------------------------
(Updated 一月 19, 2022, 3:30 a.m.)
Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Bugs: RANGER-3580
https://issues.apache.org/jira/browse/RANGER-3580
Repository: ranger
Description
-------
Ranger KMS integration with TencentKMS
- This task is to integrate the RANGER KMS Service with TencentKMS.
- To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
```
# Do you use Tencent Cloud KMS?
TENCENT_KMS_ENABLED=true
# MasterKeyID on Tencent Cloud
TENCENT_MASTERKEY_ID=YourKeyID
# Login ID
TENCENT_CLIENT_ID=YourClientLoginId
# Login password
TENCENT_CLIENT_SECRET=YourClientLoginSecret
# Tencent Cloud area, see Tencent Cloud SDK for details.
TENCENT_CLIENT_REGION=ap-beijing
```
Run the setup.sh, It will add the below configs in dbks-site.xml
```
<!--Tencent KMS START-->
<property>
<name>ranger.kms.tencentkms.enabled</name>
<value>false</value>
<description>Flag for Tencent KMS</description>
</property>
<property>
<name>ranger.kms.tencent.client.id</name>
<value></value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret</name>
<value></value>
<description>Tencent Client Secret</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret.alias</name>
<value>ranger.ks.tencent.client.secret</value>
<description>Tencent Client Secret Alias</description>
</property>
<property>
<name>ranger.kms.tencent.client.region</name>
<value>ap-beijing</value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.masterkey.id</name>
<value></value>
<description>Tencent master key name</description>
</property>
<!--Tencent KMS END-->
```
Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
Start the kms service, On start Master Key from TencentKMS should be used.
Diffs
-----
distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
kms/pom.xml 908e8841a4a8c0627622d35486ffe2dac7bbdc61
kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java 19335893a4d875743fcf71b12124b1c40a4ee6e1
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 12d485a36423d1c25a3b59cc8436e0a40863a78f
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java bd85c0d364ad06691a0f5d8f19b5124189261db3
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java d21d32320bbd32e4efbe83c701d11c94dc804765
kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
kms/src/main/resources/log4j.properties 5cd037a49c670a3a0eec9e85fdcafeeeabff1405
pom.xml c663937eba452321b4e2400cc6d3f528f74596de
Diff: https://reviews.apache.org/r/73807/diff/2/
Testing (updated)
-------
+ mvn clean compile test verify
+ Fresh setup
File Attachments
----------------
0001-add-TencentKMS-as-MasterKeyProvider.patch
https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
Thanks,
Kirby Zhou
Re: Review Request 73807: Support Ranger KMS integration with TencentKMS
Posted by Kirby Zhou <ki...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/
-----------------------------------------------------------
(Updated 一月 19, 2022, 3:09 a.m.)
Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and Velmurugan Periasamy.
Bugs: RANGER-3580
https://issues.apache.org/jira/browse/RANGER-3580
Repository: ranger
Description
-------
Ranger KMS integration with TencentKMS
- This task is to integrate the RANGER KMS Service with TencentKMS.
- To Configure RANGER KMS Service with TencentKMS below configurations need to be added in install.properties file bfore running the setup.sh
```
# Do you use Tencent Cloud KMS?
TENCENT_KMS_ENABLED=true
# MasterKeyID on Tencent Cloud
TENCENT_MASTERKEY_ID=YourKeyID
# Login ID
TENCENT_CLIENT_ID=YourClientLoginId
# Login password
TENCENT_CLIENT_SECRET=YourClientLoginSecret
# Tencent Cloud area, see Tencent Cloud SDK for details.
TENCENT_CLIENT_REGION=ap-beijing
```
Run the setup.sh, It will add the below configs in dbks-site.xml
```
<!--Tencent KMS START-->
<property>
<name>ranger.kms.tencentkms.enabled</name>
<value>false</value>
<description>Flag for Tencent KMS</description>
</property>
<property>
<name>ranger.kms.tencent.client.id</name>
<value></value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret</name>
<value></value>
<description>Tencent Client Secret</description>
</property>
<property>
<name>ranger.kms.tencent.client.secret.alias</name>
<value>ranger.ks.tencent.client.secret</value>
<description>Tencent Client Secret Alias</description>
</property>
<property>
<name>ranger.kms.tencent.client.region</name>
<value>ap-beijing</value>
<description>Tencent Client Id</description>
</property>
<property>
<name>ranger.kms.tencent.masterkey.id</name>
<value></value>
<description>Tencent master key name</description>
</property>
<!--Tencent KMS END-->
```
Generally, we don't want the account bound by KMS to have the right to create a Key in TencentKMS. So we have to create Master Key on TencentKMS web console at first.
Start the kms service, On start Master Key from TencentKMS should be used.
Diffs (updated)
-----
distro/src/main/assembly/kms.xml 32bbefa44e372f3abb41d60cd35aa0d706ca3100
kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab
kms/pom.xml 908e8841a4a8c0627622d35486ffe2dac7bbdc61
kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102
kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36
kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java 19335893a4d875743fcf71b12124b1c40a4ee6e1
kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java bacc928570283708daef7a2573707fddd7ca096e
kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 4324439ba66f9f0fb68d570f1964ed6caa8c07bd
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 12d485a36423d1c25a3b59cc8436e0a40863a78f
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java bd85c0d364ad06691a0f5d8f19b5124189261db3
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java d21d32320bbd32e4efbe83c701d11c94dc804765
kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java PRE-CREATION
kms/src/main/resources/log4j.properties 5cd037a49c670a3a0eec9e85fdcafeeeabff1405
pom.xml c663937eba452321b4e2400cc6d3f528f74596de
Diff: https://reviews.apache.org/r/73807/diff/2/
Changes: https://reviews.apache.org/r/73807/diff/1-2/
Testing
-------
File Attachments (updated)
----------------
0001-add-TencentKMS-as-MasterKeyProvider.patch
https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch
Thanks,
Kirby Zhou