You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Colm O hEigeartaigh <co...@apache.org> on 2016/04/29 12:43:46 UTC
Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/
-----------------------------------------------------------
Review request for ranger.
Repository: ranger
Description
-------
The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
Diffs
-----
kms/pom.xml a9f6c6c
kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 6ab91d9
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
Diff: https://reviews.apache.org/r/46829/diff/
Testing
-------
Thanks,
Colm O hEigeartaigh
Re: Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
Posted by Colm O hEigeartaigh <co...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/
-----------------------------------------------------------
(Updated June 7, 2016, 11:26 a.m.)
Review request for ranger.
Bugs: RANGER-961
https://issues.apache.org/jira/browse/RANGER-961
Repository: ranger
Description
-------
The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
Diffs (updated)
-----
kms/pom.xml 9c9a606
kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java b937f0c
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
Diff: https://reviews.apache.org/r/46829/diff/
Testing
-------
Thanks,
Colm O hEigeartaigh
Re: Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
Posted by Colm O hEigeartaigh <co...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/
-----------------------------------------------------------
(Updated May 27, 2016, 9:20 a.m.)
Review request for ranger.
Bugs: RANGER-961
https://issues.apache.org/jira/browse/RANGER-961
Repository: ranger
Description
-------
The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
Diffs (updated)
-----
kms/pom.xml a9f6c6c
kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java b937f0c
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
Diff: https://reviews.apache.org/r/46829/diff/
Testing
-------
Thanks,
Colm O hEigeartaigh
Re: Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
Posted by Colm O hEigeartaigh <co...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/
-----------------------------------------------------------
(Updated May 27, 2016, 9:07 a.m.)
Review request for ranger.
Bugs: RANGER-961
https://issues.apache.org/jira/browse/RANGER-961
Repository: ranger
Description
-------
The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
Diffs
-----
kms/pom.xml a9f6c6c
kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 6ab91d9
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
Diff: https://reviews.apache.org/r/46829/diff/
Testing
-------
Thanks,
Colm O hEigeartaigh
Re: Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
Posted by Colm O hEigeartaigh <co...@apache.org>.
> On May 27, 2016, 3:20 a.m., Ankita Sinha wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java, line 150
> > <https://reviews.apache.org/r/46829/diff/1/?file=1365719#file1365719line150>
> >
> > While testing with this patch changes : Encryption key operations are failing.
> >
> >
> > $ hadoop key create key1
> > key1 has not been created. java.io.IOException: Can't store key key1@0
> > java.io.IOException: Can't store key key1@0
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> > at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:552)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:510)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKeyInternal(KMSClientProvider.java:683)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKey(KMSClientProvider.java:691)
> > at org.apache.hadoop.crypto.key.KeyShell$CreateCommand.execute(KeyShell.java:483)
> > at org.apache.hadoop.crypto.key.KeyShell.run(KeyShell.java:79)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> > at org.apache.hadoop.crypto.key.KeyShell.main(KeyShell.java:515)
Could you let me know how to reproduce this error + I'll take a look?
- Colm
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/#review135143
-----------------------------------------------------------
On May 27, 2016, 9:07 a.m., Colm O hEigeartaigh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46829/
> -----------------------------------------------------------
>
> (Updated May 27, 2016, 9:07 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-961
> https://issues.apache.org/jira/browse/RANGER-961
>
>
> Repository: ranger
>
>
> Description
> -------
>
> The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
>
>
> Diffs
> -----
>
> kms/pom.xml a9f6c6c
> kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
> kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 6ab91d9
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
>
> Diff: https://reviews.apache.org/r/46829/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Colm O hEigeartaigh
>
>
Re: Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
Posted by Colm O hEigeartaigh <co...@apache.org>.
> On May 27, 2016, 3:20 a.m., Ankita Sinha wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java, line 150
> > <https://reviews.apache.org/r/46829/diff/1/?file=1365719#file1365719line150>
> >
> > While testing with this patch changes : Encryption key operations are failing.
> >
> >
> > $ hadoop key create key1
> > key1 has not been created. java.io.IOException: Can't store key key1@0
> > java.io.IOException: Can't store key key1@0
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> > at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:552)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:510)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKeyInternal(KMSClientProvider.java:683)
> > at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKey(KMSClientProvider.java:691)
> > at org.apache.hadoop.crypto.key.KeyShell$CreateCommand.execute(KeyShell.java:483)
> > at org.apache.hadoop.crypto.key.KeyShell.run(KeyShell.java:79)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> > at org.apache.hadoop.crypto.key.KeyShell.main(KeyShell.java:515)
>
> Colm O hEigeartaigh wrote:
> Could you let me know how to reproduce this error + I'll take a look?
Could you re-test with the latest patch please? I found an issue in that the old BASE-64 encoder was using the incorrect line break character. I modified the patch so that the Commons Codec BASE-64 encoder is using the (incorrect) line break to preserve backwards compatibility.
- Colm
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/#review135143
-----------------------------------------------------------
On June 7, 2016, 11:26 a.m., Colm O hEigeartaigh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46829/
> -----------------------------------------------------------
>
> (Updated June 7, 2016, 11:26 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-961
> https://issues.apache.org/jira/browse/RANGER-961
>
>
> Repository: ranger
>
>
> Description
> -------
>
> The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
>
>
> Diffs
> -----
>
> kms/pom.xml 9c9a606
> kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
> kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java b937f0c
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
>
> Diff: https://reviews.apache.org/r/46829/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Colm O hEigeartaigh
>
>
Re: Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
Posted by Ankita Sinha <an...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/#review135143
-----------------------------------------------------------
kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java (line 147)
<https://reviews.apache.org/r/46829/#comment200170>
While testing with this patch changes : Encryption key operations are failing.
$ hadoop key create key1
key1 has not been created. java.io.IOException: Can't store key key1@0
java.io.IOException: Can't store key key1@0
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:552)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:510)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKeyInternal(KMSClientProvider.java:683)
at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKey(KMSClientProvider.java:691)
at org.apache.hadoop.crypto.key.KeyShell$CreateCommand.execute(KeyShell.java:483)
at org.apache.hadoop.crypto.key.KeyShell.run(KeyShell.java:79)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.crypto.key.KeyShell.main(KeyShell.java:515)
- Ankita Sinha
On April 29, 2016, 10:43 a.m., Colm O hEigeartaigh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46829/
> -----------------------------------------------------------
>
> (Updated April 29, 2016, 10:43 a.m.)
>
>
> Review request for ranger.
>
>
> Repository: ranger
>
>
> Description
> -------
>
> The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
>
>
> Diffs
> -----
>
> kms/pom.xml a9f6c6c
> kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
> kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 6ab91d9
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
>
> Diff: https://reviews.apache.org/r/46829/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Colm O hEigeartaigh
>
>
Re: Review Request 46829: RANGER-961 - Remove KMS xml-security-impl
dependency
Posted by Velmurugan Periasamy <vp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46829/#review134992
-----------------------------------------------------------
kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java (line 109)
<https://reviews.apache.org/r/46829/#comment199980>
Colm - Patch does not apply on master branch. Please check. Also, please update BUG and BRANCH fields in the review request.
$ git apply --check -v < ~/Downloads/patches/ranger/0001-RANGER-961-Remove-KMS-xml-security-impl-dependency.patch
Checking patch kms/pom.xml...
Checking patch kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java...
Checking patch kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java...
Checking patch kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java...
Hunk #2 succeeded at 90 (offset 1 line).
error: while searching for:
if (result == true) {
logger.debug("Ranger Master Key is present in Keystore");
SecretKey key = (SecretKey)myStore.getKey(alias, password.toCharArray());
String masterKey = Base64.encode(key.getEncoded()) ;
return masterKey;
}
} catch (Exception e) {
error: patch failed: kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java:109
error: kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java: patch does not apply
Checking patch kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java...
- Velmurugan Periasamy
On April 29, 2016, 10:43 a.m., Colm O hEigeartaigh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46829/
> -----------------------------------------------------------
>
> (Updated April 29, 2016, 10:43 a.m.)
>
>
> Review request for ranger.
>
>
> Repository: ranger
>
>
> Description
> -------
>
> The KMS module uses the xml-security-impl dependency just to get a Base64 encoder/decoder. This is not necessary as one is already available in Commons Codec, and in fact is used in other classes in this module.
>
>
> Diffs
> -----
>
> kms/pom.xml a9f6c6c
> kms/src/main/java/org/apache/hadoop/crypto/key/DB2HSMMKUtil.java ca69dc0
> kms/src/main/java/org/apache/hadoop/crypto/key/HSM2DBMKUtil.java 73a5830
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerHSM.java 6ab91d9
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java d70ec4e
>
> Diff: https://reviews.apache.org/r/46829/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Colm O hEigeartaigh
>
>