You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Thejas Nair <th...@hortonworks.com> on 2014/10/27 02:36:17 UTC
Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to
use kerberos authentication when kerberos is enabled
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/
-----------------------------------------------------------
Review request for hive, Eugene Koifman and Vaibhav Gumashta.
Bugs: HIVE-8557
https://issues.apache.org/jira/browse/HIVE-8557
Repository: hive-git
Description
-------
https://issues.apache.org/jira/browse/HIVE-8557
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
hcatalog/webhcat/svr/pom.xml 6065748
itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
pom.xml c694980
ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
shims/common-secure/pom.xml 98b5ca1
shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
Diff: https://reviews.apache.org/r/27216/diff/
Testing
-------
Existing tests excercised, manually tested in a cluster.
Thanks,
Thejas Nair
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Thejas Nair <th...@hortonworks.com>.
> On Oct. 29, 2014, 12:04 a.m., Thejas Nair wrote:
> > shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java, line 170
> > <https://reviews.apache.org/r/27216/diff/2/?file=735241#file735241line170>
> >
> > The JAAS setting would be needed for reconnections, which will be done automatically by curator.
> > There is only going to be one call to this per jvm. Only one instance of the tokenstore is used by both HS2 and metastore.
> > Even if HS2 has embedded metastore, there will only be one instance of the token store, as embedded metastore does not use one.
> > The init is also being called only once per object.
I was not right about only one call per jvm, in HS2, this will be called once for token store and once for the dynamic discovery. But I tested that and it works fine, I will test once more.
- Thejas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/#review58905
-----------------------------------------------------------
On Oct. 29, 2014, 12:29 a.m., Thejas Nair wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27216/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2014, 12:29 a.m.)
>
>
> Review request for hive, Eugene Koifman and Vaibhav Gumashta.
>
>
> Bugs: HIVE-8557
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
> hcatalog/webhcat/svr/pom.xml 6065748
> itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
> pom.xml c694980
> ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
> service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
> shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
> shims/common-secure/pom.xml 98b5ca1
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
> shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
> shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
>
> Diff: https://reviews.apache.org/r/27216/diff/
>
>
> Testing
> -------
>
> Existing tests excercised, manually tested in a cluster.
>
>
> Thanks,
>
> Thejas Nair
>
>
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Thejas Nair <th...@hortonworks.com>.
> On Oct. 29, 2014, 12:04 a.m., Thejas Nair wrote:
> > shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java, line 170
> > <https://reviews.apache.org/r/27216/diff/2/?file=735241#file735241line170>
> >
> > The JAAS setting would be needed for reconnections, which will be done automatically by curator.
> > There is only going to be one call to this per jvm. Only one instance of the tokenstore is used by both HS2 and metastore.
> > Even if HS2 has embedded metastore, there will only be one instance of the token store, as embedded metastore does not use one.
> > The init is also being called only once per object.
>
> Thejas Nair wrote:
> I was not right about only one call per jvm, in HS2, this will be called once for token store and once for the dynamic discovery. But I tested that and it works fine, I will test once more.
Verified again that changes work with dynamic discovery and zk based delegation token storage both turned on.
- Thejas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/#review58905
-----------------------------------------------------------
On Oct. 29, 2014, 12:29 a.m., Thejas Nair wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27216/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2014, 12:29 a.m.)
>
>
> Review request for hive, Eugene Koifman and Vaibhav Gumashta.
>
>
> Bugs: HIVE-8557
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
> hcatalog/webhcat/svr/pom.xml 6065748
> itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
> pom.xml c694980
> ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
> service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
> shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
> shims/common-secure/pom.xml 98b5ca1
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
> shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
> shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
>
> Diff: https://reviews.apache.org/r/27216/diff/
>
>
> Testing
> -------
>
> Existing tests excercised, manually tested in a cluster.
>
>
> Thanks,
>
> Thejas Nair
>
>
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Thejas Nair <th...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/#review58905
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/27216/#comment100119>
Fixing the description, adding example.
shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java
<https://reviews.apache.org/r/27216/#comment100102>
The IDE removed the unnecessary cast. super.getSplits already returns a (InputSplit[])
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
<https://reviews.apache.org/r/27216/#comment100074>
The JAAS setting would be needed for reconnections, which will be done automatically by curator.
There is only going to be one call to this per jvm. Only one instance of the tokenstore is used by both HS2 and metastore.
Even if HS2 has embedded metastore, there will only be one instance of the token store, as embedded metastore does not use one.
The init is also being called only once per object.
- Thejas Nair
On Oct. 28, 2014, 6:35 a.m., Thejas Nair wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27216/
> -----------------------------------------------------------
>
> (Updated Oct. 28, 2014, 6:35 a.m.)
>
>
> Review request for hive, Eugene Koifman and Vaibhav Gumashta.
>
>
> Bugs: HIVE-8557
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
> hcatalog/webhcat/svr/pom.xml 6065748
> itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
> pom.xml c694980
> ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
> service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
> shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
> shims/common-secure/pom.xml 98b5ca1
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
> shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
> shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
>
> Diff: https://reviews.apache.org/r/27216/diff/
>
>
> Testing
> -------
>
> Existing tests excercised, manually tested in a cluster.
>
>
> Thanks,
>
> Thejas Nair
>
>
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Vaibhav Gumashta <vg...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/#review58961
-----------------------------------------------------------
Ship it!
Ship It!
- Vaibhav Gumashta
On Oct. 29, 2014, 12:29 a.m., Thejas Nair wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27216/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2014, 12:29 a.m.)
>
>
> Review request for hive, Eugene Koifman and Vaibhav Gumashta.
>
>
> Bugs: HIVE-8557
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
> hcatalog/webhcat/svr/pom.xml 6065748
> itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
> pom.xml c694980
> ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
> service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
> shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
> shims/common-secure/pom.xml 98b5ca1
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
> shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
> shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
>
> Diff: https://reviews.apache.org/r/27216/diff/
>
>
> Testing
> -------
>
> Existing tests excercised, manually tested in a cluster.
>
>
> Thanks,
>
> Thejas Nair
>
>
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Thejas Nair <th...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/
-----------------------------------------------------------
(Updated Oct. 29, 2014, 12:29 a.m.)
Review request for hive, Eugene Koifman and Vaibhav Gumashta.
Changes
-------
HIVE-8557.4.patch : adding missing period in config description
Bugs: HIVE-8557
https://issues.apache.org/jira/browse/HIVE-8557
Repository: hive-git
Description
-------
https://issues.apache.org/jira/browse/HIVE-8557
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
hcatalog/webhcat/svr/pom.xml 6065748
itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
pom.xml c694980
ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
shims/common-secure/pom.xml 98b5ca1
shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
Diff: https://reviews.apache.org/r/27216/diff/
Testing
-------
Existing tests excercised, manually tested in a cluster.
Thanks,
Thejas Nair
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Thejas Nair <th...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/
-----------------------------------------------------------
(Updated Oct. 29, 2014, 12:22 a.m.)
Review request for hive, Eugene Koifman and Vaibhav Gumashta.
Changes
-------
addressing review comments
Bugs: HIVE-8557
https://issues.apache.org/jira/browse/HIVE-8557
Repository: hive-git
Description
-------
https://issues.apache.org/jira/browse/HIVE-8557
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
hcatalog/webhcat/svr/pom.xml 6065748
itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
pom.xml c694980
ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
shims/common-secure/pom.xml 98b5ca1
shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
Diff: https://reviews.apache.org/r/27216/diff/
Testing
-------
Existing tests excercised, manually tested in a cluster.
Thanks,
Thejas Nair
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Thejas Nair <th...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/#review58934
-----------------------------------------------------------
service/src/java/org/apache/hive/service/server/HiveServer2.java
<https://reviews.apache.org/r/27216/#comment100121>
OK, will rename it as setZookeeperClientKerberosJaasConfig
- Thejas Nair
On Oct. 28, 2014, 6:35 a.m., Thejas Nair wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27216/
> -----------------------------------------------------------
>
> (Updated Oct. 28, 2014, 6:35 a.m.)
>
>
> Review request for hive, Eugene Koifman and Vaibhav Gumashta.
>
>
> Bugs: HIVE-8557
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
> hcatalog/webhcat/svr/pom.xml 6065748
> itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
> pom.xml c694980
> ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
> service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
> shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
> shims/common-secure/pom.xml 98b5ca1
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
> shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
> shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
>
> Diff: https://reviews.apache.org/r/27216/diff/
>
>
> Testing
> -------
>
> Existing tests excercised, manually tested in a cluster.
>
>
> Thanks,
>
> Thejas Nair
>
>
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Vaibhav Gumashta <vg...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/#review58841
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/27216/#comment99999>
Should we mention our preference for using hive.zookeeper.quorum?
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/27216/#comment99995>
Was the original config a comma separated list of ACLs or a list of servers? This description (original version) looks confusing...
service/src/java/org/apache/hive/service/server/HiveServer2.java
<https://reviews.apache.org/r/27216/#comment100054>
Minor nit: I just noticed that setZookeeperClientJaasConfig is setting the JAAS config specific to Kerberos authentication. Currently ZK SASL implementation seems to support only Kerberos login module, but it seems likely that this will get expanded to support other SASL auth modules. In that case if we decide to use it to set up ACLs, we might want to indicate that this method specifically sets up Kerberos JAAS Config.
shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java
<https://reviews.apache.org/r/27216/#comment100002>
Unintentional change?
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
<https://reviews.apache.org/r/27216/#comment100070>
I'm just wondering if after opening the client connection (which will do the authentication), we should restore the original JAAS config in the runtime. Just wondering if we can get into an inconsistent state with multiple JAAS logins happening.
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
<https://reviews.apache.org/r/27216/#comment100010>
The doc needs to use TokenStoreException.
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
<https://reviews.apache.org/r/27216/#comment100016>
Nit: it will be good to have a self documenting variable name instead of csv.
- Vaibhav Gumashta
On Oct. 28, 2014, 6:35 a.m., Thejas Nair wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27216/
> -----------------------------------------------------------
>
> (Updated Oct. 28, 2014, 6:35 a.m.)
>
>
> Review request for hive, Eugene Koifman and Vaibhav Gumashta.
>
>
> Bugs: HIVE-8557
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
> hcatalog/webhcat/svr/pom.xml 6065748
> itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
> pom.xml c694980
> ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
> service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
> shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
> shims/common-secure/pom.xml 98b5ca1
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
> shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
> shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
>
> Diff: https://reviews.apache.org/r/27216/diff/
>
>
> Testing
> -------
>
> Existing tests excercised, manually tested in a cluster.
>
>
> Thanks,
>
> Thejas Nair
>
>
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Thejas Nair <th...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/
-----------------------------------------------------------
(Updated Oct. 28, 2014, 6:35 a.m.)
Review request for hive, Eugene Koifman and Vaibhav Gumashta.
Changes
-------
HIVE-8557.2.patch - addressing comments from Lefty
Bugs: HIVE-8557
https://issues.apache.org/jira/browse/HIVE-8557
Repository: hive-git
Description
-------
https://issues.apache.org/jira/browse/HIVE-8557
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
hcatalog/webhcat/svr/pom.xml 6065748
itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
pom.xml c694980
ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
shims/common-secure/pom.xml 98b5ca1
shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
Diff: https://reviews.apache.org/r/27216/diff/
Testing
-------
Existing tests excercised, manually tested in a cluster.
Thanks,
Thejas Nair
Re: Review Request 27216: HIVE-8557 : automatically setup
ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled
Posted by Lefty Leverenz <le...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27216/#review58587
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/27216/#comment99649>
needs a period after "each of them" & doesn't need the newline after "METASTORE" & could use a final period
- Lefty Leverenz
On Oct. 27, 2014, 1:36 a.m., Thejas Nair wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27216/
> -----------------------------------------------------------
>
> (Updated Oct. 27, 2014, 1:36 a.m.)
>
>
> Review request for hive, Eugene Koifman and Vaibhav Gumashta.
>
>
> Bugs: HIVE-8557
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-8557
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7d8e5bc
> hcatalog/webhcat/svr/pom.xml 6065748
> itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java b81942a
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 8860d30
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java 83a80b4
> metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3cf43a9
> pom.xml c694980
> ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 11dd962
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java ab34d2d
> service/src/java/org/apache/hive/service/server/HiveServer2.java 0aab3f9
> shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java d18ae44
> shims/common-secure/pom.xml 98b5ca1
> shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java 606f973
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java 0bb2763
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java f3c2e48
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 56735d8
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9908aa4
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java 4ccf895
> shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 8683496
> shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 9850405
> shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java d0d6c7b
>
> Diff: https://reviews.apache.org/r/27216/diff/
>
>
> Testing
> -------
>
> Existing tests excercised, manually tested in a cluster.
>
>
> Thanks,
>
> Thejas Nair
>
>