You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Prasad Mujumdar <pr...@cloudera.com> on 2014/03/13 22:59:23 UTC
Review Request 19206: HIVE-6657: Add test coverage for Kerberos
authentication implementation using Hadoop's miniKdc
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19206/
-----------------------------------------------------------
Review request for hive, Brock Noland and Thejas Nair.
Bugs: HIVE-665
https://issues.apache.org/jira/browse/HIVE-665
Repository: hive-git
Description
-------
- Build changes
+ pull required dependenices
+ new sub-module under itest for the hive miniKdc framework, enabled for hadoop-2 only since hadoop-1 doesn't include miniKdc yet.
- A wrapper class MiniHiveKdc on top of Hadoop miniKdc that manages the kdc services that creates test principals, keytab etc
- Tests
+ simple unit test for MiniHiveKdc
+ Integrate MiniHiveKdc with MiniHS2
+ New HiveServer2 test that covers basic connection, as well as the token and proxy auth support added by HIVE-5155
Note that Hadoop does provide a secure mini-cluster yet. Hence the new tests are not executing any queries. At this point it's limited to authentication part only.
Diffs
-----
itests/hive-minikdc/pom.xml PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestMiniHiveKdc.java PRE-CREATION
itests/hive-minikdc/src/test/resources/core-site.xml PRE-CREATION
itests/hive-unit/pom.xml 6aff277
itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java PRE-CREATION
itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java PRE-CREATION
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java 095b989
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java d76750c
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHiveServer2.java 84f508c
itests/pom.xml 589ba25
pom.xml 6503c94
service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 6759903
service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 6fbc847
Diff: https://reviews.apache.org/r/19206/diff/
Testing
-------
New test cases using the miniKdc.
Thanks,
Prasad Mujumdar
Re: Review Request 19206: HIVE-6657: Add test coverage for Kerberos
authentication implementation using Hadoop's miniKdc
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19206/#review37222
-----------------------------------------------------------
Ship it!
LGTM pending tests
- Brock Noland
On March 14, 2014, 5:12 p.m., Prasad Mujumdar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19206/
> -----------------------------------------------------------
>
> (Updated March 14, 2014, 5:12 p.m.)
>
>
> Review request for hive, Brock Noland and Thejas Nair.
>
>
> Bugs: HIVE-6657
> https://issues.apache.org/jira/browse/HIVE-6657
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> - Build changes
> + pull required dependenices
> + new sub-module under itest for the hive miniKdc framework, enabled for hadoop-2 only since hadoop-1 doesn't include miniKdc yet.
> - A wrapper class MiniHiveKdc on top of Hadoop miniKdc that manages the kdc services that creates test principals, keytab etc
> - Tests
> + simple unit test for MiniHiveKdc
> + Integrate MiniHiveKdc with MiniHS2
> + New HiveServer2 test that covers basic connection, as well as the token and proxy auth support added by HIVE-5155
>
> Note that Hadoop does provide a secure mini-cluster yet. Hence the new tests are not executing any queries. At this point it's limited to authentication part only.
>
>
> Diffs
> -----
>
> itests/hive-minikdc/pom.xml PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestMiniHiveKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/resources/core-site.xml PRE-CREATION
> itests/hive-unit/pom.xml 6aff277
> itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java PRE-CREATION
> itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java PRE-CREATION
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java 095b989
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java d76750c
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHiveServer2.java 84f508c
> itests/pom.xml 589ba25
> pom.xml 6503c94
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 6759903
> service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 6fbc847
>
> Diff: https://reviews.apache.org/r/19206/diff/
>
>
> Testing
> -------
>
> New test cases using the miniKdc.
>
>
> Thanks,
>
> Prasad Mujumdar
>
>
Re: Review Request 19206: HIVE-6657: Add test coverage for Kerberos
authentication implementation using Hadoop's miniKdc
Posted by Prasad Mujumdar <pr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19206/
-----------------------------------------------------------
(Updated March 14, 2014, 5:12 p.m.)
Review request for hive, Brock Noland and Thejas Nair.
Changes
-------
Changes per review feedback
Bugs: HIVE-6657
https://issues.apache.org/jira/browse/HIVE-6657
Repository: hive-git
Description
-------
- Build changes
+ pull required dependenices
+ new sub-module under itest for the hive miniKdc framework, enabled for hadoop-2 only since hadoop-1 doesn't include miniKdc yet.
- A wrapper class MiniHiveKdc on top of Hadoop miniKdc that manages the kdc services that creates test principals, keytab etc
- Tests
+ simple unit test for MiniHiveKdc
+ Integrate MiniHiveKdc with MiniHS2
+ New HiveServer2 test that covers basic connection, as well as the token and proxy auth support added by HIVE-5155
Note that Hadoop does provide a secure mini-cluster yet. Hence the new tests are not executing any queries. At this point it's limited to authentication part only.
Diffs (updated)
-----
itests/hive-minikdc/pom.xml PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestMiniHiveKdc.java PRE-CREATION
itests/hive-minikdc/src/test/resources/core-site.xml PRE-CREATION
itests/hive-unit/pom.xml 6aff277
itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java PRE-CREATION
itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java PRE-CREATION
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java 095b989
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java d76750c
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHiveServer2.java 84f508c
itests/pom.xml 589ba25
pom.xml 6503c94
service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 6759903
service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 6fbc847
Diff: https://reviews.apache.org/r/19206/diff/
Testing
-------
New test cases using the miniKdc.
Thanks,
Prasad Mujumdar
Re: Review Request 19206: HIVE-6657: Add test coverage for Kerberos
authentication implementation using Hadoop's miniKdc
Posted by Prasad Mujumdar <pr...@cloudera.com>.
> On March 14, 2014, 1:43 p.m., Brock Noland wrote:
> > This looks awesome! I had no idea we could enable a submodule for a specific profile!
> >
> > Just curious, what is the felix plugin doing for us?
felix plugin needed for apache DS dependency of hadoop miniKdc.
> On March 14, 2014, 1:43 p.m., Brock Noland wrote:
> > itests/hive-unit/pom.xml, line 41
> > <https://reviews.apache.org/r/19206/diff/1/?file=519155#file519155line41>
> >
> > This might be a my pedantic side...but since we are moving test scope from from deps can we move those under a <!-- intra-project --> comment? I have tried to separate out test and non-test deps:
> >
> > https://github.com/apache/hive/blob/trunk/ql/pom.xml#L36
Sounds reasonable. Updated the pom to group the non-test dependencies together under 'intra-project' comment header.
> On March 14, 2014, 1:43 p.m., Brock Noland wrote:
> > itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java, line 152
> > <https://reviews.apache.org/r/19206/diff/1/?file=519157#file519157line152>
> >
> > Should this commented out line and the ones waitForStartup be removed?
> >
> > If not, please add a comment saying when we would uncomment.
removed the comment.
> On March 14, 2014, 1:43 p.m., Brock Noland wrote:
> > pom.xml, line 636
> > <https://reviews.apache.org/r/19206/diff/1/?file=519162#file519162line636>
> >
> > Can you create a property and add this version to the properties?
Done
- Prasad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19206/#review37195
-----------------------------------------------------------
On March 14, 2014, 7:06 a.m., Prasad Mujumdar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19206/
> -----------------------------------------------------------
>
> (Updated March 14, 2014, 7:06 a.m.)
>
>
> Review request for hive, Brock Noland and Thejas Nair.
>
>
> Bugs: HIVE-6657
> https://issues.apache.org/jira/browse/HIVE-6657
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> - Build changes
> + pull required dependenices
> + new sub-module under itest for the hive miniKdc framework, enabled for hadoop-2 only since hadoop-1 doesn't include miniKdc yet.
> - A wrapper class MiniHiveKdc on top of Hadoop miniKdc that manages the kdc services that creates test principals, keytab etc
> - Tests
> + simple unit test for MiniHiveKdc
> + Integrate MiniHiveKdc with MiniHS2
> + New HiveServer2 test that covers basic connection, as well as the token and proxy auth support added by HIVE-5155
>
> Note that Hadoop does provide a secure mini-cluster yet. Hence the new tests are not executing any queries. At this point it's limited to authentication part only.
>
>
> Diffs
> -----
>
> itests/hive-minikdc/pom.xml PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestMiniHiveKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/resources/core-site.xml PRE-CREATION
> itests/hive-unit/pom.xml 6aff277
> itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java PRE-CREATION
> itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java PRE-CREATION
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java 095b989
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java d76750c
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHiveServer2.java 84f508c
> itests/pom.xml 589ba25
> pom.xml 6503c94
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 6759903
> service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 6fbc847
>
> Diff: https://reviews.apache.org/r/19206/diff/
>
>
> Testing
> -------
>
> New test cases using the miniKdc.
>
>
> Thanks,
>
> Prasad Mujumdar
>
>
Re: Review Request 19206: HIVE-6657: Add test coverage for Kerberos
authentication implementation using Hadoop's miniKdc
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19206/#review37195
-----------------------------------------------------------
This looks awesome! I had no idea we could enable a submodule for a specific profile!
Just curious, what is the felix plugin doing for us?
itests/hive-unit/pom.xml
<https://reviews.apache.org/r/19206/#comment68624>
This might be a my pedantic side...but since we are moving test scope from from deps can we move those under a <!-- intra-project --> comment? I have tried to separate out test and non-test deps:
https://github.com/apache/hive/blob/trunk/ql/pom.xml#L36
itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java
<https://reviews.apache.org/r/19206/#comment68625>
Should this commented out line and the ones waitForStartup be removed?
If not, please add a comment saying when we would uncomment.
pom.xml
<https://reviews.apache.org/r/19206/#comment68626>
Can you create a property and add this version to the properties?
- Brock Noland
On March 14, 2014, 7:06 a.m., Prasad Mujumdar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19206/
> -----------------------------------------------------------
>
> (Updated March 14, 2014, 7:06 a.m.)
>
>
> Review request for hive, Brock Noland and Thejas Nair.
>
>
> Bugs: HIVE-6657
> https://issues.apache.org/jira/browse/HIVE-6657
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> - Build changes
> + pull required dependenices
> + new sub-module under itest for the hive miniKdc framework, enabled for hadoop-2 only since hadoop-1 doesn't include miniKdc yet.
> - A wrapper class MiniHiveKdc on top of Hadoop miniKdc that manages the kdc services that creates test principals, keytab etc
> - Tests
> + simple unit test for MiniHiveKdc
> + Integrate MiniHiveKdc with MiniHS2
> + New HiveServer2 test that covers basic connection, as well as the token and proxy auth support added by HIVE-5155
>
> Note that Hadoop does provide a secure mini-cluster yet. Hence the new tests are not executing any queries. At this point it's limited to authentication part only.
>
>
> Diffs
> -----
>
> itests/hive-minikdc/pom.xml PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestMiniHiveKdc.java PRE-CREATION
> itests/hive-minikdc/src/test/resources/core-site.xml PRE-CREATION
> itests/hive-unit/pom.xml 6aff277
> itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java PRE-CREATION
> itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java PRE-CREATION
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java 095b989
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java d76750c
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHiveServer2.java 84f508c
> itests/pom.xml 589ba25
> pom.xml 6503c94
> service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 6759903
> service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 6fbc847
>
> Diff: https://reviews.apache.org/r/19206/diff/
>
>
> Testing
> -------
>
> New test cases using the miniKdc.
>
>
> Thanks,
>
> Prasad Mujumdar
>
>
Re: Review Request 19206: HIVE-6657: Add test coverage for Kerberos
authentication implementation using Hadoop's miniKdc
Posted by Prasad Mujumdar <pr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19206/
-----------------------------------------------------------
(Updated March 14, 2014, 7:06 a.m.)
Review request for hive, Brock Noland and Thejas Nair.
Bugs: HIVE-6657
https://issues.apache.org/jira/browse/HIVE-6657
Repository: hive-git
Description
-------
- Build changes
+ pull required dependenices
+ new sub-module under itest for the hive miniKdc framework, enabled for hadoop-2 only since hadoop-1 doesn't include miniKdc yet.
- A wrapper class MiniHiveKdc on top of Hadoop miniKdc that manages the kdc services that creates test principals, keytab etc
- Tests
+ simple unit test for MiniHiveKdc
+ Integrate MiniHiveKdc with MiniHS2
+ New HiveServer2 test that covers basic connection, as well as the token and proxy auth support added by HIVE-5155
Note that Hadoop does provide a secure mini-cluster yet. Hence the new tests are not executing any queries. At this point it's limited to authentication part only.
Diffs
-----
itests/hive-minikdc/pom.xml PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/MiniHiveKdc.java PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithMiniKdc.java PRE-CREATION
itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestMiniHiveKdc.java PRE-CREATION
itests/hive-minikdc/src/test/resources/core-site.xml PRE-CREATION
itests/hive-unit/pom.xml 6aff277
itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java PRE-CREATION
itests/hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java PRE-CREATION
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/AbstractHiveService.java 095b989
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java d76750c
itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHiveServer2.java 84f508c
itests/pom.xml 589ba25
pom.xml 6503c94
service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 6759903
service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 6fbc847
Diff: https://reviews.apache.org/r/19206/diff/
Testing
-------
New test cases using the miniKdc.
Thanks,
Prasad Mujumdar