You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by kalyan kumar kalvagadda <kk...@cloudera.com> on 2017/07/12 16:44:37 UTC
Re: Review Request 58727: SENTRY-1708 Extend the current test classes
to handle multiple sentry servers
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58727/
-----------------------------------------------------------
(Updated July 12, 2017, 4:44 p.m.)
Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, Vamsee Yarlagadda, and Vadim Spector.
Changes
-------
There is one test failure doesn;t seem to be related to changes done.I'm looking into it.
Bugs: SENTRY-1708
https://issues.apache.org/jira/browse/SENTRY-1708
Repository: sentry
Description (updated)
-------
Here is the abstract of the code changes done. There are basically changes to the base classes used tests to be able to handle multiple servers
* List of sentry sentry servers
* List of the address used for these servers
* This is learnt so re-use the same port when the servers are re-started
* API's to start/stop all the servers
* API's to start/stop specific servers
* API to learn the server addresses
* API to update the sentry server config list for the clients to use
* API to for service failover
There are some HA test classes added to demonstrate the how these abstract base classes can be used to perform HA tests.
Model HA Tests using Abstract class providing Ha capability
* TestColumnEndToEndWithHa
* TestGrantUserToRoleWithHa
* TestPrivilegeWithHAGrantOption
Diffs (updated)
-----
sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/transport/SentryClientTransportConstants.java fd07887
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/SentryHdfsServiceIntegrationBase.java 3ee3724
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/generic/service/thrift/TestSentryGenericServiceClient.java 8959ad8
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryPolicyServiceClient.java 3b3b30e
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryPolicyServiceClientWithHa.java PRE-CREATION
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceClientPool.java fe4164d
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceFailureCase.java 2f4e8f6
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceMetrics.java a33b03a
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceWithInvalidMsgSize.java 32e67b9
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/service/thrift/SentryServiceIntegrationBase.java 6895720
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/AbstractTestWithDbProvider.java b416ef8
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestDbSentryOnFailureHookLoading.java aa99595
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestGrantUserToRole.java 5364937
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestPrivilegeWithHAGrantOption.java f3db301
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/ha/TestColumnEndToEndWithHa.java PRE-CREATION
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/ha/TestGrantUserToRoleWithHa.java PRE-CREATION
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java 6ea6763
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/InternalSentrySrv.java 4cfb1f7
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/SentrySrv.java 9139706
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/SentrySrvFactory.java 9381e88
sentry-tests/sentry-tests-kafka/src/test/java/org/apache/sentry/tests/e2e/kafka/AbstractKafkaSentryTestBase.java 7c45999
sentry-tests/sentry-tests-kafka/src/test/java/org/apache/sentry/tests/e2e/kafka/TestAclsCrud.java a52c8d6
sentry-tests/sentry-tests-sqoop/src/test/java/org/apache/sentry/tests/e2e/sqoop/AbstractSqoopSentryTestBase.java 80f158a
Diff: https://reviews.apache.org/r/58727/diff/4/
Changes: https://reviews.apache.org/r/58727/diff/3-4/
Testing (updated)
-------
Made sure that all the unit tests passed.
Thanks,
kalyan kumar kalvagadda
Re: Review Request 58727: SENTRY-1708 Extend the current test classes
to handle multiple sentry servers
Posted by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58727/
-----------------------------------------------------------
(Updated Feb. 20, 2018, 11:27 p.m.)
Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, Vamsee Yarlagadda, and Vadim Spector.
Changes
-------
Addressed review comments.
Bugs: SENTRY-1708
https://issues.apache.org/jira/browse/SENTRY-1708
Repository: sentry
Description
-------
Here is the abstract of the code changes done. There are basically changes to the base classes used tests to be able to handle multiple servers
* List of sentry sentry servers
* List of the address used for these servers
* This is learnt so re-use the same port when the servers are re-started
* API's to start/stop all the servers
* API's to start/stop specific servers
* API to learn the server addresses
* API to update the sentry server config list for the clients to use
* API to for service failover
There are some HA test classes added to demonstrate the how these abstract base classes can be used to perform HA tests.
Model HA Tests using Abstract class providing Ha capability
* TestColumnEndToEndWithHa
* TestGrantUserToRoleWithHa
* TestPrivilegeWithHAGrantOption
Diffs (updated)
-----
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java eae7861728f2bc11b4c1b44aa3b61b881a87740b
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java cf764eda1a006ce96f301e3ecb87749e05ba4a09
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/HMSFollower.java 2f2b98412e7dfdcc847ffe7975a70f452554e747
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/NotificationProcessor.java e7558370025c6acd83492615be093f2bd16a202b
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java edea5b64d8f98c93aafc1fe43fa97e00c2ce2948
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java 96c6810baa4d554db2b7d3739a28e3ff7e8b33a0
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestHMSFollower.java 79030780c35e5bda432e3ec3f01328e627cb50a6
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java 4cd00e6672730773c74b9840247d1f4d5f7bdfe4
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationTogglingConf.java PRE-CREATION
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/InternalSentrySrv.java e64f5cd687bf59133d6475c912ebdd7930601151
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/SentrySrv.java 91397061e78a6524410d35643dd3a33be8353ecc
Diff: https://reviews.apache.org/r/58727/diff/5/
Changes: https://reviews.apache.org/r/58727/diff/4-5/
Testing
-------
Made sure that all the unit tests passed.
Thanks,
kalyan kumar kalvagadda
Re: Review Request 58727: SENTRY-1708 Extend the current test classes
to handle multiple sentry servers
Posted by kalyan kumar kalvagadda <kk...@cloudera.com>.
> On July 13, 2017, 1:43 a.m., Na Li wrote:
> > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestPrivilegeWithHAGrantOption.java
> > Line 37 (original), 34 (patched)
> > <https://reviews.apache.org/r/58727/diff/4/?file=1774911#file1774911line37>
> >
> > do you want to enable these tests
i can enable it.
- kalyan kumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58727/#review180389
-----------------------------------------------------------
On July 12, 2017, 4:44 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58727/
> -----------------------------------------------------------
>
> (Updated July 12, 2017, 4:44 p.m.)
>
>
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, Vamsee Yarlagadda, and Vadim Spector.
>
>
> Bugs: SENTRY-1708
> https://issues.apache.org/jira/browse/SENTRY-1708
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Here is the abstract of the code changes done. There are basically changes to the base classes used tests to be able to handle multiple servers
> * List of sentry sentry servers
> * List of the address used for these servers
> * This is learnt so re-use the same port when the servers are re-started
> * API's to start/stop all the servers
> * API's to start/stop specific servers
> * API to learn the server addresses
> * API to update the sentry server config list for the clients to use
> * API to for service failover
>
> There are some HA test classes added to demonstrate the how these abstract base classes can be used to perform HA tests.
> Model HA Tests using Abstract class providing Ha capability
>
> * TestColumnEndToEndWithHa
> * TestGrantUserToRoleWithHa
> * TestPrivilegeWithHAGrantOption
>
>
> Diffs
> -----
>
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/transport/SentryClientTransportConstants.java fd07887
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/SentryHdfsServiceIntegrationBase.java 3ee3724
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/generic/service/thrift/TestSentryGenericServiceClient.java 8959ad8
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryPolicyServiceClient.java 3b3b30e
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryPolicyServiceClientWithHa.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceClientPool.java fe4164d
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceFailureCase.java 2f4e8f6
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceMetrics.java a33b03a
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceWithInvalidMsgSize.java 32e67b9
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/service/thrift/SentryServiceIntegrationBase.java 6895720
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/AbstractTestWithDbProvider.java b416ef8
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestDbSentryOnFailureHookLoading.java aa99595
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestGrantUserToRole.java 5364937
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestPrivilegeWithHAGrantOption.java f3db301
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/ha/TestColumnEndToEndWithHa.java PRE-CREATION
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/ha/TestGrantUserToRoleWithHa.java PRE-CREATION
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java 6ea6763
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/InternalSentrySrv.java 4cfb1f7
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/SentrySrv.java 9139706
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/SentrySrvFactory.java 9381e88
> sentry-tests/sentry-tests-kafka/src/test/java/org/apache/sentry/tests/e2e/kafka/AbstractKafkaSentryTestBase.java 7c45999
> sentry-tests/sentry-tests-kafka/src/test/java/org/apache/sentry/tests/e2e/kafka/TestAclsCrud.java a52c8d6
> sentry-tests/sentry-tests-sqoop/src/test/java/org/apache/sentry/tests/e2e/sqoop/AbstractSqoopSentryTestBase.java 80f158a
>
>
> Diff: https://reviews.apache.org/r/58727/diff/4/
>
>
> Testing
> -------
>
> Made sure that all the unit tests passed.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 58727: SENTRY-1708 Extend the current test classes
to handle multiple sentry servers
Posted by Na Li <li...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58727/#review180389
-----------------------------------------------------------
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestPrivilegeWithHAGrantOption.java
Line 37 (original), 34 (patched)
<https://reviews.apache.org/r/58727/#comment255538>
do you want to enable these tests
- Na Li
On July 12, 2017, 4:44 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58727/
> -----------------------------------------------------------
>
> (Updated July 12, 2017, 4:44 p.m.)
>
>
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, Vamsee Yarlagadda, and Vadim Spector.
>
>
> Bugs: SENTRY-1708
> https://issues.apache.org/jira/browse/SENTRY-1708
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Here is the abstract of the code changes done. There are basically changes to the base classes used tests to be able to handle multiple servers
> * List of sentry sentry servers
> * List of the address used for these servers
> * This is learnt so re-use the same port when the servers are re-started
> * API's to start/stop all the servers
> * API's to start/stop specific servers
> * API to learn the server addresses
> * API to update the sentry server config list for the clients to use
> * API to for service failover
>
> There are some HA test classes added to demonstrate the how these abstract base classes can be used to perform HA tests.
> Model HA Tests using Abstract class providing Ha capability
>
> * TestColumnEndToEndWithHa
> * TestGrantUserToRoleWithHa
> * TestPrivilegeWithHAGrantOption
>
>
> Diffs
> -----
>
> sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/transport/SentryClientTransportConstants.java fd07887
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/SentryHdfsServiceIntegrationBase.java 3ee3724
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/generic/service/thrift/TestSentryGenericServiceClient.java 8959ad8
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryPolicyServiceClient.java 3b3b30e
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryPolicyServiceClientWithHa.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceClientPool.java fe4164d
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceFailureCase.java 2f4e8f6
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceMetrics.java a33b03a
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/TestSentryServiceWithInvalidMsgSize.java 32e67b9
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/service/thrift/SentryServiceIntegrationBase.java 6895720
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/AbstractTestWithDbProvider.java b416ef8
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestDbSentryOnFailureHookLoading.java aa99595
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestGrantUserToRole.java 5364937
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestPrivilegeWithHAGrantOption.java f3db301
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/ha/TestColumnEndToEndWithHa.java PRE-CREATION
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/ha/TestGrantUserToRoleWithHa.java PRE-CREATION
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java 6ea6763
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/InternalSentrySrv.java 4cfb1f7
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/SentrySrv.java 9139706
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/minisentry/SentrySrvFactory.java 9381e88
> sentry-tests/sentry-tests-kafka/src/test/java/org/apache/sentry/tests/e2e/kafka/AbstractKafkaSentryTestBase.java 7c45999
> sentry-tests/sentry-tests-kafka/src/test/java/org/apache/sentry/tests/e2e/kafka/TestAclsCrud.java a52c8d6
> sentry-tests/sentry-tests-sqoop/src/test/java/org/apache/sentry/tests/e2e/sqoop/AbstractSqoopSentryTestBase.java 80f158a
>
>
> Diff: https://reviews.apache.org/r/58727/diff/4/
>
>
> Testing
> -------
>
> Made sure that all the unit tests passed.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>