You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Vamsee Yarlagadda <va...@cloudera.com> on 2014/01/14 03:46:41 UTC
Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE, DELETE,
RELOAD)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
Review request for sentry and Gregory Chanan.
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java, line 453
> > <https://reviews.apache.org/r/16850/diff/1/?file=422376#file422376line453>
> >
> > Can we just combine this with the function above? You can keep the function declarations the same, just call a private function with a boolean param "shouldFail" or something?
>
> Vamsee Yarlagadda wrote:
> Thanks. I understand your point. But, this is a major rework. All the other functions like deletedocsPass, deletedocsFail, queryPass, queryFail, updatePass, updateFail can also be condensed. But, i always wanted to make the functions more readable at an expense of some extra duplicate code.
I don't think it's a major rework -- it's an implementation detail on one function. I don't think you need to change every function.
The reason I noticed it this time is because the functions are not consistent, e.g. one has:
if (adminOp.equals(CollectionAction.CREATE)) {
waitForRecoveriesToFinish(collectionName, false);
}
Now, this isn't needed for the other one, but:
1) you have to spend time thinking about whether that is true or not, for no benefit
2) there's a risk something that does matter becomes inconsistent in the future.
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java, line 649
> > <https://reviews.apache.org/r/16850/diff/1/?file=422376#file422376line649>
> >
> > I think I mentioned this before, but you should at least doc that the error handling is different here.
>
> Vamsee Yarlagadda wrote:
> I added comments to the actual function being called. (verifyCollectionAdminOpPass)
>
> @param ignoreError - boolean to specify whether to ignore the error if any occurred.
> * (We may need this attribute for running DELETE command on a collection which doesn't exist)
>
Sounds good.
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java, line 64
> > <https://reviews.apache.org/r/16850/diff/1/?file=422377#file422377line64>
> >
> > Can we put these CollectionActions in an EnumList and then just iterate through them?
>
> Vamsee Yarlagadda wrote:
> Same reason as above. These admin tests are a bit different based on the expected behavior. For example, let's say a user who has enough privileges on admin will be able to CREATE, RELOAD, DELETE a collection and this part can be automated by iterating it over a loop. But, when it comes to testing a user with insufficient privileges, we can test out CREATE by automated means(by running over a loop). But in order for us to test RELOAD, DELETE; there should be a collection already present and this cannot be done by the current user. So in order to achieve this, i may need use specialized "if" blocks to execute depending on the operation being performed. That being said, i felt running the commands explicitly makes the code more readable.
>
Are you going to add more test cases to this file or would those go in other files? I understand if you can't put things in a loop in general, I was just thinking for this specific case it would work.
- Gregory
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31791
-----------------------------------------------------------
On Jan. 14, 2014, 2:46 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:46 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java, line 64
> > <https://reviews.apache.org/r/16850/diff/1/?file=422377#file422377line64>
> >
> > Can we put these CollectionActions in an EnumList and then just iterate through them?
>
> Vamsee Yarlagadda wrote:
> Same reason as above. These admin tests are a bit different based on the expected behavior. For example, let's say a user who has enough privileges on admin will be able to CREATE, RELOAD, DELETE a collection and this part can be automated by iterating it over a loop. But, when it comes to testing a user with insufficient privileges, we can test out CREATE by automated means(by running over a loop). But in order for us to test RELOAD, DELETE; there should be a collection already present and this cannot be done by the current user. So in order to achieve this, i may need use specialized "if" blocks to execute depending on the operation being performed. That being said, i felt running the commands explicitly makes the code more readable.
>
>
> Gregory Chanan wrote:
> Are you going to add more test cases to this file or would those go in other files? I understand if you can't put things in a loop in general, I was just thinking for this specific case it would work.
Ah, I see the setupCollection in the failure test...maybe comment that?
- Gregory
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31791
-----------------------------------------------------------
On Jan. 14, 2014, 2:46 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:46 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java, line 453
> > <https://reviews.apache.org/r/16850/diff/1/?file=422376#file422376line453>
> >
> > Can we just combine this with the function above? You can keep the function declarations the same, just call a private function with a boolean param "shouldFail" or something?
Thanks. I understand your point. But, this is a major rework. All the other functions like deletedocsPass, deletedocsFail, queryPass, queryFail, updatePass, updateFail can also be condensed. But, i always wanted to make the functions more readable at an expense of some extra duplicate code.
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java, line 649
> > <https://reviews.apache.org/r/16850/diff/1/?file=422376#file422376line649>
> >
> > I think I mentioned this before, but you should at least doc that the error handling is different here.
I added comments to the actual function being called. (verifyCollectionAdminOpPass)
@param ignoreError - boolean to specify whether to ignore the error if any occurred.
* (We may need this attribute for running DELETE command on a collection which doesn't exist)
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java, line 64
> > <https://reviews.apache.org/r/16850/diff/1/?file=422377#file422377line64>
> >
> > Can we put these CollectionActions in an EnumList and then just iterate through them?
Same reason as above. These admin tests are a bit different based on the expected behavior. For example, let's say a user who has enough privileges on admin will be able to CREATE, RELOAD, DELETE a collection and this part can be automated by iterating it over a loop. But, when it comes to testing a user with insufficient privileges, we can test out CREATE by automated means(by running over a loop). But in order for us to test RELOAD, DELETE; there should be a collection already present and this cannot be done by the current user. So in order to achieve this, i may need use specialized "if" blocks to execute depending on the operation being performed. That being said, i felt running the commands explicitly makes the code more readable.
- Vamsee
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31791
-----------------------------------------------------------
On Jan. 14, 2014, 2:46 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:46 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
> On Jan. 14, 2014, 8:46 p.m., Gregory Chanan wrote:
> > sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java, line 64
> > <https://reviews.apache.org/r/16850/diff/1/?file=422377#file422377line64>
> >
> > Can we put these CollectionActions in an EnumList and then just iterate through them?
>
> Vamsee Yarlagadda wrote:
> Same reason as above. These admin tests are a bit different based on the expected behavior. For example, let's say a user who has enough privileges on admin will be able to CREATE, RELOAD, DELETE a collection and this part can be automated by iterating it over a loop. But, when it comes to testing a user with insufficient privileges, we can test out CREATE by automated means(by running over a loop). But in order for us to test RELOAD, DELETE; there should be a collection already present and this cannot be done by the current user. So in order to achieve this, i may need use specialized "if" blocks to execute depending on the operation being performed. That being said, i felt running the commands explicitly makes the code more readable.
>
>
> Gregory Chanan wrote:
> Are you going to add more test cases to this file or would those go in other files? I understand if you can't put things in a loop in general, I was just thinking for this specific case it would work.
>
> Gregory Chanan wrote:
> Ah, I see the setupCollection in the failure test...maybe comment that?
Yeah, added documentation.
- Vamsee
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31791
-----------------------------------------------------------
On Jan. 15, 2014, 12:26 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 15, 2014, 12:26 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31791
-----------------------------------------------------------
Looks good, some minor suggestions below.
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java
<https://reviews.apache.org/r/16850/#comment60514>
Can we just combine this with the function above? You can keep the function declarations the same, just call a private function with a boolean param "shouldFail" or something?
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java
<https://reviews.apache.org/r/16850/#comment60515>
I think I mentioned this before, but you should at least doc that the error handling is different here.
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java
<https://reviews.apache.org/r/16850/#comment60516>
Can we put these CollectionActions in an EnumList and then just iterate through them?
- Gregory Chanan
On Jan. 14, 2014, 2:46 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:46 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31826
-----------------------------------------------------------
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java
<https://reviews.apache.org/r/16850/#comment60543>
Is there a reason we don't push this into the calling function? The calling function could just catch and ignore the exception?
This just seems like something that only the calling function cares about.
- Gregory Chanan
On Jan. 14, 2014, 2:46 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2014, 2:46 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31900
-----------------------------------------------------------
This looks good. My concern is I don't see a way (by passing some parameters or maybe via a profile) to run all the tests. Did you look into that?
Also, I had a couple of performance ideas. I know you said that creating the collections is taking the most time, but I don't think we've broken it down further. For example:
1) (see below) it looks like you are uploading the configs to ZK each time a collection is created, but this doesn't seem necessary. Maybe this is causing a significant percentage of the time? Can you measure that?
2) It depends on what exactly the bottleneck is, but you may want to investigate multithreading this. I think this is probably not straightforward because you'll need different collection names for the different threads so they don't conflict, but it may be worth looking into. Maybe use ExecutorService? I don't think this is necessary at this point, just a thought.
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java
<https://reviews.apache.org/r/16850/#comment60661>
this isn't a max, this is the actual number you are going to run, right? Rename it if so.
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java
<https://reviews.apache.org/r/16850/#comment60663>
is this necessary to do each time we create a collection? Nothing is different each run, right? Maybe we can save some time by doing it once at the beginning of the test.
- Gregory Chanan
On Jan. 15, 2014, 2:44 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 15, 2014, 2:44 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review32011
-----------------------------------------------------------
Ship it!
Ship It!
- Gregory Chanan
On Jan. 16, 2014, 2:03 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 16, 2014, 2:03 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestQueryOperations.java a9b9f28dc43b0f26c89f56ac79ab3bb0d4f68fee
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestUpdateOperations.java b57e6c66e47212479a90785bdaf3b41dab856565
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
(Updated Jan. 16, 2014, 2:03 a.m.)
Review request for sentry and Gregory Chanan.
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs (updated)
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestQueryOperations.java a9b9f28dc43b0f26c89f56ac79ab3bb0d4f68fee
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestUpdateOperations.java b57e6c66e47212479a90785bdaf3b41dab856565
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
(Updated Jan. 16, 2014, 12:08 a.m.)
Review request for sentry and Gregory Chanan.
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs (updated)
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestQueryOperations.java a9b9f28dc43b0f26c89f56ac79ab3bb0d4f68fee
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestUpdateOperations.java b57e6c66e47212479a90785bdaf3b41dab856565
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
(Updated Jan. 15, 2014, 11:59 p.m.)
Review request for sentry and Gregory Chanan.
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs (updated)
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestQueryOperations.java a9b9f28dc43b0f26c89f56ac79ab3bb0d4f68fee
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestUpdateOperations.java b57e6c66e47212479a90785bdaf3b41dab856565
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31962
-----------------------------------------------------------
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java
<https://reviews.apache.org/r/16850/#comment60725>
I think it makes more sense to have the default be ~15. That way someone just downloading sentry to try it out can run something in a reasonable amount of time.
If someone wants to have comprehensive tests they can set the parameter to something else. It would probably be good to also have some special value like "all" that represents max as well. That way you don't have to remember what the max value is.
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java
<https://reviews.apache.org/r/16850/#comment60722>
can we just parse the integer once?
- Gregory Chanan
On Jan. 15, 2014, 11:34 p.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 15, 2014, 11:34 p.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestQueryOperations.java a9b9f28dc43b0f26c89f56ac79ab3bb0d4f68fee
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestUpdateOperations.java b57e6c66e47212479a90785bdaf3b41dab856565
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
(Updated Jan. 15, 2014, 11:34 p.m.)
Review request for sentry and Gregory Chanan.
Changes
-------
Added support for reading env variable that defines the max tests to run per class.
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs (updated)
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestQueryOperations.java a9b9f28dc43b0f26c89f56ac79ab3bb0d4f68fee
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestUpdateOperations.java b57e6c66e47212479a90785bdaf3b41dab856565
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
(Updated Jan. 15, 2014, 8:24 p.m.)
Review request for sentry and Gregory Chanan.
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs (updated)
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestQueryOperations.java a9b9f28dc43b0f26c89f56ac79ab3bb0d4f68fee
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestUpdateOperations.java b57e6c66e47212479a90785bdaf3b41dab856565
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
(Updated Jan. 15, 2014, 2:44 a.m.)
Review request for sentry and Gregory Chanan.
Changes
-------
Running only a subset number of test combinations
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs (updated)
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Gregory Chanan <gc...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/#review31834
-----------------------------------------------------------
Ship it!
Ship It!
- Gregory Chanan
On Jan. 15, 2014, 12:26 a.m., Vamsee Yarlagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16850/
> -----------------------------------------------------------
>
> (Updated Jan. 15, 2014, 12:26 a.m.)
>
>
> Review request for sentry and Gregory Chanan.
>
>
> Bugs: SENTRY-88
> https://issues.apache.org/jira/browse/SENTRY-88
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Tests for checking a subset of collection/admin commands with Sentry.
> - .ini file has been pre-populated with all possible privileges.
> - Validates - CREATE,DELETE,RELOAD functions of collection API commands.
>
> Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
>
>
> Diffs
> -----
>
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
> sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
> sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
>
> Diff: https://reviews.apache.org/r/16850/diff/
>
>
> Testing
> -------
>
> Unit tests passed on MiniDFS cluster.
>
>
> Thanks,
>
> Vamsee Yarlagadda
>
>
Re: Review Request 16850: SENTRY-88: Solr Collection Admin tests (CREATE,
DELETE, RELOAD)
Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16850/
-----------------------------------------------------------
(Updated Jan. 15, 2014, 12:26 a.m.)
Review request for sentry and Gregory Chanan.
Changes
-------
Minor refactoring.
Bugs: SENTRY-88
https://issues.apache.org/jira/browse/SENTRY-88
Repository: sentry
Description
-------
Tests for checking a subset of collection/admin commands with Sentry.
- .ini file has been pre-populated with all possible privileges.
- Validates - CREATE,DELETE,RELOAD functions of collection API commands.
Takes around 460sec to complete. I timed the tests to figure out which step is taking long time. Seems like the bottleneck is around CREATEing collections(6-8sec) and this will be run for 64 different combinations. So, this explains why the test takes so long to complete.
Diffs (updated)
-----
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/AbstractSolrSentryTestBase.java fcbc67c05e90335c41b0abc93bbe43b67480f27a
sentry-tests/sentry-tests-solr/src/test/java/org/apache/sentry/tests/e2e/solr/TestCollAdminCoreOperations.java PRE-CREATION
sentry-tests/sentry-tests-solr/src/test/resources/solr/sentry/test-authz-provider.ini f988eaebea3f0bb4e67dfab179fdaff298ac55ca
Diff: https://reviews.apache.org/r/16850/diff/
Testing
-------
Unit tests passed on MiniDFS cluster.
Thanks,
Vamsee Yarlagadda