You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Oliver Szabo <os...@hortonworks.com> on 2017/02/14 18:03:45 UTC
Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done, still writing unit tests for atlas/ranger additions.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
> On Feb. 14, 2017, 6:32 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py, line 283
> > <https://reviews.apache.org/r/56671/diff/1/?file=1633410#file1633410line283>
> >
> > Make this robust, if solr_hosts is empty, then this will throw an index out of bounds exception.
we cannot reach that state as we are only using that method if we have at least one element.
> On Feb. 14, 2017, 6:32 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py, line 262
> > <https://reviews.apache.org/r/56671/diff/1/?file=1633410#file1633410line262>
> >
> > Make this more robust. This assumes that "@" exists.
if it does not exist it returns with the full string.
> On Feb. 14, 2017, 6:32 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py, line 270
> > <https://reviews.apache.org/r/56671/diff/1/?file=1633410#file1633410line270>
> >
> > domain is never defined above so if the len is not 2 it will throw an error.
actually its defined as we get it as a parameter (from kerberos env)
but the name itself is wrong so i will rename it to realm. accidentally i used wrong naming here
- Oliver
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165538
-----------------------------------------------------------
On Feb. 14, 2017, 6:03 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 6:03 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165538
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py (line 262)
<https://reviews.apache.org/r/56671/#comment237403>
Make this more robust. This assumes that "@" exists.
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py (line 270)
<https://reviews.apache.org/r/56671/#comment237404>
domain is never defined above so if the len is not 2 it will throw an error.
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py (line 283)
<https://reviews.apache.org/r/56671/#comment237406>
Make this robust, if solr_hosts is empty, then this will throw an index out of bounds exception.
- Alejandro Fernandez
On Feb. 14, 2017, 6:03 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 6:03 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
> On Feb. 14, 2017, 6:44 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py, line 270
> > <https://reviews.apache.org/r/56671/diff/1/?file=1633410#file1633410line270>
> >
> > My point was that domain variable is never defined in the same scope as the return statement, so it assumes that the len will always be 2.
domain is overriden only if the len is 2. otherwise it uses that its in kerberos-env (with that we can define users with custom REALM)
> On Feb. 14, 2017, 6:44 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py, line 283
> > <https://reviews.apache.org/r/56671/diff/1/?file=1633410#file1633410line283>
> >
> > My point is that we should make our python code robust, since callers are free to change behavior. Just good robustness practice
i do not feel its really good, to doulbe check, but at least what i can do is throw an exception when the method is used wrongly.
- Oliver
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165546
-----------------------------------------------------------
On Feb. 14, 2017, 6:40 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 6:40 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165546
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py (line 270)
<https://reviews.apache.org/r/56671/#comment237411>
My point was that domain variable is never defined in the same scope as the return statement, so it assumes that the len will always be 2.
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py (line 283)
<https://reviews.apache.org/r/56671/#comment237413>
My point is that we should make our python code robust, since callers are free to change behavior. Just good robustness practice
- Alejandro Fernandez
On Feb. 14, 2017, 6:40 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 6:40 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
> On Feb. 15, 2017, 11:31 a.m., Mugdha Varadkar wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py, line 75
> > <https://reviews.apache.org/r/56671/diff/3/?file=1633514#file1633514line75>
> >
> > How is this going to get uploaded to zookeeper node ? or is this a manual task for user using custom-security.json ?
> >
> > If infra-solr is restarted the custom-security.json will get override with static security.json template
yes you are right, thanks.
if the custom-security.json is not empty, i will use that instead of the template one in solr_cloud_util.setup_kerberos_plugin function
- Oliver
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165683
-----------------------------------------------------------
On Feb. 14, 2017, 7:06 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 7:06 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165683
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py (line 75)
<https://reviews.apache.org/r/56671/#comment237536>
How is this going to get uploaded to zookeeper node ? or is this a manual task for user using custom-security.json ?
If infra-solr is restarted the custom-security.json will get override with static security.json template
- Mugdha Varadkar
On Feb. 14, 2017, 7:06 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 7:06 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Robert Nettleton <rn...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165584
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Nettleton
On Feb. 14, 2017, 7:06 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 7:06 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165571
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On Feb. 14, 2017, 7:06 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 7:06 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
> On Feb. 15, 2017, 11:45 a.m., Mugdha Varadkar wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json, line 75
> > <https://reviews.apache.org/r/56671/diff/3/?file=1633521#file1633521line75>
> >
> > 0.6.0 version is extended from stack 2.5 which is released.
> > If user upgrading to ambari with this fix, will infra-solr principal get generated for ranger-admin host after ambari upgrade ?
it can be a possible issue because i do not think so any restart will regenerate keytabs. (on update you need to restart ambari managed services). regenerating keytabs will be needed after upgrade
- Oliver
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165686
-----------------------------------------------------------
On Feb. 14, 2017, 7:06 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 7:06 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Robert Levas <rl...@hortonworks.com>.
> On Feb. 15, 2017, 6:45 a.m., Mugdha Varadkar wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json, line 75
> > <https://reviews.apache.org/r/56671/diff/3/?file=1633521#file1633521line75>
> >
> > 0.6.0 version is extended from stack 2.5 which is released.
> > If user upgrading to ambari with this fix, will infra-solr principal get generated for ranger-admin host after ambari upgrade ?
>
> Oliver Szabo wrote:
> it can be a possible issue because i do not think so any restart will regenerate keytabs. (on update you need to restart ambari managed services). regenerating keytabs will be needed after upgrade
Ambari will not create the missing Kerberos identities on restart. The user will need to manually perform a "Regenerate Keytabs" operation (selecting the "missing only" option) in order to create the new Kerberos identities and distribute the keytab files where needed.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165686
-----------------------------------------------------------
On Feb. 15, 2017, 9:36 a.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2017, 9:36 a.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
> ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
> ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
> ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
> ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165686
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json (line 75)
<https://reviews.apache.org/r/56671/#comment237540>
0.6.0 version is extended from stack 2.5 which is released.
If user upgrading to ambari with this fix, will infra-solr principal get generated for ranger-admin host after ambari upgrade ?
- Mugdha Varadkar
On Feb. 14, 2017, 7:06 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 7:06 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for atlas/ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
> On Feb. 15, 2017, 7:31 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml, line 103
> > <https://reviews.apache.org/r/56671/diff/5/?file=1634467#file1634467line103>
> >
> > If the user simply upgrades Ambari and not the stack, should these properties be added?
yes, this should be added during ambari upgrade
> On Feb. 15, 2017, 7:31 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py, line 218
> > <https://reviews.apache.org/r/56671/diff/5/?file=1634472#file1634472line218>
> >
> > Add some documentation to the new functions.
it only dispatches to solr_cloud_util.secure_znode function which already has the documentation.
- Oliver
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165749
-----------------------------------------------------------
On Feb. 15, 2017, 2:36 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2017, 2:36 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
> ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
> ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
> ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
> ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165749
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml (line 89)
<https://reviews.apache.org/r/56671/#comment237603>
If the user simply upgrades Ambari and not the stack, should these properties be added?
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py (line 218)
<https://reviews.apache.org/r/56671/#comment237602>
Add some documentation to the new functions.
- Alejandro Fernandez
On Feb. 15, 2017, 2:36 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2017, 2:36 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
> ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
> ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
> ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
> ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
> On Feb. 15, 2017, 5:36 p.m., Mugdha Varadkar wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py, line 679
> > <https://reviews.apache.org/r/56671/diff/5/?file=1634479#file1634479line679>
> >
> > Please add defaults for atlas, yarn and kms. For kms the default principal name is rangerkms.
thanks, i forget to add yarn and used wrong default principal for kms. actually atlas is not needed because that is done on atlas side (also atlas needs other permissions as well to access its own collections)
- Oliver
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165727
-----------------------------------------------------------
On Feb. 15, 2017, 2:36 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2017, 2:36 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
> ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
> ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
> ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
> ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165727
-----------------------------------------------------------
Fix it, then Ship it!
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py (line 679)
<https://reviews.apache.org/r/56671/#comment237578>
Please add defaults for atlas, yarn and kms. For kms the default principal name is rangerkms.
- Mugdha Varadkar
On Feb. 15, 2017, 2:36 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2017, 2:36 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
> ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
> ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
> ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
> ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 16, 2017, 7:28 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
- update solr test (update permission)
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 16, 2017, 3:57 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
small additions:
- make sure both security.json are generated
- do not accept spaces for custom-security.json
- change both security.json files permission
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 16, 2017, 1:28 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
add missing end property tag
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 15, 2017, 7:58 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
- put documentation before "import params"
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 15, 2017, 7:56 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
wrong patch uploaded - fix it
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 15, 2017, 7:55 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
- add rangerkms, yarn principals
- move get_ranger_plugin_principals function into ranger stack
- add ranger audit permission to atlas
- use orderedDict instead of simple map -> as we are asserting the call during upgrade, its needed to have the same order every time
- upgrade tests
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/MessageStatus.java PRE-CREATION
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/ApiDocConfig.java 86c1edd
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java 0568fd7
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java caf0636
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java 2dc0ef7
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java f960250
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/AuditLogsResource.java d5b6525
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/rest/ServiceLogsResource.java 40247a8
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 15, 2017, 2:36 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
- add changes to ranger admin
- put back check znode for logsearch
- add "dev" role for atlas/ranger users
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description (updated)
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py 0b4532b
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py ae49c4f
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py db9cbb9
ambari-server/src/test/python/stacks/2.5/RANGER/test_ranger_admin.py b01e7da
ambari-server/src/test/python/stacks/2.6/RANGER/test_ranger_admin.py 8dda363
Diff: https://reviews.apache.org/r/56671/diff/
Testing (updated)
-------
unit tests done.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Miklos Gergely <mg...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/#review165702
-----------------------------------------------------------
Ship it!
Ship It!
- Miklos Gergely
On Feb. 15, 2017, 12:27 p.m., Oliver Szabo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56671/
> -----------------------------------------------------------
>
> (Updated Feb. 15, 2017, 12:27 p.m.)
>
>
> Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
>
>
> Bugs: AMBARI-20013
> https://issues.apache.org/jira/browse/AMBARI-20013
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
> - created a new solr_cloud_util method called add roles (curl for adding user roles)
> - updated secure znode method (adding infra-solr by default)
> - move generated security.json into a static file
> - added a custom security.json which can be used instead of the generated security.json
>
> This patch still does not contain ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
>
> In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
>
> solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
> solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
> jaas_file=params.logsearch_jaas_file,
> java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
>
> and
>
> solr_cloud_util.add_solr_roles(params.config,
> roles = ["ranger_audit_user"],
> new_service_principals = ["audituser1", "audituser2"])
>
> about ranger plugins:
>
> there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
> ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
> ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
> ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
>
> Diff: https://reviews.apache.org/r/56671/diff/
>
>
> Testing
> -------
>
> unit tests done, still writing unit tests for ranger additions.
>
>
> Thanks,
>
> Oliver Szabo
>
>
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 15, 2017, 12:27 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
added atlas, so remove from the description
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description (updated)
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
This patch still does not contain ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
Diff: https://reviews.apache.org/r/56671/diff/
Testing (updated)
-------
unit tests done, still writing unit tests for ranger additions.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 15, 2017, 12:22 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
- use custom-security.json if not empty
- add role & securing znode changes to atlas stack
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 2232bb2
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 682fc9f
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 1bbf75e
ambari-server/src/test/python/stacks/2.3/configs/secure.json 4501b81
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done, still writing unit tests for atlas/ranger additions.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 7:06 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
throw Exception in case of principal, realm or solr hosts param is missing.
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done, still writing unit tests for atlas/ranger additions.
Thanks,
Oliver Szabo
Re: Review Request 56671: Add Solr authorization settings during
LogSearch/Atlas/Ranger startup
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56671/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 6:40 p.m.)
Review request for Ambari, Miklos Gergely, Mugdha Varadkar, and Robert Nettleton.
Changes
-------
rename domain to realm
Bugs: AMBARI-20013
https://issues.apache.org/jira/browse/AMBARI-20013
Repository: ambari
Description
-------
- make sure solr keytab and solr user is created on logsearch/ranger/atlas hosts
- created a new solr_cloud_util method called add roles (curl for adding user roles)
- updated secure znode method (adding infra-solr by default)
- move generated security.json into a static file
- added a custom security.json which can be used instead of the generated security.json
This patch still does not contain atlas/ranger fixes (it will, that review is mosly for review, i will include ranger/atlas changes later).
In ranger/atlas stack, the following calls can be used for securing znode and update user-roles:
solr_cloud_util.secure_znode(config=params.config, zookeeper_quorum=params.zookeeper_quorum,
solr_znode=format("{infra_solr_znode}/collections/mycollectionznode"),
jaas_file=params.logsearch_jaas_file,
java64_home=params.java64_home, sasl_users=["myuser1@HOST", "myuser2"])
and
solr_cloud_util.add_solr_roles(params.config,
roles = ["ranger_audit_user"],
new_service_principals = ["audituser1", "audituser2"])
about ranger plugins:
there is a way to use these methods in other services like storm etc. , the problem is its needed to infra-solr be started, so if we include these in a lot of other services, it can really slows down the deployment. that is the reason why i kept the property dependencies in infra-solr-security-json config. other then that, with the changes ranger can start successfully if its added later to the cluster, and for plugins, infra-solr will be flagged to be restarted.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4628211
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-security-json.xml e193a8c
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py ab9aa61
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py 8d72f42
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/templates/infra-solr-security.json.j2 PRE-CREATION
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-security.json.j2 ed764f0
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json bc8e351
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/kerberos.json 49d1b10
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py fecd802
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py ba91e20
ambari-server/src/main/resources/common-services/RANGER/0.6.0/kerberos.json 253e32e
ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py cd88fec
Diff: https://reviews.apache.org/r/56671/diff/
Testing
-------
unit tests done, still writing unit tests for atlas/ranger additions.
Thanks,
Oliver Szabo