You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Qiang Zhang <zh...@zte.com.cn> on 2018/12/13 06:52:03 UTC
Re: Review Request 68128: RANGER-2170:Ranger supports plugin to
enable, monitor and manage Elasticsearch
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/
-----------------------------------------------------------
(Updated 十二月 13, 2018, 6:52 a.m.)
Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy.
Changes
-------
Update to resolve file confilict~
Bugs: RANGER-2170
https://issues.apache.org/jira/browse/RANGER-2170
Repository: ranger
Description
-------
Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases.
Like Apache Solr, it is also an index server based on Lucence.
Ranger supports plugin to enable, monitor and manage Elasticsearch,
to control index security of Elasticsearch.
As there is X-Pack plugin for the Elasticsearch, but it is not free.
X-Pack is an Elastic Stack extension that bundles security, alerting, monitoring, reporting,
and graph capabilities into one easy-to-install package.
We refer to the Indices Privileges design of X-Pack,
by keeping the permissions consistent,
to make user use ranger Elasticsearch plugin easily.
Reference X-Pack Indices Privileges:
https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 6.2.2.
Elasticsearch 6.2.2 was released in February 20, 2018, reference release-notes:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
Not like other system, Elasticsearch has no basic authentication,
it uses X-pack plugin to support basic authentication,
role-based access control, SSL/TLS encryption, LDAP and so on.
Not like X-pack, our Ranger Elasticsearch plugin is designed to do authorization,
it is to control index of Elasticsearch without authentication,
this plugin should work with other Elasticsearch plugin to authenticate users.
Diffs (updated)
-----
agents-common/scripts/enable-agent.sh ce0dc8c
agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java e654f2b
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 118af1f
agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json PRE-CREATION
plugin-elasticsearch/.gitignore PRE-CREATION
plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg PRE-CREATION
plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION
plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg PRE-CREATION
plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION
plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION
plugin-elasticsearch/pom.xml PRE-CREATION
plugin-elasticsearch/scripts/install.properties PRE-CREATION
plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java PRE-CREATION
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java PRE-CREATION
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java PRE-CREATION
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java PRE-CREATION
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java PRE-CREATION
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java PRE-CREATION
pom.xml a11cf51
ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION
ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties PRE-CREATION
ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION
ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java PRE-CREATION
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java PRE-CREATION
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java PRE-CREATION
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java PRE-CREATION
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java PRE-CREATION
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/rest/filter/RangerSecurityRestFilter.java PRE-CREATION
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/utils/RequestUtils.java PRE-CREATION
src/main/assembly/admin-web.xml b3ec885
src/main/assembly/plugin-elasticsearch.xml PRE-CREATION
Diff: https://reviews.apache.org/r/68128/diff/3/
Changes: https://reviews.apache.org/r/68128/diff/2-3/
Testing
-------
#Test Steps:
1.Intall
Ranger Elasticsearch Plugin Installation Guide
https://cwiki.apache.org/confluence/display/RANGER/Elasticsearch+Plugin
Include install Elasticsearch and Ranger Elasticsearch Plugin,
and verify install result.
2.Create policy in Ranger Admin
User "elasticsearch" has all permissions on all indices.
User "yuwen" has permission "read" on index "twitter".
3.Test permission
3.1 successful:
curl -u elasticsearch:xxx -X GET "localhost:9200/twitter/_stats?pretty"
curl -u elasticsearch:xxx -X GET "localhost:9200/twitter2/_stats?pretty"
curl -u yuwen:xxx -X GET "localhost:9200/twitter/_stats?pretty"
3.2 failed:
curl -X GET "localhost:9200/twitter/_stats?pretty"
{
"error" : {
"root_cause" : [
{
"type" : "status_exception",
"reason" : "Error: User is null, the request requires user authentication."
}
],
"type" : "status_exception",
"reason" : "Error: User is null, the request requires user authentication."
},
"status" : 401
}
curl -u yuwen:xxx -X GET "localhost:9200/twitter2/_stats?pretty"
{
"error" : {
"root_cause" : [
{
"type" : "status_exception",
"reason" : "Error: User[yuwen] could not do action[indices:monitor/stats] on index[twitter2]"
}
],
"type" : "status_exception",
"reason" : "Error: User[yuwen] could not do action[indices:monitor/stats] on index[twitter2]"
},
"status" : 403
}
Thanks,
Qiang Zhang
Re: Review Request 68128: RANGER-2170:Ranger supports plugin to
enable, monitor and manage Elasticsearch
Posted by pengjianhua <pe...@zte.com.cn>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/#review211549
-----------------------------------------------------------
Ship it!
Ship It!
- pengjianhua
On 十二月 13, 2018, 6:52 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68128/
> -----------------------------------------------------------
>
> (Updated 十二月 13, 2018, 6:52 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2170
> https://issues.apache.org/jira/browse/RANGER-2170
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases.
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
>
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, monitoring, reporting,
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
>
> Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 6.2.2.
> Elasticsearch 6.2.2 was released in February 20, 2018, reference release-notes:
> https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
> Not like other system, Elasticsearch has no basic authentication,
> it uses X-pack plugin to support basic authentication,
> role-based access control, SSL/TLS encryption, LDAP and so on.
> Not like X-pack, our Ranger Elasticsearch plugin is designed to do authorization,
> it is to control index of Elasticsearch without authentication,
> this plugin should work with other Elasticsearch plugin to authenticate users.
>
>
> Diffs
> -----
>
> agents-common/scripts/enable-agent.sh ce0dc8c
> agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java e654f2b
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 118af1f
> agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json PRE-CREATION
> plugin-elasticsearch/.gitignore PRE-CREATION
> plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg PRE-CREATION
> plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION
> plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg PRE-CREATION
> plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION
> plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
> plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION
> plugin-elasticsearch/pom.xml PRE-CREATION
> plugin-elasticsearch/scripts/install.properties PRE-CREATION
> plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java PRE-CREATION
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java PRE-CREATION
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java PRE-CREATION
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java PRE-CREATION
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java PRE-CREATION
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java PRE-CREATION
> pom.xml a11cf51
> ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION
> ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties PRE-CREATION
> ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION
> ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java PRE-CREATION
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java PRE-CREATION
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java PRE-CREATION
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java PRE-CREATION
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java PRE-CREATION
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/rest/filter/RangerSecurityRestFilter.java PRE-CREATION
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/utils/RequestUtils.java PRE-CREATION
> src/main/assembly/admin-web.xml b3ec885
> src/main/assembly/plugin-elasticsearch.xml PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/68128/diff/3/
>
>
> Testing
> -------
>
> #Test Steps:
>
> 1.Intall
> Ranger Elasticsearch Plugin Installation Guide
> https://cwiki.apache.org/confluence/display/RANGER/Elasticsearch+Plugin
> Include install Elasticsearch and Ranger Elasticsearch Plugin,
> and verify install result.
>
> 2.Create policy in Ranger Admin
> User "elasticsearch" has all permissions on all indices.
> User "yuwen" has permission "read" on index "twitter".
>
> 3.Test permission
>
> 3.1 successful:
> curl -u elasticsearch:xxx -X GET "localhost:9200/twitter/_stats?pretty"
> curl -u elasticsearch:xxx -X GET "localhost:9200/twitter2/_stats?pretty"
> curl -u yuwen:xxx -X GET "localhost:9200/twitter/_stats?pretty"
>
> 3.2 failed:
> curl -X GET "localhost:9200/twitter/_stats?pretty"
> {
> "error" : {
> "root_cause" : [
> {
> "type" : "status_exception",
> "reason" : "Error: User is null, the request requires user authentication."
> }
> ],
> "type" : "status_exception",
> "reason" : "Error: User is null, the request requires user authentication."
> },
> "status" : 401
> }
>
> curl -u yuwen:xxx -X GET "localhost:9200/twitter2/_stats?pretty"
> {
> "error" : {
> "root_cause" : [
> {
> "type" : "status_exception",
> "reason" : "Error: User[yuwen] could not do action[indices:monitor/stats] on index[twitter2]"
> }
> ],
> "type" : "status_exception",
> "reason" : "Error: User[yuwen] could not do action[indices:monitor/stats] on index[twitter2]"
> },
> "status" : 403
> }
>
>
> Thanks,
>
> Qiang Zhang
>
>