You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Emil Anca <ea...@hortonworks.com> on 2015/05/19 16:11:51 UTC
Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
Bugs: AMBARI-11238
https://issues.apache.org/jira/browse/AMBARI-11238
Repository: ambari
Description
-------
STR:
1. Install HDP 2.2 cluster.
2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
"hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
"hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
# Problem:
The Kerberos descriptor updates the values and overrides hbase ranger plugin props
# Solution:
Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
Diffs
-----
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 500d1ec
Diff: https://reviews.apache.org/r/34408/diff/
Testing
-------
HDP 2.2 with and without security/hbase ranger plugin
HDP 2.3 with and without security/hbase ranger plugin
mvn clean test -pl ambari-server
Thanks,
Emil Anca
Re: Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/#review84312
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Levas
On May 19, 2015, 10:11 a.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34408/
> -----------------------------------------------------------
>
> (Updated May 19, 2015, 10:11 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-11238
> https://issues.apache.org/jira/browse/AMBARI-11238
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> 1. Install HDP 2.2 cluster.
> 2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
> hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> 3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
> "hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
> "hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
>
> # Problem:
>
> The Kerberos descriptor updates the values and overrides hbase ranger plugin props
>
> # Solution:
>
> Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 500d1ec
>
> Diff: https://reviews.apache.org/r/34408/diff/
>
>
> Testing
> -------
>
> HDP 2.2 with and without security/hbase ranger plugin
> HDP 2.3 with and without security/hbase ranger plugin
>
> mvn clean test -pl ambari-server
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
Posted by Robert Levas <rl...@hortonworks.com>.
> On May 19, 2015, 10:13 a.m., Andrew Onischuk wrote:
> >
Unless I misunderstood what you are getting at, I think that issue is larger than time will allow to fix. Essentually, we don't want to make changes like this in the HDP stack, but in the common services. I think we (Ambari developers) are failing to use the common services and their versions as they were indended.
It appears that we are using a common service version and making updates to it within a specific stack to create a new version of that service.
What we should be doing is createing new versions of the common services and referencing them from the relevant stacks.
HBASE version 0.96.0.2.0 is referenced by HDP 2.0.6 and by inherritance is included in HDP 2.2 and HDP 2.3. However in HDP 2.2, HBASE is declared as version 0.98.4.2.2 and contains updates to the service definition it inherrited. In HDP 2.3, HBASE is declared as version 1.1.0.2.3. Technically, there should be new common service versions for HBASE (0.98.4.2.2 and 1.1.0.2.3) such that HDP 2.2 and HDP 2.3 can refernce, as well as other available stacks.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/#review84313
-----------------------------------------------------------
On May 19, 2015, 10:11 a.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34408/
> -----------------------------------------------------------
>
> (Updated May 19, 2015, 10:11 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-11238
> https://issues.apache.org/jira/browse/AMBARI-11238
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> 1. Install HDP 2.2 cluster.
> 2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
> hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> 3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
> "hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
> "hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
>
> # Problem:
>
> The Kerberos descriptor updates the values and overrides hbase ranger plugin props
>
> # Solution:
>
> Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 500d1ec
>
> Diff: https://reviews.apache.org/r/34408/diff/
>
>
> Testing
> -------
>
> HDP 2.2 with and without security/hbase ranger plugin
> HDP 2.3 with and without security/hbase ranger plugin
>
> mvn clean test -pl ambari-server
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
Posted by Emil Anca <ea...@hortonworks.com>.
> On May 19, 2015, 2:13 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py, line 277
> > <https://reviews.apache.org/r/34408/diff/1/?file=963829#file963829line277>
> >
> > We should add this as configurations as a properties. And use properties inheritance via stack.
> >
> > Rather than this checks which is basically a bad practice.
Based on our latest discussion will use xml_configurations_supported from ranger-env to determine if we are using HDP 2.3+, instead of Script.is_hdp_stack*
I'll update this once I go through all the testing needed.
- Emil
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/#review84313
-----------------------------------------------------------
On May 19, 2015, 2:11 p.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34408/
> -----------------------------------------------------------
>
> (Updated May 19, 2015, 2:11 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-11238
> https://issues.apache.org/jira/browse/AMBARI-11238
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> 1. Install HDP 2.2 cluster.
> 2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
> hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> 3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
> "hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
> "hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
>
> # Problem:
>
> The Kerberos descriptor updates the values and overrides hbase ranger plugin props
>
> # Solution:
>
> Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 500d1ec
>
> Diff: https://reviews.apache.org/r/34408/diff/
>
>
> Testing
> -------
>
> HDP 2.2 with and without security/hbase ranger plugin
> HDP 2.3 with and without security/hbase ranger plugin
>
> mvn clean test -pl ambari-server
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/#review84313
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
<https://reviews.apache.org/r/34408/#comment135527>
We should add this as configurations as a properties. And use properties inheritance via stack.
Rather than this checks which is basically a bad practice.
- Andrew Onischuk
On May 19, 2015, 2:11 p.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34408/
> -----------------------------------------------------------
>
> (Updated May 19, 2015, 2:11 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-11238
> https://issues.apache.org/jira/browse/AMBARI-11238
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> 1. Install HDP 2.2 cluster.
> 2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
> hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> 3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
> "hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
> "hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
>
> # Problem:
>
> The Kerberos descriptor updates the values and overrides hbase ranger plugin props
>
> # Solution:
>
> Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 500d1ec
>
> Diff: https://reviews.apache.org/r/34408/diff/
>
>
> Testing
> -------
>
> HDP 2.2 with and without security/hbase ranger plugin
> HDP 2.3 with and without security/hbase ranger plugin
>
> mvn clean test -pl ambari-server
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
Posted by Emil Anca <ea...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/
-----------------------------------------------------------
(Updated May 20, 2015, 2:47 p.m.)
Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
Changes
-------
Updated the 2.0.6 test stack with the xml_configurations_supported prop.
Bugs: AMBARI-11238
https://issues.apache.org/jira/browse/AMBARI-11238
Repository: ambari
Description
-------
STR:
1. Install HDP 2.2 cluster.
2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
"hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
"hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
# Problem:
The Kerberos descriptor updates the values and overrides hbase ranger plugin props
# Solution:
Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 410a690
ambari-server/src/test/python/stacks/2.0.6/configs/default.json 485e882
ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 8616f3d
Diff: https://reviews.apache.org/r/34408/diff/
Testing
-------
HDP 2.2 with and without security/hbase ranger plugin
HDP 2.3 with and without security/hbase ranger plugin
mvn clean test -pl ambari-server
Thanks,
Emil Anca
Re: Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/#review84496
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On May 20, 2015, 11:03 a.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34408/
> -----------------------------------------------------------
>
> (Updated May 20, 2015, 11:03 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-11238
> https://issues.apache.org/jira/browse/AMBARI-11238
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> 1. Install HDP 2.2 cluster.
> 2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
> hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
> 3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
> "hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
> "hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
>
> # Problem:
>
> The Kerberos descriptor updates the values and overrides hbase ranger plugin props
>
> # Solution:
>
> Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 500d1ec
>
> Diff: https://reviews.apache.org/r/34408/diff/
>
>
> Testing
> -------
>
> HDP 2.2 with and without security/hbase ranger plugin
> HDP 2.3 with and without security/hbase ranger plugin
>
> mvn clean test -pl ambari-server
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 34408: When Kerberizing a cluster with Ranger HBase
plugin enabled,
HBase coprocessor properties in hbase-site are overwritten (and breaks
Ranger HBase plugin)
Posted by Emil Anca <ea...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34408/
-----------------------------------------------------------
(Updated May 20, 2015, 11:03 a.m.)
Review request for Ambari, Andrew Onischuk, Robert Levas, and Vitalyi Brodetskyi.
Changes
-------
Andressed reviewer comments - Script.is_hdp_stack* check replaced with xml_configurations_supported
Bugs: AMBARI-11238
https://issues.apache.org/jira/browse/AMBARI-11238
Repository: ambari
Description
-------
STR:
1. Install HDP 2.2 cluster.
2. Add Ranger to the cluster via Add Service Wizard. While Ranger is added, enable Ranger HBase plugin. This changes the following properties in hbase-site:
hbase.coprocessor.master.classes: com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
hbase.coprocessor.region.classes: org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
3. Kerberization API, resets the following properties in hbase-site and invalidates settings that are needed for Ranger HBase plugin:
"hbase.coprocessor.master.classes": "org.apache.hadoop.hbase.security.access.AccessController"
"hbase.coprocessor.region.classes": "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.hadoop.hbase.security.access.AccessController"
# Problem:
The Kerberos descriptor updates the values and overrides hbase ranger plugin props
# Solution:
Since the Kerberos descriptor does not currently support conditional configuration updates, it will hold references to the two props which will be then dynamically decided on the agent side before the config file is written, during the lifecycle of hbase.
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/kerberos.json 125a9c9
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 500d1ec
Diff: https://reviews.apache.org/r/34408/diff/
Testing
-------
HDP 2.2 with and without security/hbase ranger plugin
HDP 2.3 with and without security/hbase ranger plugin
mvn clean test -pl ambari-server
Thanks,
Emil Anca