You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2017/08/21 16:34:35 UTC
Review Request 61787: zeppelin principal and livy.superusers property
do not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61787/
-----------------------------------------------------------
Review request for Ambari, Bikas Saha, Balázs Bence Sári, Eugene Chekanskiy, Laszlo Puskas, Sebastian Toader, and Jeff Zhang.
Bugs: AMBARI-21769
https://issues.apache.org/jira/browse/AMBARI-21769
Repository: ambari
Description
-------
In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are installed and Kerberos is enabled, it is expected that that `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` contain the principal name of the Zeppelin user. However, this value is not always set, depending on the order in which the services were installed, when Kerberos was enabled, and whether an Ambari or stack upgrade was involved. And if it is set, the value may be incorrect since the Kerberos descriptor assumes the Zeppelin principal is `zeppelin-<clustername>`
The solution is to move the logic to set the `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` to the stack advisor to the appropriate value can be added as needed. Also while upgrading to Ambari 2.5.2, the value(s) should be fixed if necessary.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DeconstructedPrincipal.java 764324bf0c
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java 2227675478
ambari-server/src/main/resources/common-services/SPARK/2.2.0/service_advisor.py b876cd7714
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/service_advisor.py 9ff9b8bc46
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json aa81edb2a9
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 246bbcc121
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json 872f78b4c7
ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK2/kerberos.json 0f99bbb634
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py e9b8d15b5b
ambari-server/src/main/resources/stacks/stack_advisor.py 321ac4ec71
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java b71b335b95
ambari-server/src/test/python/common-services/SPARK/2.2.0/test_service_advisor.py PRE-CREATION
ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 51d1678b9f
ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 63e2229195
Diff: https://reviews.apache.org/r/61787/diff/1/
Testing
-------
Manually tested various scenarios
# Local test results: PENDING
# Jenkins test results: PENDING
Thanks,
Robert Levas
Re: Review Request 61787: zeppelin principal and livy.superusers
property do
not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
Posted by Robert Levas <rl...@hortonworks.com>.
> On Aug. 21, 2017, 3:48 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
> > Lines 187-189 (patched)
> > <https://reviews.apache.org/r/61787/diff/1/?file=1800813#file1800813line187>
> >
> > How could this be null?
You are correct... looking at the _current_ implemention of KerberosDescriptorFactory it wouldn't be null. I sometimes get paranoid about `null`s.
> On Aug. 21, 2017, 3:48 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
> > Lines 224 (patched)
> > <https://reviews.apache.org/r/61787/diff/1/?file=1800813#file1800813line224>
> >
> > nit: MapUtils.isEmpty(map)
I should get used to using the *Utils classes. If I make any changes, I will look into that.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61787/#review183373
-----------------------------------------------------------
On Aug. 21, 2017, 12:34 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61787/
> -----------------------------------------------------------
>
> (Updated Aug. 21, 2017, 12:34 p.m.)
>
>
> Review request for Ambari, Bikas Saha, Balázs Bence Sári, Eugene Chekanskiy, Laszlo Puskas, Sebastian Toader, and Jeff Zhang.
>
>
> Bugs: AMBARI-21769
> https://issues.apache.org/jira/browse/AMBARI-21769
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are installed and Kerberos is enabled, it is expected that that `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` contain the principal name of the Zeppelin user. However, this value is not always set, depending on the order in which the services were installed, when Kerberos was enabled, and whether an Ambari or stack upgrade was involved. And if it is set, the value may be incorrect since the Kerberos descriptor assumes the Zeppelin principal is `zeppelin-<clustername>`
>
> The solution is to move the logic to set the `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` to the stack advisor to the appropriate value can be added as needed. Also while upgrading to Ambari 2.5.2, the value(s) should be fixed if necessary.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DeconstructedPrincipal.java 764324bf0c
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java 2227675478
> ambari-server/src/main/resources/common-services/SPARK/2.2.0/service_advisor.py b876cd7714
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/service_advisor.py 9ff9b8bc46
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json aa81edb2a9
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 246bbcc121
> ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json 872f78b4c7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK2/kerberos.json 0f99bbb634
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py e9b8d15b5b
> ambari-server/src/main/resources/stacks/stack_advisor.py 321ac4ec71
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java b71b335b95
> ambari-server/src/test/python/common-services/SPARK/2.2.0/test_service_advisor.py PRE-CREATION
> ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 51d1678b9f
> ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 63e2229195
>
>
> Diff: https://reviews.apache.org/r/61787/diff/1/
>
>
> Testing
> -------
>
> Manually tested various scenarios
>
> # Local test results: PENDING
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 61787: zeppelin principal and livy.superusers
property do
not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61787/#review183373
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 187-189 (patched)
<https://reviews.apache.org/r/61787/#comment259397>
How could this be null?
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
Lines 224 (patched)
<https://reviews.apache.org/r/61787/#comment259399>
nit: MapUtils.isEmpty(map)
- Nate Cole
On Aug. 21, 2017, 12:34 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61787/
> -----------------------------------------------------------
>
> (Updated Aug. 21, 2017, 12:34 p.m.)
>
>
> Review request for Ambari, Bikas Saha, Balázs Bence Sári, Eugene Chekanskiy, Laszlo Puskas, Sebastian Toader, and Jeff Zhang.
>
>
> Bugs: AMBARI-21769
> https://issues.apache.org/jira/browse/AMBARI-21769
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are installed and Kerberos is enabled, it is expected that that `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` contain the principal name of the Zeppelin user. However, this value is not always set, depending on the order in which the services were installed, when Kerberos was enabled, and whether an Ambari or stack upgrade was involved. And if it is set, the value may be incorrect since the Kerberos descriptor assumes the Zeppelin principal is `zeppelin-<clustername>`
>
> The solution is to move the logic to set the `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` to the stack advisor to the appropriate value can be added as needed. Also while upgrading to Ambari 2.5.2, the value(s) should be fixed if necessary.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DeconstructedPrincipal.java 764324bf0c
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java 2227675478
> ambari-server/src/main/resources/common-services/SPARK/2.2.0/service_advisor.py b876cd7714
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/service_advisor.py 9ff9b8bc46
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json aa81edb2a9
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 246bbcc121
> ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json 872f78b4c7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK2/kerberos.json 0f99bbb634
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py e9b8d15b5b
> ambari-server/src/main/resources/stacks/stack_advisor.py 321ac4ec71
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java b71b335b95
> ambari-server/src/test/python/common-services/SPARK/2.2.0/test_service_advisor.py PRE-CREATION
> ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 51d1678b9f
> ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 63e2229195
>
>
> Diff: https://reviews.apache.org/r/61787/diff/1/
>
>
> Testing
> -------
>
> Manually tested various scenarios
>
> # Local test results: PENDING
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 61787: zeppelin principal and livy.superusers
property do
not match on upgraded cluster from Ambari 2.4.2 -and HDP 2.5.5
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61787/#review183387
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Aug. 21, 2017, 12:34 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61787/
> -----------------------------------------------------------
>
> (Updated Aug. 21, 2017, 12:34 p.m.)
>
>
> Review request for Ambari, Bikas Saha, Balázs Bence Sári, Eugene Chekanskiy, Laszlo Puskas, Sebastian Toader, and Jeff Zhang.
>
>
> Bugs: AMBARI-21769
> https://issues.apache.org/jira/browse/AMBARI-21769
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In a cluster where the Spark and/or Spark2 Livy servers and Zeppelin are installed and Kerberos is enabled, it is expected that that `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` contain the principal name of the Zeppelin user. However, this value is not always set, depending on the order in which the services were installed, when Kerberos was enabled, and whether an Ambari or stack upgrade was involved. And if it is set, the value may be incorrect since the Kerberos descriptor assumes the Zeppelin principal is `zeppelin-<clustername>`
>
> The solution is to move the logic to set the `livy-conf/livy.superusers` and `livy2-conf/livy.superusers` to the stack advisor to the appropriate value can be added as needed. Also while upgrading to Ambari 2.5.2, the value(s) should be fixed if necessary.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DeconstructedPrincipal.java 764324bf0c
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java 2227675478
> ambari-server/src/main/resources/common-services/SPARK/2.2.0/service_advisor.py b876cd7714
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/service_advisor.py 9ff9b8bc46
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json aa81edb2a9
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 246bbcc121
> ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json 872f78b4c7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK2/kerberos.json 0f99bbb634
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py e9b8d15b5b
> ambari-server/src/main/resources/stacks/stack_advisor.py 321ac4ec71
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java b71b335b95
> ambari-server/src/test/python/common-services/SPARK/2.2.0/test_service_advisor.py PRE-CREATION
> ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 51d1678b9f
> ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 63e2229195
>
>
> Diff: https://reviews.apache.org/r/61787/diff/1/
>
>
> Testing
> -------
>
> Manually tested various scenarios
>
> # Local test results: PENDING
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>