You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2014/06/18 16:05:12 UTC
Review Request 22730: Secure cluster: JCE policy files not distributed on
non-client hosts.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
Review request for Ambari and Dmitro Lisnichenko.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py fbb358f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 4a9bc42
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Jaimin Jetly <ja...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/#review46545
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py
<https://reviews.apache.org/r/22730/#comment81987>
Moving the code from before-install to before-start shared_initialization might be an issue for client-only nodes in kerberized cluster. Such nodes will not get the jce policy files
- Jaimin Jetly
On June 24, 2014, 4:34 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22730/
> -----------------------------------------------------------
>
> (Updated June 24, 2014, 4:34 p.m.)
>
>
> Review request for Ambari and Jaimin Jetly.
>
>
> Bugs: AMBARI-6185
> https://issues.apache.org/jira/browse/AMBARI-6185
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Looks like code to distribute JCE policy is in before-install hooks
> [code](https://git-wip-
> us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
> server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
> _initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
> So if no INSTALL task has executed on a host in secure cluster (for agent
> hadoop.security.authentication=kerberos is security enabled) then JCE policy
> will not be distributed and unzipped on that host
>
> Cluster can easily fall in a situation where a host has no client component.
> Following are example scenarios
>
> 1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
> 2. Adding a new host with slave components but no client components.
>
> This leads to failure of starting serviceComponent that has no client
> installed with them on a host in secure cluster.
>
> I discovered this bug while securing a cluster with just HDFS+ZK+STORM
> installed. Security wizard start all services failed with ZK quorum check
> failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
> then all services came up in secure cluster.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 6f22e79
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py 07858ad
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py fbb358f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 4a9bc42
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
>
> Diff: https://reviews.apache.org/r/22730/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
> On June 25, 2014, 4:39 p.m., Nate Cole wrote:
> > ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py, lines 53-58
> > <https://reviews.apache.org/r/22730/diff/4/?file=616603#file616603line53>
> >
> > This assertion is still required. You need tests to reflect that JCE requirements are satisfied with before-INSTALL and before-START
it will fail the test, since this thing is done in other hook which is mocked here, and will never be called, so the best would be to create separate test for BEFORE-ANY
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/#review46643
-----------------------------------------------------------
On June 25, 2014, 4:32 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22730/
> -----------------------------------------------------------
>
> (Updated June 25, 2014, 4:32 p.m.)
>
>
> Review request for Ambari, Dmytro Sen and Nate Cole.
>
>
> Bugs: AMBARI-6185
> https://issues.apache.org/jira/browse/AMBARI-6185
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Looks like code to distribute JCE policy is in before-install hooks
> [code](https://git-wip-
> us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
> server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
> _initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
> So if no INSTALL task has executed on a host in secure cluster (for agent
> hadoop.security.authentication=kerberos is security enabled) then JCE policy
> will not be distributed and unzipped on that host
>
> Cluster can easily fall in a situation where a host has no client component.
> Following are example scenarios
>
> 1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
> 2. Adding a new host with slave components but no client components.
>
> This leads to failure of starting serviceComponent that has no client
> installed with them on a host in secure cluster.
>
> I discovered this bug while securing a cluster with just HDFS+ZK+STORM
> installed. Security wizard start all services failed with ZK quorum check
> failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
> then all services came up in secure cluster.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
>
> Diff: https://reviews.apache.org/r/22730/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Nate Cole <nc...@hortonworks.com>.
> On June 25, 2014, 12:39 p.m., Nate Cole wrote:
> > ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py, lines 53-58
> > <https://reviews.apache.org/r/22730/diff/4/?file=616603#file616603line53>
> >
> > This assertion is still required. You need tests to reflect that JCE requirements are satisfied with before-INSTALL and before-START
>
> Andrew Onischuk wrote:
> it will fail the test, since this thing is done in other hook which is mocked here, and will never be called, so the best would be to create separate test for BEFORE-ANY
Then how do you know that before-ANY is invoked from before-START or before-INSTALL? Those should be mocked out and asserted that it's called.
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/#review46643
-----------------------------------------------------------
On June 25, 2014, 12:32 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22730/
> -----------------------------------------------------------
>
> (Updated June 25, 2014, 12:32 p.m.)
>
>
> Review request for Ambari, Dmytro Sen and Nate Cole.
>
>
> Bugs: AMBARI-6185
> https://issues.apache.org/jira/browse/AMBARI-6185
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Looks like code to distribute JCE policy is in before-install hooks
> [code](https://git-wip-
> us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
> server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
> _initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
> So if no INSTALL task has executed on a host in secure cluster (for agent
> hadoop.security.authentication=kerberos is security enabled) then JCE policy
> will not be distributed and unzipped on that host
>
> Cluster can easily fall in a situation where a host has no client component.
> Following are example scenarios
>
> 1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
> 2. Adding a new host with slave components but no client components.
>
> This leads to failure of starting serviceComponent that has no client
> installed with them on a host in secure cluster.
>
> I discovered this bug while securing a cluster with just HDFS+ZK+STORM
> installed. Security wizard start all services failed with ZK quorum check
> failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
> then all services came up in secure cluster.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
>
> Diff: https://reviews.apache.org/r/22730/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/#review46643
-----------------------------------------------------------
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py
<https://reviews.apache.org/r/22730/#comment82172>
This assertion is still required. You need tests to reflect that JCE requirements are satisfied with before-INSTALL and before-START
- Nate Cole
On June 25, 2014, 12:32 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22730/
> -----------------------------------------------------------
>
> (Updated June 25, 2014, 12:32 p.m.)
>
>
> Review request for Ambari, Dmytro Sen and Nate Cole.
>
>
> Bugs: AMBARI-6185
> https://issues.apache.org/jira/browse/AMBARI-6185
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Looks like code to distribute JCE policy is in before-install hooks
> [code](https://git-wip-
> us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
> server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
> _initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
> So if no INSTALL task has executed on a host in secure cluster (for agent
> hadoop.security.authentication=kerberos is security enabled) then JCE policy
> will not be distributed and unzipped on that host
>
> Cluster can easily fall in a situation where a host has no client component.
> Following are example scenarios
>
> 1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
> 2. Adding a new host with slave components but no client components.
>
> This leads to failure of starting serviceComponent that has no client
> installed with them on a host in secure cluster.
>
> I discovered this bug while securing a cluster with just HDFS+ZK+STORM
> installed. Security wizard start all services failed with ZK quorum check
> failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
> then all services came up in secure cluster.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
>
> Diff: https://reviews.apache.org/r/22730/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 25, 2014, 4:32 p.m.)
Review request for Ambari, Dmytro Sen and Nate Cole.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/#review46641
-----------------------------------------------------------
Ship it!
Ship It!
- Dmytro Sen
On June 25, 2014, 4:24 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22730/
> -----------------------------------------------------------
>
> (Updated June 25, 2014, 4:24 p.m.)
>
>
> Review request for Ambari and Nate Cole.
>
>
> Bugs: AMBARI-6185
> https://issues.apache.org/jira/browse/AMBARI-6185
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Looks like code to distribute JCE policy is in before-install hooks
> [code](https://git-wip-
> us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
> server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
> _initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
> So if no INSTALL task has executed on a host in secure cluster (for agent
> hadoop.security.authentication=kerberos is security enabled) then JCE policy
> will not be distributed and unzipped on that host
>
> Cluster can easily fall in a situation where a host has no client component.
> Following are example scenarios
>
> 1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
> 2. Adding a new host with slave components but no client components.
>
> This leads to failure of starting serviceComponent that has no client
> installed with them on a host in secure cluster.
>
> I discovered this bug while securing a cluster with just HDFS+ZK+STORM
> installed. Security wizard start all services failed with ZK quorum check
> failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
> then all services came up in secure cluster.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
> ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
>
> Diff: https://reviews.apache.org/r/22730/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 25, 2014, 4:24 p.m.)
Review request for Ambari and Nate Cole.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 25, 2014, 4:02 p.m.)
Review request for Ambari and Dmytro Sen.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 25, 2014, 4:01 p.m.)
Review request for Ambari and Jaimin Jetly.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs (updated)
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 626b199
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 6904e9d
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 05977c3
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 24, 2014, 4:34 p.m.)
Review request for Ambari and Jaimin Jetly.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 6f22e79
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py 07858ad
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py fbb358f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 4a9bc42
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 24, 2014, 3:32 p.m.)
Review request for Ambari and Dmytro Sen.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 6f22e79
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py 07858ad
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py fbb358f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 4a9bc42
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 24, 2014, 3:32 p.m.)
Review request for Ambari and Dmitro Lisnichenko.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs (updated)
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 6f22e79
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py 07858ad
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py fbb358f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 4a9bc42
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 58084d0
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py fc906b1
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 92eed49
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9274fd5
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/#review46071
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On June 18, 2014, 2:08 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22730/
> -----------------------------------------------------------
>
> (Updated June 18, 2014, 2:08 p.m.)
>
>
> Review request for Ambari and Dmitro Lisnichenko.
>
>
> Bugs: AMBARI-6185
> https://issues.apache.org/jira/browse/AMBARI-6185
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Looks like code to distribute JCE policy is in before-install hooks
> [code](https://git-wip-
> us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
> server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
> _initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
> So if no INSTALL task has executed on a host in secure cluster (for agent
> hadoop.security.authentication=kerberos is security enabled) then JCE policy
> will not be distributed and unzipped on that host
>
> Cluster can easily fall in a situation where a host has no client component.
> Following are example scenarios
>
> 1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
> 2. Adding a new host with slave components but no client components.
>
> This leads to failure of starting serviceComponent that has no client
> installed with them on a host in secure cluster.
>
> I discovered this bug while securing a cluster with just HDFS+ZK+STORM
> installed. Security wizard start all services failed with ZK quorum check
> failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
> then all services came up in secure cluster.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 6f22e79
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py 07858ad
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py fbb358f
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 4a9bc42
>
> Diff: https://reviews.apache.org/r/22730/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 22730: Secure cluster: JCE policy files not distributed
on non-client hosts.
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22730/
-----------------------------------------------------------
(Updated June 18, 2014, 2:08 p.m.)
Review request for Ambari and Dmitro Lisnichenko.
Bugs: AMBARI-6185
https://issues.apache.org/jira/browse/AMBARI-6185
Repository: ambari
Description
-------
Looks like code to distribute JCE policy is in before-install hooks
[code](https://git-wip-
us.apache.org/repos/asf/ambari/repo?p=ambari.git;a=blob;f=ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared
_initialization.py;h=a1196a8d2c997be37d65760aa3cd5de13e2cc747;hb=HEAD#l210).
So if no INSTALL task has executed on a host in secure cluster (for agent
hadoop.security.authentication=kerberos is security enabled) then JCE policy
will not be distributed and unzipped on that host
Cluster can easily fall in a situation where a host has no client component.
Following are example scenarios
1. While installing partial set of services with default selection for serviceComponent allocation to hosts in installer wizard
2. Adding a new host with slave components but no client components.
This leads to failure of starting serviceComponent that has no client
installed with them on a host in secure cluster.
I discovered this bug while securing a cluster with just HDFS+ZK+STORM
installed. Security wizard start all services failed with ZK quorum check
failure. Once I installed HDFS_CLIENT on all hosts and restarted all services
then all services came up in secure cluster.
Diffs (updated)
-----
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 3a0bf93
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 8ad8c70
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 6f22e79
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py 07858ad
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py a1196a8
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 703889e
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py fbb358f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 4a9bc42
Diff: https://reviews.apache.org/r/22730/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk