You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/12/14 07:25:59 UTC
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/
-----------------------------------------------------------
(Updated Dec. 14, 2015, 6:25 a.m.)
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
Changes
-------
Addressed comments and fixed unit tests.
Summary (updated)
-----------------
AMBARI-14307. RU: Kafka broker restart failed on downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Bugs: AMBARI-14307
https://issues.apache.org/jira/browse/AMBARI-14307
Repository: ambari
Description
-------
* Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
* HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
* RU from to 2.3.4.0 and reach the Finalize screen.
* Then downgrade,
Kafka broker restart failed on downgrade
```
Traceback (most recent call last):
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
BeforeStartHook().execute()
File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
method(env)
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
create_topology_script_and_mapping()
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
create_topology_mapping()
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
group=params.user_group)
File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
self.env.run()
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
self.run_action(resource, action)
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
provider_action()
File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
```
This failed because /etc/hadoop/conf is now a circular symlink
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7fd64b5
ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25
ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db
ambari-server/src/test/python/custom_actions/test_ru_set_all.py 186b847
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py d2ffb3c
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9e4560c
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 5fd5ef6
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 72cb050
Diff: https://reviews.apache.org/r/41273/diff/
Testing (updated)
-------
Tested on the following code paths,
1. Fresh install of HDP 2.1
2. Installed bits for HDP 2.3 and verified the conf folder structure
3. EU to HDP 2.3
4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
1. Fresh install of HDP 2.2
2. Installed bits for HDP 2.3 and verfied the conf folder structure
3. RU to HDP 2.3 and downgrade
Python unit tests passed,
----------------------------------------------------------------------
Total run:853
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110240
-----------------------------------------------------------
Ship it!
Looks good
- Dmitro Lisnichenko
On Dec. 14, 2015, 9:17 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2015, 9:17 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14307
> https://issues.apache.org/jira/browse/AMBARI-14307
>
>
> Repository: ambari
>
>
> Description
> -------
>
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
>
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
> BeforeStartHook().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
> method(env)
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
> create_topology_script_and_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
> create_topology_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
> group=params.user_group)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
> raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
>
>
> This failed because /etc/hadoop/conf is now a circular symlink
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7fd64b5
> ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py 186b847
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py d2ffb3c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9e4560c
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 5fd5ef6
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 72cb050
>
> Diff: https://reviews.apache.org/r/41273/diff/
>
>
> Testing
> -------
>
> Tested on the following code paths,
>
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
>
>
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:853
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Dec. 14, 2015, 6:52 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py, lines 188-190
> > <https://reviews.apache.org/r/41273/diff/6/?file=1162113#file1162113line188>
> >
> > Just found this bug and thought you should fix it as part of your review.
> >
> > We should _only_ being doing the symlinking if coming from < 2.3 to >= 2.3
> >
> > Consider this case. You install 2.3.0.0 and all links are correct. /etc/<component>/conf is correctly a pointer to /usr/hdp/current/<component>/conf
> >
> > But then you distribute 2.3.4.0 in preparation for an upgrade. This code will unset the above link and put it back to /etc/<component>/conf.backup
> >
> > Basically we need to add 1 more if-statement here that does this:
> >
> > if existing-current-version >= HDP 2.3:
> > Logger.info("Configuration symlinks are already correct on {0}".format(stack_version))
> > return
Actually, this is already handled in install_packges.py
# Version installing must be 2.3.0.0 or higher
if version.compare_versions(version.format_hdp_stack_version(args[1]), "2.3.0.0") < 0:
Logger.info("Configuration symlinks are not needed for {0}, only HDP-2.3+".format(stack_version))
return
# Current HDP stack must be less than 2.3
# if already on HDP 2.3, then there's nothing to do in terms of linking configs
if self.current_hdp_stack_version and compare_versions(self.current_hdp_stack_version, '2.3') >= 0:
Logger.info("The current cluster stack of {0} does not require linking configurations".format(stack_version))
return
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110286
-----------------------------------------------------------
On Dec. 14, 2015, 7:17 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2015, 7:17 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14307
> https://issues.apache.org/jira/browse/AMBARI-14307
>
>
> Repository: ambari
>
>
> Description
> -------
>
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
>
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
> BeforeStartHook().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
> method(env)
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
> create_topology_script_and_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
> create_topology_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
> group=params.user_group)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
> raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
>
>
> This failed because /etc/hadoop/conf is now a circular symlink
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7fd64b5
> ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py 186b847
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py d2ffb3c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9e4560c
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 5fd5ef6
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 72cb050
>
> Diff: https://reviews.apache.org/r/41273/diff/
>
>
> Testing
> -------
>
> Tested on the following code paths,
>
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
>
>
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:853
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110286
-----------------------------------------------------------
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py (lines 188 - 190)
<https://reviews.apache.org/r/41273/#comment170100>
Just found this bug and thought you should fix it as part of your review.
We should _only_ being doing the symlinking if coming from < 2.3 to >= 2.3
Consider this case. You install 2.3.0.0 and all links are correct. /etc/<component>/conf is correctly a pointer to /usr/hdp/current/<component>/conf
But then you distribute 2.3.4.0 in preparation for an upgrade. This code will unset the above link and put it back to /etc/<component>/conf.backup
Basically we need to add 1 more if-statement here that does this:
if existing-current-version >= HDP 2.3:
Logger.info("Configuration symlinks are already correct on {0}".format(stack_version))
return
- Jonathan Hurley
On Dec. 14, 2015, 2:17 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2015, 2:17 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14307
> https://issues.apache.org/jira/browse/AMBARI-14307
>
>
> Repository: ambari
>
>
> Description
> -------
>
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
>
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
> BeforeStartHook().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
> method(env)
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
> create_topology_script_and_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
> create_topology_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
> group=params.user_group)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
> raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
>
>
> This failed because /etc/hadoop/conf is now a circular symlink
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7fd64b5
> ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py 186b847
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py d2ffb3c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9e4560c
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 5fd5ef6
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 72cb050
>
> Diff: https://reviews.apache.org/r/41273/diff/
>
>
> Testing
> -------
>
> Tested on the following code paths,
>
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
>
>
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:853
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110248
-----------------------------------------------------------
Ship it!
Jonathan's comment is important, mine are cosmetic.
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py (line 364)
<https://reviews.apache.org/r/41273/#comment170053>
Remove 'pass' here
ambari-common/src/main/python/resource_management/libraries/script/script.py (line 306)
<https://reviews.apache.org/r/41273/#comment170049>
Could probably format this a little for readability. Use default()?
- Nate Cole
On Dec. 14, 2015, 2:17 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2015, 2:17 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14307
> https://issues.apache.org/jira/browse/AMBARI-14307
>
>
> Repository: ambari
>
>
> Description
> -------
>
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
>
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
> BeforeStartHook().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
> method(env)
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
> create_topology_script_and_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
> create_topology_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
> group=params.user_group)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
> raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
>
>
> This failed because /etc/hadoop/conf is now a circular symlink
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7fd64b5
> ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py 186b847
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py d2ffb3c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9e4560c
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 5fd5ef6
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 72cb050
>
> Diff: https://reviews.apache.org/r/41273/diff/
>
>
> Testing
> -------
>
> Tested on the following code paths,
>
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
>
>
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:853
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110217
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py (lines 302 - 306)
<https://reviews.apache.org/r/41273/#comment170010>
If the stack is 2.3, then the first if-statement will be true - the 2nd if-statement will never be entered.
- Jonathan Hurley
On Dec. 14, 2015, 2:17 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2015, 2:17 a.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14307
> https://issues.apache.org/jira/browse/AMBARI-14307
>
>
> Repository: ambari
>
>
> Description
> -------
>
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
>
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
> BeforeStartHook().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
> method(env)
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
> create_topology_script_and_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
> create_topology_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
> group=params.user_group)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
> raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
>
>
> This failed because /etc/hadoop/conf is now a circular symlink
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7fd64b5
> ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py 186b847
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py d2ffb3c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9e4560c
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 5fd5ef6
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 72cb050
>
> Diff: https://reviews.apache.org/r/41273/diff/
>
>
> Testing
> -------
>
> Tested on the following code paths,
>
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
>
>
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:853
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Dec. 14, 2015, 10:03 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py, lines 192-195
> > <https://reviews.apache.org/r/41273/diff/6-7/?file=1162113#file1162113line192>
> >
> > +1! Glad this is back. It's needed.
That breaks AMBARI-14293 back.
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110298
-----------------------------------------------------------
On Dec. 14, 2015, 10 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2015, 10 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14307
> https://issues.apache.org/jira/browse/AMBARI-14307
>
>
> Repository: ambari
>
>
> Description
> -------
>
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
>
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
> BeforeStartHook().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
> method(env)
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
> create_topology_script_and_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
> create_topology_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
> group=params.user_group)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
> raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
>
>
> This failed because /etc/hadoop/conf is now a circular symlink
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py c9c70de
> ambari-common/src/main/python/resource_management/libraries/script/script.py 9dc402d
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 470661a
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py 885d650
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py 2c39614
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 3260715
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 75aa4a1
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 7edd867
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 48d55b1
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 490b3bf
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 1c782bd
>
> Diff: https://reviews.apache.org/r/41273/diff/
>
>
> Testing
> -------
>
> Tested on the following code paths,
>
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
>
>
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:853
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110298
-----------------------------------------------------------
Ship it!
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py (lines 192 - 195)
<https://reviews.apache.org/r/41273/#comment170114>
+1! Glad this is back. It's needed.
- Jonathan Hurley
On Dec. 14, 2015, 3 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2015, 3 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-14307
> https://issues.apache.org/jira/browse/AMBARI-14307
>
>
> Repository: ambari
>
>
> Description
> -------
>
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
>
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
> BeforeStartHook().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
> method(env)
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
> create_topology_script_and_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
> create_topology_mapping()
> File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
> group=params.user_group)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
> raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
>
>
> This failed because /etc/hadoop/conf is now a circular symlink
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py c9c70de
> ambari-common/src/main/python/resource_management/libraries/script/script.py 9dc402d
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
> ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 470661a
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py 885d650
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py 2c39614
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 3260715
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 75aa4a1
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 7edd867
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 48d55b1
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 490b3bf
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 1c782bd
>
> Diff: https://reviews.apache.org/r/41273/diff/
>
>
> Testing
> -------
>
> Tested on the following code paths,
>
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
>
>
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:853
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/
-----------------------------------------------------------
(Updated Dec. 14, 2015, 8 p.m.)
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
Bugs: AMBARI-14307
https://issues.apache.org/jira/browse/AMBARI-14307
Repository: ambari
Description
-------
* Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
* HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
* RU from to 2.3.4.0 and reach the Finalize screen.
* Then downgrade,
Kafka broker restart failed on downgrade
```
Traceback (most recent call last):
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
BeforeStartHook().execute()
File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
method(env)
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
create_topology_script_and_mapping()
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
create_topology_mapping()
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
group=params.user_group)
File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
self.env.run()
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
self.run_action(resource, action)
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
provider_action()
File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
```
This failed because /etc/hadoop/conf is now a circular symlink
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py c9c70de
ambari-common/src/main/python/resource_management/libraries/script/script.py 9dc402d
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 470661a
ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py 885d650
ambari-server/src/test/python/custom_actions/test_ru_set_all.py 2c39614
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 3260715
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 75aa4a1
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 7edd867
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 48d55b1
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 490b3bf
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 1c782bd
Diff: https://reviews.apache.org/r/41273/diff/
Testing
-------
Tested on the following code paths,
1. Fresh install of HDP 2.1
2. Installed bits for HDP 2.3 and verified the conf folder structure
3. EU to HDP 2.3
4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
1. Fresh install of HDP 2.2
2. Installed bits for HDP 2.3 and verfied the conf folder structure
3. RU to HDP 2.3 and downgrade
Python unit tests passed,
----------------------------------------------------------------------
Total run:853
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
Re: Review Request 41273: AMBARI-14307. RU: Kafka broker restart
failed on
downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/
-----------------------------------------------------------
(Updated Dec. 14, 2015, 7:17 a.m.)
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
Bugs: AMBARI-14307
https://issues.apache.org/jira/browse/AMBARI-14307
Repository: ambari
Description
-------
* Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
* HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
* RU from to 2.3.4.0 and reach the Finalize screen.
* Then downgrade,
Kafka broker restart failed on downgrade
```
Traceback (most recent call last):
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 39, in <module>
BeforeStartHook().execute()
File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 217, in execute
method(env)
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\", line 36, in hook
create_topology_script_and_mapping()
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 69, in create_topology_script_and_mapping
create_topology_mapping()
File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\", line 36, in create_topology_mapping
group=params.user_group)
File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
self.env.run()
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
self.run_action(resource, action)
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
provider_action()
File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 144, in action_create
raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\" % (self.resource, path))
resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed, looped symbolic links found while resolving /etc/hadoop/conf
Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py', 'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START', '/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
```
This failed because /etc/hadoop/conf is now a circular symlink
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 7fd64b5
ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 6fcf08a
ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25
ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db
ambari-server/src/test/python/custom_actions/test_ru_set_all.py 186b847
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py d2ffb3c
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9e4560c
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py 5fd5ef6
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 72cb050
Diff: https://reviews.apache.org/r/41273/diff/
Testing
-------
Tested on the following code paths,
1. Fresh install of HDP 2.1
2. Installed bits for HDP 2.3 and verified the conf folder structure
3. EU to HDP 2.3
4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked it to /usr/hdp/current/hadoop-client/conf
1. Fresh install of HDP 2.2
2. Installed bits for HDP 2.3 and verfied the conf folder structure
3. RU to HDP 2.3 and downgrade
Python unit tests passed,
----------------------------------------------------------------------
Total run:853
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez