You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/05/13 01:52:52 UTC
Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
Bugs: AMBARI-11086
https://issues.apache.org/jira/browse/AMBARI-11086
Repository: ambari
Description
-------
There are two major changes in this patch:
- The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
- hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
ambari-server/src/test/python/TestUtils.py 8370986
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
Diff: https://reviews.apache.org/r/34132/diff/
Testing
-------
Upgrade performed, new tests, mvn clean test
Thanks,
Jonathan Hurley
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On May 12, 2015, 8:14 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py, line 31
> > <https://reviews.apache.org/r/34132/diff/1/?file=957138#file957138line31>
> >
> > Finally, it's all in one place! :-)
> >
> > Do the clients need to be on here?
No, they do not. This is only for running services, not clients. Clients won't have this issue.
> On May 12, 2015, 8:14 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py, line 192
> > <https://reviews.apache.org/r/34132/diff/1/?file=957138#file957138line192>
> >
> >
What's the issue here?
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83528
-----------------------------------------------------------
On May 12, 2015, 7:52 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 12, 2015, 7:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83528
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
<https://reviews.apache.org/r/34132/#comment134526>
Finally, it's all in one place! :-)
Do the clients need to be on here?
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
<https://reviews.apache.org/r/34132/#comment134528>
- Alejandro Fernandez
On May 12, 2015, 11:52 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 12, 2015, 11:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On May 13, 2015, 12:21 a.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py, line 83
> > <https://reviews.apache.org/r/34132/diff/1/?file=957138#file957138line83>
> >
> > This function name is misleading
>
> Jonathan Hurley wrote:
> I can change this in conf-select.
Cool, fix it and ship it!
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83530
-----------------------------------------------------------
On May 13, 2015, 12:32 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 12:32 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On May 12, 2015, 8:21 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py, line 83
> > <https://reviews.apache.org/r/34132/diff/1/?file=957138#file957138line83>
> >
> > This function name is misleading
I can change this in conf-select.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83530
-----------------------------------------------------------
On May 12, 2015, 7:52 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 12, 2015, 7:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83530
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
<https://reviews.apache.org/r/34132/#comment134530>
This function name is misleading
- Alejandro Fernandez
On May 12, 2015, 11:52 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 12, 2015, 11:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83536
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On May 13, 2015, 12:32 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 12:32 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83599
-----------------------------------------------------------
Ship it!
Ship It!
- Tom Beerbower
On May 13, 2015, 12:32 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 13, 2015, 12:32 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/#review83603
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On May 12, 2015, 8:32 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34132/
> -----------------------------------------------------------
>
> (Updated May 12, 2015, 8:32 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-11086
> https://issues.apache.org/jira/browse/AMBARI-11086
>
>
> Repository: ambari
>
>
> Description
> -------
>
> There are two major changes in this patch:
>
> - The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
>
> - hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
> ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
> ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
> ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
> ambari-server/src/test/python/TestUtils.py 8370986
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
> ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34132/diff/
>
>
> Testing
> -------
>
> Upgrade performed, new tests, mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/
-----------------------------------------------------------
(Updated May 12, 2015, 8:32 p.m.)
Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
Bugs: AMBARI-11086
https://issues.apache.org/jira/browse/AMBARI-11086
Repository: ambari
Description
-------
There are two major changes in this patch:
- The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
- hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
ambari-server/src/test/python/TestUtils.py 8370986
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
Diff: https://reviews.apache.org/r/34132/diff/
Testing
-------
Upgrade performed, new tests, mvn clean test
Thanks,
Jonathan Hurley
Re: Review Request 34132: Upgrade Pack Configure Task Must Preserve
Additions When Deleting
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34132/
-----------------------------------------------------------
(Updated May 12, 2015, 8:31 p.m.)
Review request for Ambari, Alejandro Fernandez, Nate Cole, and Tom Beerbower.
Changes
-------
Some better naming and doc.
Bugs: AMBARI-11086
https://issues.apache.org/jira/browse/AMBARI-11086
Repository: ambari
Description
-------
There are two major changes in this patch:
- The configure task should preserve additions, not just conflicts, when DELETE * is used with `preserve-edits`
- hdp-select has been turned into a module so that we can have custom functionality when its invoked. When invoked during an upgrade, if the hdp-select version of the component being upgraded is still on the old version, some directories need to be recalculated. However, python doesn't re-import modules like params/params_linux/status_params. As a result, directories that were calculated as part of the restart were never changed, causing upgrade problems. We are now called reload(...) on these modules on an hdp-select to force the recalculation.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py 63b474f
ambari-common/src/main/python/resource_management/libraries/functions/format.py 0dfc45d
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/checks/ClientRetryPropertyCheck.java 5fbb5e4
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java 5c17408
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigMergeHelper.java c0fc596
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py b6ac9b9
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py 722f437
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py 8845a82
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py a331755
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py e4d4653
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py ad94b3b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py 0a67ef2
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py 0158644
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9e6079a
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py 2514ccf
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py c1d145c
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py a33857c
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py 44df132
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py ecdb798
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py bc254c5
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py b72fc46
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dddebc9
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py 4b6cb88
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py 43a0787
ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py eb83e98
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 8c05a8e
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py cc977d1
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py ca26d10
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/drpc_server.py 74cfc59
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus.py f6af281
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/nimbus_prod.py 310fec9
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/rest_api.py 6768dd4
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor.py 36e3ca8
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/supervisor_prod.py 7ce4764
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/ui_server.py 4ab37c5
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py 7394e68
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 9cd555d
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py e625a0d
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 38775b6
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py f756c36
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py df41938
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py 89e3a50
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper.py f1cd7fd
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_client.py 179c978
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py 2e19100
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java d9638bc
ambari-server/src/test/python/TestUtils.py 8370986
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 8978b18
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4a6c5ab
ambari-server/src/test/resources/stacks/HDP/2.0.5/services/ZOOKEEPER/configuration/zoo.cfg.xml PRE-CREATION
Diff: https://reviews.apache.org/r/34132/diff/
Testing
-------
Upgrade performed, new tests, mvn clean test
Thanks,
Jonathan Hurley