You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2016/02/11 22:19:37 UTC
Review Request 43501: RU/EU: Upgrading Oozie database fails since new
configs are not yet written to /usr/hdp/current/oozie-server/conf
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/
-----------------------------------------------------------
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
Bugs: AMBARI-15004
https://issues.apache.org/jira/browse/AMBARI-15004
Repository: ambari
Description
-------
Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
1. Stop all Oozie Servers
2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
The step upgrade_oozie_database_and_sharelib() invokes
/usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
Diff: https://reviews.apache.org/r/43501/diff/
Testing
-------
Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
Thanks,
Alejandro Fernandez
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/
-----------------------------------------------------------
(Updated Feb. 12, 2016, 1:08 a.m.)
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
Bugs: AMBARI-15004
https://issues.apache.org/jira/browse/AMBARI-15004
Repository: ambari
Description
-------
Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
1. Stop all Oozie Servers
2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
The step upgrade_oozie_database_and_sharelib() invokes
/usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
Diff: https://reviews.apache.org/r/43501/diff/
Testing (updated)
-------
Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
Unit tests passed,
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:15:28.721s
[INFO] Finished at: Thu Feb 11 16:45:42 PST 2016
[INFO] Final Memory: 138M/4042M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Feb. 11, 2016, 10:38 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml, line 806
> > <https://reviews.apache.org/r/43501/diff/1/?file=1240402#file1240402line806>
> >
> > Missing hosts=first and comment in this upgrade pack
> >
> > <!-- It is extremely important that both of these tasks run on the exact same host. Hence, pick the first alphabetically. -->
> > <task xsi:type="configure_function" hosts="first" />
Actually, not needed in HDP 2.1 to 2.3 since it calls configure on all hosts because we remove the HDP 2.1 bits
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/#review118943
-----------------------------------------------------------
On Feb. 11, 2016, 9:19 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43501/
> -----------------------------------------------------------
>
> (Updated Feb. 11, 2016, 9:19 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15004
> https://issues.apache.org/jira/browse/AMBARI-15004
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
> Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
>
> 1. Stop all Oozie Servers
> 2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
> The step upgrade_oozie_database_and_sharelib() invokes
>
> /usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
>
> However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
>
> I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
>
> Diff: https://reviews.apache.org/r/43501/diff/
>
>
> Testing
> -------
>
> Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/#review118943
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml (line 806)
<https://reviews.apache.org/r/43501/#comment180251>
Missing hosts=first and comment in this upgrade pack
<!-- It is extremely important that both of these tasks run on the exact same host. Hence, pick the first alphabetically. -->
<task xsi:type="configure_function" hosts="first" />
- Jayush Luniya
On Feb. 11, 2016, 9:19 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43501/
> -----------------------------------------------------------
>
> (Updated Feb. 11, 2016, 9:19 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15004
> https://issues.apache.org/jira/browse/AMBARI-15004
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
> Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
>
> 1. Stop all Oozie Servers
> 2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
> The step upgrade_oozie_database_and_sharelib() invokes
>
> /usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
>
> However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
>
> I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
>
> Diff: https://reviews.apache.org/r/43501/diff/
>
>
> Testing
> -------
>
> Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/#review118944
-----------------------------------------------------------
Ship it!
Fix ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml and ship
- Jayush Luniya
On Feb. 11, 2016, 9:19 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43501/
> -----------------------------------------------------------
>
> (Updated Feb. 11, 2016, 9:19 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15004
> https://issues.apache.org/jira/browse/AMBARI-15004
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
> Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
>
> 1. Stop all Oozie Servers
> 2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
> The step upgrade_oozie_database_and_sharelib() invokes
>
> /usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
>
> However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
>
> I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
>
> Diff: https://reviews.apache.org/r/43501/diff/
>
>
> Testing
> -------
>
> Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/#review118927
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml (line 808)
<https://reviews.apache.org/r/43501/#comment180238>
No hosts="first"
- Nate Cole
On Feb. 11, 2016, 4:19 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43501/
> -----------------------------------------------------------
>
> (Updated Feb. 11, 2016, 4:19 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15004
> https://issues.apache.org/jira/browse/AMBARI-15004
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
> Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
>
> 1. Stop all Oozie Servers
> 2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
> The step upgrade_oozie_database_and_sharelib() invokes
>
> /usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
>
> However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
>
> I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
>
> Diff: https://reviews.apache.org/r/43501/diff/
>
>
> Testing
> -------
>
> Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/#review118928
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Feb. 11, 2016, 4:19 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43501/
> -----------------------------------------------------------
>
> (Updated Feb. 11, 2016, 4:19 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15004
> https://issues.apache.org/jira/browse/AMBARI-15004
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
> Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
>
> 1. Stop all Oozie Servers
> 2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
> The step upgrade_oozie_database_and_sharelib() invokes
>
> /usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
>
> However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
>
> I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
>
> Diff: https://reviews.apache.org/r/43501/diff/
>
>
> Testing
> -------
>
> Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/#review118925
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java (lines 236 - 240)
<https://reviews.apache.org/r/43501/#comment180236>
Why FIRST and not ANY? There's no saying that the alpha-sorted set of hosts that the first one is any better than the second one. The upgrade pack changes imply that you're picking to run on a single host before doing any others.
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java (lines 48 - 49)
<https://reviews.apache.org/r/43501/#comment180237>
See comment about FIRST vs ANY. The code doesn't care which one is more appropriate.
- Nate Cole
On Feb. 11, 2016, 4:19 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43501/
> -----------------------------------------------------------
>
> (Updated Feb. 11, 2016, 4:19 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15004
> https://issues.apache.org/jira/browse/AMBARI-15004
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
> Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
>
> 1. Stop all Oozie Servers
> 2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
> The step upgrade_oozie_database_and_sharelib() invokes
>
> /usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
>
> However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
>
> I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
>
> Diff: https://reviews.apache.org/r/43501/diff/
>
>
> Testing
> -------
>
> Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 43501: RU/EU: Upgrading Oozie database fails since
new configs are not yet written to /usr/hdp/current/oozie-server/conf
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43501/#review118921
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml (line 767)
<https://reviews.apache.org/r/43501/#comment180234>
The problem was that the function oozie_server_upgrade failed if hdp-select, conf-select, and the new configs had not yet been generated.
However, the configure() function doesn't work correctly if called from oozie_server_upgrade.py due to the different home directory and because of the way our Template() resource works.
For this reason, I had to reuse the "configure_function" task and make sure it runs on the same host that is used to upgrade the DB.
- Alejandro Fernandez
On Feb. 11, 2016, 9:19 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43501/
> -----------------------------------------------------------
>
> (Updated Feb. 11, 2016, 9:19 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush Luniya, and Nate Cole.
>
>
> Bugs: AMBARI-15004
> https://issues.apache.org/jira/browse/AMBARI-15004
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Prepare Oozie Server fails at upgrade_oozie_database_and_sharelib() step.
> Oozie is a special case. In PRE-UPGRADE phase we perform the following steps
>
> 1. Stop all Oozie Servers
> 2. On one Oozie Server execute upgrade_oozie_database_and_sharelib step to create the Oozie DB and Sharelib.
> The step upgrade_oozie_database_and_sharelib() invokes
>
> /usr/hdp/{new_version}/oozie/bin/ooziedb.sh upgrade -run
>
> However, we still haven't executed conf-select, hdp-select, and generated configs, which happens in UPGRADE phase at pre_upgrade_restart() step leading to upgrade db step to fail.
>
> I tried modifying the upgrade_oozie_database_and_sharelib() function to generate the configs, but it is a custom command that is called and cannot correctly import the templates since it first calls ru
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5e21da5
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java b36dca4
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskWrapperBuilder.java 81a3a4d
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py dc00b13
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 4d68f03
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py 66ea832
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 084a0c5
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml f12aa18
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml 67c2d3d
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml d31914f
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 9fbe235
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 750b8b4
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml c9fabb2
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 19e9601
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml 3500eea
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 77dcfae
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 32e3764
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml 3863877
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml e45e851
>
> Diff: https://reviews.apache.org/r/43501/diff/
>
>
> Testing
> -------
>
> Verified during RU and EU from HDP 2.2 to 2.3 with Oozie, including downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>