You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmitro Lisnichenko <dl...@hortonworks.com> on 2015/09/09 16:49:38 UTC
Review Request 38220: [Preview] Stop-and-Start Upgrade: apply configs
after all services have been stopped
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description
-------
Goal:
for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Does such form look good?
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 01022b8
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Preview
Thanks,
Dmitro Lisnichenko
Re: Review Request 38220: [Preview] Stop-and-Start Upgrade: apply
configs after all services have been stopped
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review98228
-----------------------------------------------------------
This looks good. I am only a couple of hours away from submitting an upgrade pack for HDP 2.1->2.3, so if you don't mind, can you wait until my patch is in so you can repeat this config change in that upgrade pack?
- Alejandro Fernandez
On Sept. 9, 2015, 2:52 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2015, 2:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Goal:
> for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
>
> I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 01022b8
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Preview
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: [Preview] Stop-and-Start Upgrade: apply
configs after all services have been stopped
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Sept. 11, 2015, 2:51 p.m., Jonathan Hurley wrote:
> > Why is this change needed for stop-the-world? You've already stopped the hive server and prevented clients from draining. You should not need to adjust the ports like we do for normal rolling upgrades.
>
> Alejandro Fernandez wrote:
> You're right. Stop-the-World Upgrade doesn't need to temporarily change the hive port.
Ok, I'll not include that into the final patch. But I still need some place to debug application of config changes during the SWU. As of now, 2.2->2.2+ SWU is the ony upgrade available, and that is the only config change for this patch.
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review98609
-----------------------------------------------------------
On Sept. 9, 2015, 2:52 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2015, 2:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Goal:
> for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
>
> I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 01022b8
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Preview
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: [Preview] Stop-and-Start Upgrade: apply
configs after all services have been stopped
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Sept. 11, 2015, 2:51 p.m., Jonathan Hurley wrote:
> > Why is this change needed for stop-the-world? You've already stopped the hive server and prevented clients from draining. You should not need to adjust the ports like we do for normal rolling upgrades.
You're right. Stop-the-World Upgrade doesn't need to temporarily change the hive port.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review98609
-----------------------------------------------------------
On Sept. 9, 2015, 2:52 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2015, 2:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Goal:
> for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
>
> I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 01022b8
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Preview
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: [Preview] Stop-and-Start Upgrade: apply
configs after all services have been stopped
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review98609
-----------------------------------------------------------
Why is this change needed for stop-the-world? You've already stopped the hive server and prevented clients from draining. You should not need to adjust the ports like we do for normal rolling upgrades.
- Jonathan Hurley
On Sept. 9, 2015, 10:52 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2015, 10:52 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Goal:
> for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
>
> I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 01022b8
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Preview
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99253
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Sept. 16, 2015, 1:49 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 1:49 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 16, 2015, 5:49 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Addressed comments
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description
-------
When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
The same should apply once 2.2 -> 2.4 is available.
This is meant to handle the following cases,
HDP 2.2 config changes:
stack already defines property a
add property b
drop property c
rename c to d
HDP 2.3 config changes:
add property e
drop property a
rename d to e
So the overall effect is,
drop property a
add b, and e
rename c to e
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
Results :
Tests in error:
UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Views ...................................... SUCCESS [3.193s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
[INFO] Ambari Server ..................................... FAILURE [51:24.274s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:29.595s
[INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
[INFO] Final Memory: 69M/1708M
Thanks,
Dmitro Lisnichenko
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Sept. 16, 2015, 5:39 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java, line 587
> > <https://reviews.apache.org/r/38220/diff/2/?file=1075609#file1075609line587>
> >
> > Any reason this was renamed from configUpgradePack? That was more human-readable than confUpgrPack
Names seemed too long to me, renamed back
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99247
-----------------------------------------------------------
On Sept. 16, 2015, 3:42 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 3:42 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Sept. 16, 2015, 5:39 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml, lines 26-27
> > <https://reviews.apache.org/r/38220/diff/2/?file=1075612#file1075612line26>
> >
> > Spaces at the end of the string? Also, should this be 2.2.0 or 2.2? If it represents an intermediate stack to upgrade from, I'd think it would be 2.2 and 2.3 and not 2.2.0 and 2.3.0.
Good catch! Looks like a last-minute edit when I moved this fragment from another file.
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99247
-----------------------------------------------------------
On Sept. 16, 2015, 3:42 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 3:42 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99247
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java (line 587)
<https://reviews.apache.org/r/38220/#comment156173>
Any reason this was renamed from configUpgradePack? That was more human-readable than confUpgrPack
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml (lines 26 - 27)
<https://reviews.apache.org/r/38220/#comment156174>
Spaces at the end of the string? Also, should this be 2.2.0 or 2.2? If it represents an intermediate stack to upgrade from, I'd think it would be 2.2 and 2.3 and not 2.2.0 and 2.3.0.
- Jonathan Hurley
On Sept. 16, 2015, 11:42 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 11:42 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 16, 2015, 3:42 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description (updated)
-------
When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
The same should apply once 2.2 -> 2.4 is available.
This is meant to handle the following cases,
HDP 2.2 config changes:
stack already defines property a
add property b
drop property c
rename c to d
HDP 2.3 config changes:
add property e
drop property a
rename d to e
So the overall effect is,
drop property a
add b, and e
rename c to e
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
Results :
Tests in error:
UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Views ...................................... SUCCESS [3.193s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
[INFO] Ambari Server ..................................... FAILURE [51:24.274s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:29.595s
[INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
[INFO] Final Memory: 69M/1708M
Thanks,
Dmitro Lisnichenko
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 16, 2015, 3:41 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Summary (updated)
-----------------
Stop-and-Start Upgrade: Apply configs from multiple major stack versions.
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description
-------
Goal:
for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
Results :
Tests in error:
UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Views ...................................... SUCCESS [3.193s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
[INFO] Ambari Server ..................................... FAILURE [51:24.274s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:29.595s
[INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
[INFO] Final Memory: 69M/1708M
Thanks,
Dmitro Lisnichenko
Re: Review Request 38220: Stop-and-Start Upgrade: apply configs after
all services have been stopped
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99228
-----------------------------------------------------------
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java has been deleted, because it should be deleted as a part of previous patch by Dmitry G
- Dmitro Lisnichenko
On Sept. 16, 2015, 3:40 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 3:40 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Goal:
> for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
>
> I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: apply configs after
all services have been stopped
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 16, 2015, 3:40 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Published a patch that should be good to go in
Summary (updated)
-----------------
Stop-and-Start Upgrade: apply configs after all services have been stopped
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description
-------
Goal:
for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/38220/diff/
Testing (updated)
-------
Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
Results :
Tests in error:
UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Views ...................................... SUCCESS [3.193s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
[INFO] Ambari Server ..................................... FAILURE [51:24.274s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:29.595s
[INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
[INFO] Final Memory: 69M/1708M
Thanks,
Dmitro Lisnichenko
Re: Review Request 38220: [Preview] Stop-and-Start Upgrade: apply
configs after all services have been stopped
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 9, 2015, 2:52 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description (updated)
-------
Goal:
for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 01022b8
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Preview
Thanks,
Dmitro Lisnichenko