You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmitro Lisnichenko <dl...@hortonworks.com> on 2015/09/16 17:41:53 UTC
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 16, 2015, 3:41 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Summary (updated)
-----------------
Stop-and-Start Upgrade: Apply configs from multiple major stack versions.
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description
-------
Goal:
for STW Upgrade, we'll need to apply configs after all services have been stopped, and before calling hdp-select set all.
I think, we have to add separate groups before and after restart of HIVE server to enforce config application order during UPGRADE and DONGRADE. Looks like that is the only config change required for 2.2->2.2+ upgrade. Does such upgrade pack definition look good?
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
Results :
Tests in error:
UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Views ...................................... SUCCESS [3.193s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
[INFO] Ambari Server ..................................... FAILURE [51:24.274s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:29.595s
[INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
[INFO] Final Memory: 69M/1708M
Thanks,
Dmitro Lisnichenko
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99253
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Sept. 16, 2015, 1:49 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 1:49 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 16, 2015, 5:49 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Addressed comments
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description
-------
When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
The same should apply once 2.2 -> 2.4 is available.
This is meant to handle the following cases,
HDP 2.2 config changes:
stack already defines property a
add property b
drop property c
rename c to d
HDP 2.3 config changes:
add property e
drop property a
rename d to e
So the overall effect is,
drop property a
add b, and e
rename c to e
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
Results :
Tests in error:
UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Views ...................................... SUCCESS [3.193s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
[INFO] Ambari Server ..................................... FAILURE [51:24.274s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:29.595s
[INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
[INFO] Final Memory: 69M/1708M
Thanks,
Dmitro Lisnichenko
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Sept. 16, 2015, 5:39 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java, line 587
> > <https://reviews.apache.org/r/38220/diff/2/?file=1075609#file1075609line587>
> >
> > Any reason this was renamed from configUpgradePack? That was more human-readable than confUpgrPack
Names seemed too long to me, renamed back
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99247
-----------------------------------------------------------
On Sept. 16, 2015, 3:42 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 3:42 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Sept. 16, 2015, 5:39 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml, lines 26-27
> > <https://reviews.apache.org/r/38220/diff/2/?file=1075612#file1075612line26>
> >
> > Spaces at the end of the string? Also, should this be 2.2.0 or 2.2? If it represents an intermediate stack to upgrade from, I'd think it would be 2.2 and 2.3 and not 2.2.0 and 2.3.0.
Good catch! Looks like a last-minute edit when I moved this fragment from another file.
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99247
-----------------------------------------------------------
On Sept. 16, 2015, 3:42 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 3:42 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/#review99247
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java (line 587)
<https://reviews.apache.org/r/38220/#comment156173>
Any reason this was renamed from configUpgradePack? That was more human-readable than confUpgrPack
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml (lines 26 - 27)
<https://reviews.apache.org/r/38220/#comment156174>
Spaces at the end of the string? Also, should this be 2.2.0 or 2.2? If it represents an intermediate stack to upgrade from, I'd think it would be 2.2 and 2.3 and not 2.2.0 and 2.3.0.
- Jonathan Hurley
On Sept. 16, 2015, 11:42 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38220/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2015, 11:42 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-13048
> https://issues.apache.org/jira/browse/AMBARI-13048
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
> The same should apply once 2.2 -> 2.4 is available.
> This is meant to handle the following cases,
> HDP 2.2 config changes:
> stack already defines property a
> add property b
> drop property c
> rename c to d
> HDP 2.3 config changes:
> add property e
> drop property a
> rename d to e
> So the overall effect is,
> drop property a
> add b, and e
> rename c to e
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
> ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
> ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38220/diff/
>
>
> Testing
> -------
>
> Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
>
> Results :
>
> Tests in error:
> UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
>
> Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Views ...................................... SUCCESS [3.193s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
> [INFO] Ambari Server ..................................... FAILURE [51:24.274s]
> [INFO] Ambari Agent ...................................... SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 51:29.595s
> [INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
> [INFO] Final Memory: 69M/1708M
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 38220: Stop-and-Start Upgrade: Apply configs from
multiple major stack versions.
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38220/
-----------------------------------------------------------
(Updated Sept. 16, 2015, 3:42 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-13048
https://issues.apache.org/jira/browse/AMBARI-13048
Repository: ambari
Description (updated)
-------
When performing a Stop-and-Start Upgrade from HDP 2.1 -> 2.3, we need to apply configs for multiple stack versions (2.2 and 2.3).
The same should apply once 2.2 -> 2.4 is available.
This is meant to handle the following cases,
HDP 2.2 config changes:
stack already defines property a
add property b
drop property c
rename c to d
HDP 2.3 config changes:
add property e
drop property a
rename d to e
So the overall effect is,
drop property a
add b, and e
rename c to e
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 8f4870c
ambari-server/src/main/java/org/apache/ambari/server/state/stack/ConfigUpgradePack.java 2896255
ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java 6ae0a15
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 743204c0
ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckStackVersionTest.java 8d8b08f
ambari-server/src/test/java/org/apache/ambari/server/state/stack/ConfigUpgradePackTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/38220/diff/
Testing
-------
Failed test mentioned below fails also on latest branch-dev-stop-all-upgrade commits
Results :
Tests in error:
UpgradeResourceProviderHDP22Test.testCreateIntraStackUpgrade:266 » System An i...
Tests run: 3138, Failures: 0, Errors: 1, Skipped: 27
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Views ...................................... SUCCESS [3.193s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.556s]
[INFO] Ambari Server ..................................... FAILURE [51:24.274s]
[INFO] Ambari Agent ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:29.595s
[INFO] Finished at: Wed Sep 16 18:32:55 EEST 2015
[INFO] Final Memory: 69M/1708M
Thanks,
Dmitro Lisnichenko