You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2017/08/02 00:32:40 UTC
Review Request 61317: Configs For Target Stack Are Not Created During
Upgrade
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61317/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
Bugs: AMBARI-21626
https://issues.apache.org/jira/browse/AMBARI-21626
Repository: ambari
Description
-------
During an upgrade from HDP 2.5 to HDP 2.6, the new configurations are not created in HDP 2.6 - they're created in HDP 2.5 which causes services to fail on restart.
STR:
- Create an HDP 2.5 cluster with ZK + HDFS
- Do an EU to HDP 2.6
The EU will fail right after the "move stack version" step.
It looks like the bug is in {{ConfigHelper}} where it ignores the passed-in {{StackId}} and uses the Cluster's stack.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java ff0becc822
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 16fc971e27
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java a5b40ff256
Diff: https://reviews.apache.org/r/61317/diff/1/
Testing
-------
Full upgrade/downgrade
UTs PENDING...
Thanks,
Jonathan Hurley
Re: Review Request 61317: Configs For Target Stack Are Not Created
During Upgrade
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61317/#review182035
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On Aug. 2, 2017, 12:32 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61317/
> -----------------------------------------------------------
>
> (Updated Aug. 2, 2017, 12:32 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-21626
> https://issues.apache.org/jira/browse/AMBARI-21626
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an upgrade from HDP 2.5 to HDP 2.6, the new configurations are not created in HDP 2.6 - they're created in HDP 2.5 which causes services to fail on restart.
>
> STR:
> - Create an HDP 2.5 cluster with ZK + HDFS
> - Do an EU to HDP 2.6
>
> The EU will fail right after the "move stack version" step.
>
> It looks like the bug is in {{ConfigHelper}} where it ignores the passed-in {{StackId}} and uses the Cluster's stack.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java ff0becc822
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b4f7120836
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 4b2e0f1ace
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java a5b40ff256
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 06b6217051
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java 5ac7752ddf
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 9b85242ac8
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java 3a29298161
> ambari-server/src/test/resources/stacks/HDP/2.2.0/upgrades/upgrade_test_checks.xml 4d4d972685
>
>
> Diff: https://reviews.apache.org/r/61317/diff/3/
>
>
> Testing
> -------
>
> Full upgrade/downgrade
>
> Tests run: 4803, Failures: 0, Errors: 0, Skipped: 36
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24:33 min
> [INFO] Finished at: 2017-08-02T09:18:35-04:00
> [INFO] Final Memory: 71M/959M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 61317: Configs For Target Stack Are Not Created
During Upgrade
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61317/#review181980
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On Aug. 2, 2017, 3:32 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61317/
> -----------------------------------------------------------
>
> (Updated Aug. 2, 2017, 3:32 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-21626
> https://issues.apache.org/jira/browse/AMBARI-21626
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an upgrade from HDP 2.5 to HDP 2.6, the new configurations are not created in HDP 2.6 - they're created in HDP 2.5 which causes services to fail on restart.
>
> STR:
> - Create an HDP 2.5 cluster with ZK + HDFS
> - Do an EU to HDP 2.6
>
> The EU will fail right after the "move stack version" step.
>
> It looks like the bug is in {{ConfigHelper}} where it ignores the passed-in {{StackId}} and uses the Cluster's stack.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java ff0becc822
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b4f7120836
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 16fc971e27
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java a5b40ff256
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 06b6217051
>
>
> Diff: https://reviews.apache.org/r/61317/diff/2/
>
>
> Testing
> -------
>
> Full upgrade/downgrade
>
> UTs PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 61317: Configs For Target Stack Are Not Created
During Upgrade
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61317/#review181975
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Aug. 1, 2017, 8:32 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61317/
> -----------------------------------------------------------
>
> (Updated Aug. 1, 2017, 8:32 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-21626
> https://issues.apache.org/jira/browse/AMBARI-21626
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an upgrade from HDP 2.5 to HDP 2.6, the new configurations are not created in HDP 2.6 - they're created in HDP 2.5 which causes services to fail on restart.
>
> STR:
> - Create an HDP 2.5 cluster with ZK + HDFS
> - Do an EU to HDP 2.6
>
> The EU will fail right after the "move stack version" step.
>
> It looks like the bug is in {{ConfigHelper}} where it ignores the passed-in {{StackId}} and uses the Cluster's stack.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java ff0becc822
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b4f7120836
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 16fc971e27
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java a5b40ff256
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 06b6217051
>
>
> Diff: https://reviews.apache.org/r/61317/diff/2/
>
>
> Testing
> -------
>
> Full upgrade/downgrade
>
> UTs PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 61317: Configs For Target Stack Are Not Created
During Upgrade
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61317/#review181936
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java
Lines 551-560 (patched)
<https://reviews.apache.org/r/61317/#comment257763>
So it turns out that in branch-2.6, we need to reintroduce this method since the whole idea of a cluster -> stack still exists.
The problem with the previous implementation was that reverting by service config versions missed things like cluster-env which has no service.
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java
Lines 874-883 (patched)
<https://reviews.apache.org/r/61317/#comment257764>
Since the cluster still has a stack in branch-2.6, we can use current/desired to determine if configs need to be reverted
- Jonathan Hurley
On Aug. 1, 2017, 8:32 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61317/
> -----------------------------------------------------------
>
> (Updated Aug. 1, 2017, 8:32 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-21626
> https://issues.apache.org/jira/browse/AMBARI-21626
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an upgrade from HDP 2.5 to HDP 2.6, the new configurations are not created in HDP 2.6 - they're created in HDP 2.5 which causes services to fail on restart.
>
> STR:
> - Create an HDP 2.5 cluster with ZK + HDFS
> - Do an EU to HDP 2.6
>
> The EU will fail right after the "move stack version" step.
>
> It looks like the bug is in {{ConfigHelper}} where it ignores the passed-in {{StackId}} and uses the Cluster's stack.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java ff0becc822
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b4f7120836
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 16fc971e27
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java a5b40ff256
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 06b6217051
>
>
> Diff: https://reviews.apache.org/r/61317/diff/2/
>
>
> Testing
> -------
>
> Full upgrade/downgrade
>
> UTs PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 61317: Configs For Target Stack Are Not Created
During Upgrade
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61317/#review181928
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java
Lines 147-149 (patched)
<https://reviews.apache.org/r/61317/#comment257750>
Since this code was taken from trunk (where there is no stack ID of a cluster anymore), we needed to add this back into the branch-2.6 line...
This allows the configurations created during upgrade to be associated with the correct stack.
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
Lines 1241-1242 (patched)
<https://reviews.apache.org/r/61317/#comment257752>
Not sure what happened here with the backport, but this method was missing actually populating the default config types. Without this statement, we ended up with maps that looked like:
hdfs-site = {}, yarn-env = {}, etc
This caused empty configuration types to be created on upgrade.
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java
Lines 893-896 (original), 893-896 (patched)
<https://reviews.apache.org/r/61317/#comment257753>
This needs to also go into trunk, as the bug exists there too - we have to undo the configurations for _every_ service - we should not be returning here, but continuing.
- Jonathan Hurley
On Aug. 1, 2017, 8:32 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61317/
> -----------------------------------------------------------
>
> (Updated Aug. 1, 2017, 8:32 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-21626
> https://issues.apache.org/jira/browse/AMBARI-21626
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an upgrade from HDP 2.5 to HDP 2.6, the new configurations are not created in HDP 2.6 - they're created in HDP 2.5 which causes services to fail on restart.
>
> STR:
> - Create an HDP 2.5 cluster with ZK + HDFS
> - Do an EU to HDP 2.6
>
> The EU will fail right after the "move stack version" step.
>
> It looks like the bug is in {{ConfigHelper}} where it ignores the passed-in {{StackId}} and uses the Cluster's stack.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java ff0becc822
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 16fc971e27
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java a5b40ff256
>
>
> Diff: https://reviews.apache.org/r/61317/diff/1/
>
>
> Testing
> -------
>
> Full upgrade/downgrade
>
> UTs PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>