You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2017/07/03 04:17:53 UTC
Review Request 60586: Use the Correct Effective Stack ID for Commands
During an Express Upgrade
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60586/
-----------------------------------------------------------
Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, and Sid Wagle.
Bugs: AMBARI-21391
https://issues.apache.org/jira/browse/AMBARI-21391
Repository: ambari
Description
-------
During an EU, the effective stack ID needs to be correctly determined based on whether the desired stack has changed.
STR:
- Create a stack with BIgInsights 4.2.5
- Register HDP 2.6
- Perform an EU
The "STOP" steps should be sending down host_params/stack_version = 4.2.5.0. All commands after the desired stack ID change should adjust to using the target (HDP 2.6) value.
{code}
"configuration_credentials": {},
"roleCommand": "CUSTOM_COMMAND",
"credentialStoreEnabled": "false",
"hostLevelParams": {
"stack_name": "BigInsights",
"current_version": "4.2.5.0-0000",
"custom_command": "RESTART",
"stack_version": "4.2.5",
...
},
"commandParams": {
"upgrade_direction": "upgrade",
"service_package_folder": "common-services/ZOOKEEPER/3.4.5/package",
"hooks_folder": "HDP/2.0.6/hooks",
"clusterName": "c1",
"script": "scripts/zookeeper_server.py",
"request_id": "10",
"version": "2.6.0.0-334",
"forceRefreshConfigTagsBeforeExecution": "true",
"upgrade_type": "nonrolling_upgrade",
"command_timeout": "900",
"script_type": "PYTHON"
},
{code}
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 25e8cab4a4
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 1bb3d77d8d
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c4f28b2ab0
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java 9ca0d4e9ee
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 8b1f4effb0
Diff: https://reviews.apache.org/r/60586/diff/1/
Testing
-------
PENDING...
Thanks,
Jonathan Hurley
Re: Review Request 60586: Use the Correct Effective Stack ID for
Commands During an Express Upgrade
Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60586/#review179572
-----------------------------------------------------------
Ship it!
Ship It!
- Attila Doroszlai
On July 4, 2017, 6:40 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60586/
> -----------------------------------------------------------
>
> (Updated July 4, 2017, 6:40 a.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-21391
> https://issues.apache.org/jira/browse/AMBARI-21391
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an EU, the effective stack ID needs to be correctly determined based on whether the desired stack has changed.
>
> STR:
> - Create a stack with BIgInsights 4.2.5
> - Register HDP 2.6
> - Perform an EU
>
> The "STOP" steps should be sending down host_params/stack_version = 4.2.5.0. All commands after the desired stack ID change should adjust to using the target (HDP 2.6) value.
>
> {code}
> "configuration_credentials": {},
> "roleCommand": "CUSTOM_COMMAND",
> "credentialStoreEnabled": "false",
> "hostLevelParams": {
> "stack_name": "BigInsights",
> "current_version": "4.2.5.0-0000",
> "custom_command": "RESTART",
> "stack_version": "4.2.5",
> ...
> },
> "commandParams": {
> "upgrade_direction": "upgrade",
> "service_package_folder": "common-services/ZOOKEEPER/3.4.5/package",
> "hooks_folder": "HDP/2.0.6/hooks",
> "clusterName": "c1",
> "script": "scripts/zookeeper_server.py",
> "request_id": "10",
> "version": "2.6.0.0-334",
> "forceRefreshConfigTagsBeforeExecution": "true",
> "upgrade_type": "nonrolling_upgrade",
> "command_timeout": "900",
> "script_type": "PYTHON"
> },
> {code}
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 25e8cab4a4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 2fc492bfba
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java cca87bc874
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 1bb3d77d8d
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c4f28b2ab0
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java 9ca0d4e9ee
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java 3a2616409a
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java 46949eaa97
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 8b1f4effb0
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java 9bb9b4597f
>
>
> Diff: https://reviews.apache.org/r/60586/diff/2/
>
>
> Testing
> -------
>
> Most UTs passing now.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 60586: Use the Correct Effective Stack ID for
Commands During an Express Upgrade
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On July 4, 2017, 6:21 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
> > Line 369 (original), 371 (patched)
> > <https://reviews.apache.org/r/60586/diff/2/?file=1768894#file1768894line371>
> >
> > `StackId.equals` also compares version, so inequality does not imply cross-stack downgrade, does it?
Inequality of this always implies cross-stack. For example:
HDP-2.4 / HDP-2.5 (cross stack)
FOO-1.0 / BAR-1.0 (cross stack)
> On July 4, 2017, 6:21 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java
> > Line 133 (original), 131 (patched)
> > <https://reviews.apache.org/r/60586/diff/2/?file=1768895#file1768895line136>
> >
> > Shouldn't this check `sourceStackId` instead of `targetStackId`? Also, why restrict the check to downgrade?
No, this check is ensuring that the target of the downgrade is the version that the cluster is currently on. It's a specific check to downgrade. Consider:
- Cluster is on HDP-2.2
- An upgrade to HDP-2.3 begins and gets to the end
- At this point, the CURRENT version is still HDP-2.2
When you downgrade, the "source" version is HDP-2.3 and the target is HDP-2.2. Therefore, this check is ensuring that on finalization, we are putting the cluster back exactly as it was.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60586/#review179555
-----------------------------------------------------------
On July 4, 2017, 12:40 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60586/
> -----------------------------------------------------------
>
> (Updated July 4, 2017, 12:40 a.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-21391
> https://issues.apache.org/jira/browse/AMBARI-21391
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an EU, the effective stack ID needs to be correctly determined based on whether the desired stack has changed.
>
> STR:
> - Create a stack with BIgInsights 4.2.5
> - Register HDP 2.6
> - Perform an EU
>
> The "STOP" steps should be sending down host_params/stack_version = 4.2.5.0. All commands after the desired stack ID change should adjust to using the target (HDP 2.6) value.
>
> {code}
> "configuration_credentials": {},
> "roleCommand": "CUSTOM_COMMAND",
> "credentialStoreEnabled": "false",
> "hostLevelParams": {
> "stack_name": "BigInsights",
> "current_version": "4.2.5.0-0000",
> "custom_command": "RESTART",
> "stack_version": "4.2.5",
> ...
> },
> "commandParams": {
> "upgrade_direction": "upgrade",
> "service_package_folder": "common-services/ZOOKEEPER/3.4.5/package",
> "hooks_folder": "HDP/2.0.6/hooks",
> "clusterName": "c1",
> "script": "scripts/zookeeper_server.py",
> "request_id": "10",
> "version": "2.6.0.0-334",
> "forceRefreshConfigTagsBeforeExecution": "true",
> "upgrade_type": "nonrolling_upgrade",
> "command_timeout": "900",
> "script_type": "PYTHON"
> },
> {code}
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 25e8cab4a4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 2fc492bfba
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java cca87bc874
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 1bb3d77d8d
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c4f28b2ab0
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java 9ca0d4e9ee
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java 3a2616409a
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java 46949eaa97
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 8b1f4effb0
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java 9bb9b4597f
>
>
> Diff: https://reviews.apache.org/r/60586/diff/2/
>
>
> Testing
> -------
>
> Most UTs passing now.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 60586: Use the Correct Effective Stack ID for
Commands During an Express Upgrade
Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60586/#review179555
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
Line 369 (original), 371 (patched)
<https://reviews.apache.org/r/60586/#comment254320>
`StackId.equals` also compares version, so inequality does not imply cross-stack downgrade, does it?
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java
Line 133 (original), 131 (patched)
<https://reviews.apache.org/r/60586/#comment254328>
Shouldn't this check `sourceStackId` instead of `targetStackId`? Also, why restrict the check to downgrade?
- Attila Doroszlai
On July 4, 2017, 6:40 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60586/
> -----------------------------------------------------------
>
> (Updated July 4, 2017, 6:40 a.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-21391
> https://issues.apache.org/jira/browse/AMBARI-21391
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During an EU, the effective stack ID needs to be correctly determined based on whether the desired stack has changed.
>
> STR:
> - Create a stack with BIgInsights 4.2.5
> - Register HDP 2.6
> - Perform an EU
>
> The "STOP" steps should be sending down host_params/stack_version = 4.2.5.0. All commands after the desired stack ID change should adjust to using the target (HDP 2.6) value.
>
> {code}
> "configuration_credentials": {},
> "roleCommand": "CUSTOM_COMMAND",
> "credentialStoreEnabled": "false",
> "hostLevelParams": {
> "stack_name": "BigInsights",
> "current_version": "4.2.5.0-0000",
> "custom_command": "RESTART",
> "stack_version": "4.2.5",
> ...
> },
> "commandParams": {
> "upgrade_direction": "upgrade",
> "service_package_folder": "common-services/ZOOKEEPER/3.4.5/package",
> "hooks_folder": "HDP/2.0.6/hooks",
> "clusterName": "c1",
> "script": "scripts/zookeeper_server.py",
> "request_id": "10",
> "version": "2.6.0.0-334",
> "forceRefreshConfigTagsBeforeExecution": "true",
> "upgrade_type": "nonrolling_upgrade",
> "command_timeout": "900",
> "script_type": "PYTHON"
> },
> {code}
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 25e8cab4a4
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 2fc492bfba
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java cca87bc874
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 1bb3d77d8d
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c4f28b2ab0
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java 9ca0d4e9ee
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java 3a2616409a
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java 46949eaa97
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 8b1f4effb0
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java 9bb9b4597f
>
>
> Diff: https://reviews.apache.org/r/60586/diff/2/
>
>
> Testing
> -------
>
> Most UTs passing now.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 60586: Use the Correct Effective Stack ID for
Commands During an Express Upgrade
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60586/
-----------------------------------------------------------
(Updated July 4, 2017, 12:40 a.m.)
Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-21391
https://issues.apache.org/jira/browse/AMBARI-21391
Repository: ambari
Description
-------
During an EU, the effective stack ID needs to be correctly determined based on whether the desired stack has changed.
STR:
- Create a stack with BIgInsights 4.2.5
- Register HDP 2.6
- Perform an EU
The "STOP" steps should be sending down host_params/stack_version = 4.2.5.0. All commands after the desired stack ID change should adjust to using the target (HDP 2.6) value.
{code}
"configuration_credentials": {},
"roleCommand": "CUSTOM_COMMAND",
"credentialStoreEnabled": "false",
"hostLevelParams": {
"stack_name": "BigInsights",
"current_version": "4.2.5.0-0000",
"custom_command": "RESTART",
"stack_version": "4.2.5",
...
},
"commandParams": {
"upgrade_direction": "upgrade",
"service_package_folder": "common-services/ZOOKEEPER/3.4.5/package",
"hooks_folder": "HDP/2.0.6/hooks",
"clusterName": "c1",
"script": "scripts/zookeeper_server.py",
"request_id": "10",
"version": "2.6.0.0-334",
"forceRefreshConfigTagsBeforeExecution": "true",
"upgrade_type": "nonrolling_upgrade",
"command_timeout": "900",
"script_type": "PYTHON"
},
{code}
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 25e8cab4a4
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 2fc492bfba
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java cca87bc874
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 1bb3d77d8d
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c4f28b2ab0
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java 9ca0d4e9ee
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java 3a2616409a
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java 46949eaa97
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 8b1f4effb0
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterEffectiveVersionTest.java 9bb9b4597f
Diff: https://reviews.apache.org/r/60586/diff/2/
Changes: https://reviews.apache.org/r/60586/diff/1-2/
Testing
-------
Most UTs passing now.
Thanks,
Jonathan Hurley
Re: Review Request 60586: Use the Correct Effective Stack ID for
Commands During an Express Upgrade
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60586/
-----------------------------------------------------------
(Updated July 4, 2017, 12:40 a.m.)
Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-21391
https://issues.apache.org/jira/browse/AMBARI-21391
Repository: ambari
Description
-------
During an EU, the effective stack ID needs to be correctly determined based on whether the desired stack has changed.
STR:
- Create a stack with BIgInsights 4.2.5
- Register HDP 2.6
- Perform an EU
The "STOP" steps should be sending down host_params/stack_version = 4.2.5.0. All commands after the desired stack ID change should adjust to using the target (HDP 2.6) value.
{code}
"configuration_credentials": {},
"roleCommand": "CUSTOM_COMMAND",
"credentialStoreEnabled": "false",
"hostLevelParams": {
"stack_name": "BigInsights",
"current_version": "4.2.5.0-0000",
"custom_command": "RESTART",
"stack_version": "4.2.5",
...
},
"commandParams": {
"upgrade_direction": "upgrade",
"service_package_folder": "common-services/ZOOKEEPER/3.4.5/package",
"hooks_folder": "HDP/2.0.6/hooks",
"clusterName": "c1",
"script": "scripts/zookeeper_server.py",
"request_id": "10",
"version": "2.6.0.0-334",
"forceRefreshConfigTagsBeforeExecution": "true",
"upgrade_type": "nonrolling_upgrade",
"command_timeout": "900",
"script_type": "PYTHON"
},
{code}
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 25e8cab4a4
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 1bb3d77d8d
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c4f28b2ab0
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java 9ca0d4e9ee
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 8b1f4effb0
Diff: https://reviews.apache.org/r/60586/diff/1/
Testing (updated)
-------
Most UTs passing now.
Thanks,
Jonathan Hurley