You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2017/04/28 21:47:17 UTC
Review Request 58857: Setting Desired Stack Ids to Correct Values
During Service and Patch Upgrades
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58857/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
Bugs: AMBARI-20894
https://issues.apache.org/jira/browse/AMBARI-20894
Repository: ambari
Description
-------
Long story short here is that this is the first major breaking change for supporting service/patch upgrades. Some highlights:
- Services and components will now track their "desired repository"
- The cluster_version is deprecated and I've removed the use of it in many places in order to get this compiling. It will be further removed by a future commit
- The host components are now compared to their components to verify if the upgrade has succeeded
- Some things are still very wrong
-- I have about 300 unit test compile errors. Once I fix those, I'm sure they will all fail. It's something we plan to fix in iterations as more code is changed to support service and patch
-- There are DDL changes here; we will do the work to upgrade the database in a separate Jira
-- Lots of useless code has been removed
-- The python code still doesn't understand per-component versions; that is a follow-up Jira.
Please review and keep in mind that many areas are partially complete since there are many other Jiras tracking this work. That's why it's in a feature branch :)
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/script/script.py da47351
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java c1028dc
ambari-server/src/main/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnable.java 7dfbe47
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java b2a03e4
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 04b72ea
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java c3e66fc
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java ff8d0be
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 6556852
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 623851a
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java abf8e6b
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java 930a535
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntity.java ea2938b
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java 4ed9617
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentDesiredStateEntity.java 17fd323
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceDesiredStateEntity.java 885f995
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/AbstractUpgradeServerAction.java 5d73fac
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckAction.java 52c0cf2
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 32d6151
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java 7bcb9d0
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 1ef204d
ambari-server/src/main/java/org/apache/ambari/server/state/Service.java 5964e33
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java b5b6821
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java 104e456
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java 4cfb250
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceFactory.java a3a041b
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java a0c0db1
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 97f5003
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 92e01c2
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 228cf79
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java f1e93ac
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java 6c59784
ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeHelper.java 0aab0b5
ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeUtil.java b258aa8
ambari-server/src/main/python/ambari-server.py 737be6a
ambari-server/src/main/python/ambari_server/serverUpgrade.py 6f17900
ambari-server/src/main/python/ambari_server/setupActions.py 358bfc9
ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql b241dc2
ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 670bf17
ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 00b3248
ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql f6af968
ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 64a0137
ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 22b2c3d
ambari-server/src/test/python/TestAmbariServer.py 9579c22
ambari-server/src/test/python/TestServerUpgrade.py 9235ff9
Diff: https://reviews.apache.org/r/58857/diff/1/
Testing
-------
I upgraded a single service and did a happy dance when it finalized. That's about the best we're going to get with this patch...
Thanks,
Jonathan Hurley
Re: Review Request 58857: Setting Desired Stack Ids to Correct Values
During Service and Patch Upgrades
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58857/#review173410
-----------------------------------------------------------
Fix it, then Ship it!
I realize things are in flux here, but it looks reasonable. Only one small comment and let's get it in.
Once we get this and the patch I'm working on, we'll work on cleanup to get to a demo state. (My work will be required to show for the demo - adding a component using the new version).
Agreed that we'll get tests fixed up as we go.
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/AbstractUpgradeServerAction.java
Lines 102-103 (patched)
<https://reviews.apache.org/r/58857/#comment246412>
Should this be done in the constructor that takes the UpgradeEntity?
- Nate Cole
On April 28, 2017, 5:47 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58857/
> -----------------------------------------------------------
>
> (Updated April 28, 2017, 5:47 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-20894
> https://issues.apache.org/jira/browse/AMBARI-20894
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Long story short here is that this is the first major breaking change for supporting service/patch upgrades. Some highlights:
> - Services and components will now track their "desired repository"
> - The cluster_version is deprecated and I've removed the use of it in many places in order to get this compiling. It will be further removed by a future commit
> - The host components are now compared to their components to verify if the upgrade has succeeded
> - Some things are still very wrong
> -- I have about 300 unit test compile errors. Once I fix those, I'm sure they will all fail. It's something we plan to fix in iterations as more code is changed to support service and patch
> -- There are DDL changes here; we will do the work to upgrade the database in a separate Jira
> -- Lots of useless code has been removed
> -- The python code still doesn't understand per-component versions; that is a follow-up Jira.
>
> Please review and keep in mind that many areas are partially complete since there are many other Jiras tracking this work. That's why it's in a feature branch :)
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/script.py da47351
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java c1028dc
> ambari-server/src/main/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnable.java 7dfbe47
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java b2a03e4
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 04b72ea
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java c3e66fc
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java ff8d0be
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 6556852
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 623851a
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java abf8e6b
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java 930a535
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntity.java ea2938b
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java 4ed9617
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentDesiredStateEntity.java 17fd323
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceDesiredStateEntity.java 885f995
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/AbstractUpgradeServerAction.java 5d73fac
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckAction.java 52c0cf2
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 32d6151
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java 7bcb9d0
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 1ef204d
> ambari-server/src/main/java/org/apache/ambari/server/state/Service.java 5964e33
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java b5b6821
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java 104e456
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java 4cfb250
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceFactory.java a3a041b
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java a0c0db1
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 97f5003
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 92e01c2
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 228cf79
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java f1e93ac
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java 6c59784
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeHelper.java 0aab0b5
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeUtil.java b258aa8
> ambari-server/src/main/python/ambari-server.py 737be6a
> ambari-server/src/main/python/ambari_server/serverUpgrade.py 6f17900
> ambari-server/src/main/python/ambari_server/setupActions.py 358bfc9
> ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql b241dc2
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 670bf17
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 00b3248
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql f6af968
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 64a0137
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 22b2c3d
> ambari-server/src/test/python/TestAmbariServer.py 9579c22
> ambari-server/src/test/python/TestServerUpgrade.py 9235ff9
>
>
> Diff: https://reviews.apache.org/r/58857/diff/1/
>
>
> Testing
> -------
>
> I upgraded a single service and did a happy dance when it finalized. That's about the best we're going to get with this patch...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 58857: Setting Desired Stack Ids to Correct Values
During Service and Patch Upgrades
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58857/#review173720
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On May 2, 2017, 11:04 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58857/
> -----------------------------------------------------------
>
> (Updated May 2, 2017, 11:04 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-20894
> https://issues.apache.org/jira/browse/AMBARI-20894
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Long story short here is that this is the first major breaking change for supporting service/patch upgrades. Some highlights:
> - Services and components will now track their "desired repository"
> - The cluster_version is deprecated and I've removed the use of it in many places in order to get this compiling. It will be further removed by a future commit
> - The host components are now compared to their components to verify if the upgrade has succeeded
> - Some things are still very wrong
> -- I have about 300 unit test compile errors. Once I fix those, I'm sure they will all fail. It's something we plan to fix in iterations as more code is changed to support service and patch
> -- There are DDL changes here; we will do the work to upgrade the database in a separate Jira
> -- Lots of useless code has been removed
> -- The python code still doesn't understand per-component versions; that is a follow-up Jira.
>
> Please review and keep in mind that many areas are partially complete since there are many other Jiras tracking this work. That's why it's in a feature branch :)
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/script.py da47351
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java c1028dc
> ambari-server/src/main/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnable.java 7dfbe47
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java b2a03e4
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 04b72ea
> ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceComponentHostRequest.java 5b6c739
> ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceComponentHostResponse.java f6993f8
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java c3e66fc
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java ff8d0be
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java 46e791b
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 6556852
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 623851a
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java abf8e6b
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java 930a535
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntity.java ea2938b
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntity.java 0b3d8ce
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java 4ed9617
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentDesiredStateEntity.java 17fd323
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceDesiredStateEntity.java 885f995
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareDisableKerberosServerAction.java 5b65833
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/AbstractUpgradeServerAction.java 5d73fac
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckAction.java 52c0cf2
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 32d6151
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java 7bcb9d0
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 1ef204d
> ambari-server/src/main/java/org/apache/ambari/server/state/Service.java 5964e33
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java b5b6821
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java 104e456
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java 4cfb250
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceFactory.java a3a041b
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java a0c0db1
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 97f5003
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 92e01c2
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 228cf79
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java f1e93ac
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java 6c59784
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeHelper.java 0aab0b5
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeUtil.java b258aa8
> ambari-server/src/main/python/ambari-server.py 737be6a
> ambari-server/src/main/python/ambari_server/serverUpgrade.py 6f17900
> ambari-server/src/main/python/ambari_server/setupActions.py 358bfc9
> ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql b241dc2
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 670bf17
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 00b3248
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql f6af968
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 64a0137
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 22b2c3d
> ambari-server/src/main/resources/properties.json 9b2bbf8
> ambari-server/src/test/java/org/apache/ambari/server/agent/DummyHeartbeatConstants.java a7c53d5
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java 2dd91c0
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatTestHelper.java 2e65e8d
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java 909bf69
> ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java caf64be
> ambari-server/src/test/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnableTest.java 98f6f44
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java c9acfe9
> ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java 3a93fbf
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 628a56f
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 5275580
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProviderTest.java e7c1588
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentResourceProviderTest.java a2b0a0c
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java 8772b24
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ServiceResourceProviderTest.java 44c2493
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java 1c45589
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java 36889b2
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java e587f28
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeSummaryResourceProviderTest.java bee8983
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/state/GeneralServiceCalculatedStateTest.java 09d5569
> ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 6fb0028
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java fef9276
> ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListenerTest.java 7b8b68a
> ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 574ffa4
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckActionTest.java 73ab5e6
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java 976dea4
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java 5513271
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java 2d589ed
> ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java fc12e44
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 3dc34e3
> ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertEventPublisherTest.java 8c23b69
> ambari-server/src/test/java/org/apache/ambari/server/state/alerts/InitialAlertEventTest.java 890464d
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java d479ca2
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java 76f9130
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java e5e2643
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java 1a112d6
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java c6cef26
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ConcurrentServiceConfigVersionTest.java 46a039d
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ServiceComponentHostConcurrentWriteDeadlockTest.java 8f37ad7
> ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java 6db820b
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/StackUpgradeUtilTest.java 3ebf4e7
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java c209671
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog210Test.java c40eac6
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog220Test.java 4413ca8
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 4c11d10
> ambari-server/src/test/java/org/apache/ambari/server/utils/StageUtilsTest.java e9bd27c
> ambari-server/src/test/python/TestAmbariServer.py 9579c22
> ambari-server/src/test/python/TestServerUpgrade.py 9235ff9
>
>
> Diff: https://reviews.apache.org/r/58857/diff/3/
>
>
> Testing
> -------
>
> I upgraded a single service and did a happy dance when it finalized. That's about the best we're going to get with this patch...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 58857: Setting Desired Stack Ids to Correct Values
During Service and Patch Upgrades
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58857/
-----------------------------------------------------------
(Updated May 2, 2017, 4:04 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
Bugs: AMBARI-20894
https://issues.apache.org/jira/browse/AMBARI-20894
Repository: ambari
Description
-------
Long story short here is that this is the first major breaking change for supporting service/patch upgrades. Some highlights:
- Services and components will now track their "desired repository"
- The cluster_version is deprecated and I've removed the use of it in many places in order to get this compiling. It will be further removed by a future commit
- The host components are now compared to their components to verify if the upgrade has succeeded
- Some things are still very wrong
-- I have about 300 unit test compile errors. Once I fix those, I'm sure they will all fail. It's something we plan to fix in iterations as more code is changed to support service and patch
-- There are DDL changes here; we will do the work to upgrade the database in a separate Jira
-- Lots of useless code has been removed
-- The python code still doesn't understand per-component versions; that is a follow-up Jira.
Please review and keep in mind that many areas are partially complete since there are many other Jiras tracking this work. That's why it's in a feature branch :)
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/script/script.py da47351
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java c1028dc
ambari-server/src/main/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnable.java 7dfbe47
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java b2a03e4
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 04b72ea
ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceComponentHostRequest.java 5b6c739
ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceComponentHostResponse.java f6993f8
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java c3e66fc
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java ff8d0be
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java 46e791b
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 6556852
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 623851a
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java abf8e6b
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java 930a535
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntity.java ea2938b
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntity.java 0b3d8ce
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java 4ed9617
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentDesiredStateEntity.java 17fd323
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceDesiredStateEntity.java 885f995
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareDisableKerberosServerAction.java 5b65833
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/AbstractUpgradeServerAction.java 5d73fac
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckAction.java 52c0cf2
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 32d6151
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java 7bcb9d0
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 1ef204d
ambari-server/src/main/java/org/apache/ambari/server/state/Service.java 5964e33
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java b5b6821
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java 104e456
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java 4cfb250
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceFactory.java a3a041b
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java a0c0db1
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 97f5003
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 92e01c2
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 228cf79
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java f1e93ac
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java 6c59784
ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeHelper.java 0aab0b5
ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeUtil.java b258aa8
ambari-server/src/main/python/ambari-server.py 737be6a
ambari-server/src/main/python/ambari_server/serverUpgrade.py 6f17900
ambari-server/src/main/python/ambari_server/setupActions.py 358bfc9
ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql b241dc2
ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 670bf17
ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 00b3248
ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql f6af968
ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 64a0137
ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 22b2c3d
ambari-server/src/main/resources/properties.json 9b2bbf8
ambari-server/src/test/java/org/apache/ambari/server/agent/DummyHeartbeatConstants.java a7c53d5
ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java 2dd91c0
ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatTestHelper.java 2e65e8d
ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatHandler.java 909bf69
ambari-server/src/test/java/org/apache/ambari/server/agent/TestHeartbeatMonitor.java caf64be
ambari-server/src/test/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnableTest.java 98f6f44
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java c9acfe9
ambari-server/src/test/java/org/apache/ambari/server/configuration/RecoveryConfigHelperTest.java 3a93fbf
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 628a56f
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 5275580
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProviderTest.java e7c1588
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentResourceProviderTest.java a2b0a0c
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostResourceProviderTest.java 8772b24
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ServiceResourceProviderTest.java 44c2493
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java 1c45589
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java 36889b2
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java e587f28
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeSummaryResourceProviderTest.java bee8983
ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/state/GeneralServiceCalculatedStateTest.java 09d5569
ambari-server/src/test/java/org/apache/ambari/server/events/EventsTest.java 6fb0028
ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListenerTest.java fef9276
ambari-server/src/test/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListenerTest.java 7b8b68a
ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java 574ffa4
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckActionTest.java 73ab5e6
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java 976dea4
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/UpgradeActionTest.java 5513271
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java 2d589ed
ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java fc12e44
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 3dc34e3
ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertEventPublisherTest.java 8c23b69
ambari-server/src/test/java/org/apache/ambari/server/state/alerts/InitialAlertEventTest.java 890464d
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterDeadlockTest.java d479ca2
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterImplTest.java 76f9130
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java e5e2643
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java 1a112d6
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java c6cef26
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ConcurrentServiceConfigVersionTest.java 46a039d
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ServiceComponentHostConcurrentWriteDeadlockTest.java 8f37ad7
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java 6db820b
ambari-server/src/test/java/org/apache/ambari/server/upgrade/StackUpgradeUtilTest.java 3ebf4e7
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java c209671
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog210Test.java c40eac6
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog220Test.java 4413ca8
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java 4c11d10
ambari-server/src/test/java/org/apache/ambari/server/utils/StageUtilsTest.java e9bd27c
ambari-server/src/test/python/TestAmbariServer.py 9579c22
ambari-server/src/test/python/TestServerUpgrade.py 9235ff9
Diff: https://reviews.apache.org/r/58857/diff/3/
Changes: https://reviews.apache.org/r/58857/diff/2-3/
Testing
-------
I upgraded a single service and did a happy dance when it finalized. That's about the best we're going to get with this patch...
Thanks,
Jonathan Hurley
Re: Review Request 58857: Setting Desired Stack Ids to Correct Values
During Service and Patch Upgrades
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58857/
-----------------------------------------------------------
(Updated April 29, 2017, 10:42 a.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
Bugs: AMBARI-20894
https://issues.apache.org/jira/browse/AMBARI-20894
Repository: ambari
Description
-------
Long story short here is that this is the first major breaking change for supporting service/patch upgrades. Some highlights:
- Services and components will now track their "desired repository"
- The cluster_version is deprecated and I've removed the use of it in many places in order to get this compiling. It will be further removed by a future commit
- The host components are now compared to their components to verify if the upgrade has succeeded
- Some things are still very wrong
-- I have about 300 unit test compile errors. Once I fix those, I'm sure they will all fail. It's something we plan to fix in iterations as more code is changed to support service and patch
-- There are DDL changes here; we will do the work to upgrade the database in a separate Jira
-- Lots of useless code has been removed
-- The python code still doesn't understand per-component versions; that is a follow-up Jira.
Please review and keep in mind that many areas are partially complete since there are many other Jiras tracking this work. That's why it's in a feature branch :)
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/script/script.py da47351
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java c1028dc
ambari-server/src/main/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnable.java 7dfbe47
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java b2a03e4
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 04b72ea
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java c3e66fc
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ComponentResourceProvider.java ff8d0be
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceResourceProvider.java 6556852
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 623851a
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java abf8e6b
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostVersionDAO.java 930a535
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntity.java ea2938b
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostVersionEntity.java 4ed9617
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceComponentDesiredStateEntity.java 17fd323
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceDesiredStateEntity.java 885f995
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/AbstractUpgradeServerAction.java 5d73fac
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ComponentVersionCheckAction.java 52c0cf2
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java 32d6151
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpdateDesiredStackAction.java 7bcb9d0
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 1ef204d
ambari-server/src/main/java/org/apache/ambari/server/state/Service.java 5964e33
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java b5b6821
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentHost.java 104e456
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java 4cfb250
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceFactory.java a3a041b
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java a0c0db1
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeContext.java 97f5003
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java 92e01c2
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 228cf79
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java f1e93ac
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java 6c59784
ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeHelper.java 0aab0b5
ambari-server/src/main/java/org/apache/ambari/server/upgrade/StackUpgradeUtil.java b258aa8
ambari-server/src/main/python/ambari-server.py 737be6a
ambari-server/src/main/python/ambari_server/serverUpgrade.py 6f17900
ambari-server/src/main/python/ambari_server/setupActions.py 358bfc9
ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql b241dc2
ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 670bf17
ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 00b3248
ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql f6af968
ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 64a0137
ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 22b2c3d
ambari-server/src/test/java/org/apache/ambari/server/alerts/ComponentVersionAlertRunnableTest.java 98f6f44
ambari-server/src/test/python/TestAmbariServer.py 9579c22
ambari-server/src/test/python/TestServerUpgrade.py 9235ff9
Diff: https://reviews.apache.org/r/58857/diff/2/
Changes: https://reviews.apache.org/r/58857/diff/1-2/
Testing
-------
I upgraded a single service and did a happy dance when it finalized. That's about the best we're going to get with this patch...
Thanks,
Jonathan Hurley