You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/01/27 05:08:21 UTC
Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/
-----------------------------------------------------------
Review request for Ambari and Alejandro Fernandez.
Bugs: AMBARI-9340
https://issues.apache.org/jira/browse/AMBARI-9340
Repository: ambari
Description
-------
New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
Diff: https://reviews.apache.org/r/30306/diff/
Testing
-------
Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
Unit tests are in progress.
Thanks,
Alejandro Fernandez
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Yurii Shylov <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69847
-----------------------------------------------------------
Ship it!
Ship It!
- Yurii Shylov
On Янв. 27, 2015, 4:10 д.п., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Янв. 27, 2015, 4:10 д.п.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> Unit tests are in progress.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69889
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
<https://reviews.apache.org/r/30306/#comment114720>
No reason you can't also pass in the cluster here to the constructor of the HostComponentVersionEvent so that the event doesn't need to do the lookup.
- Jonathan Hurley
On Jan. 27, 2015, 3:49 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2015, 3:49 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java 7924ba8494fbed595e5419bee4eae1c2facbeba6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java 1ad0a0a261f596dad7271e26dd197ea755c715c9
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 7944908d4cdb9d10cf27fcd8b5e580471960b48d
> ambari-server/src/main/java/org/apache/ambari/server/events/HostComponentVersionEvent.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java f79db0ccc072cd600ba29bcb2f69292baf7dbdf1
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 159408f7d0adadd7e394a03c3d08a36556b325f7
> ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java cbf16e3d02ee32d8885cb896810f866d7ca60157
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c8abdf214d3074de165d11a14ec28c644a428f0b
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
> ambari-server/src/main/resources/common-services/AMS/0.1.0/metainfo.xml e62211c2555ba23c768fefc363df89d361f364b7
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 0f43ed5db69efa916f38c6bd576101f309fd8756
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml cbf55e04f2ccee65446960094e23184a77d2a5e4
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/metainfo.xml 9a05b6f004a4c217ac4a6306af40fd054ff920ff
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> Unit tests are in progress.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69916
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Jan. 27, 2015, 5:33 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2015, 5:33 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java 7924ba8494fbed595e5419bee4eae1c2facbeba6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java 1ad0a0a261f596dad7271e26dd197ea755c715c9
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 7944908d4cdb9d10cf27fcd8b5e580471960b48d
> ambari-server/src/main/java/org/apache/ambari/server/events/HostComponentVersionEvent.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java f79db0ccc072cd600ba29bcb2f69292baf7dbdf1
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 159408f7d0adadd7e394a03c3d08a36556b325f7
> ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java cbf16e3d02ee32d8885cb896810f866d7ca60157
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c8abdf214d3074de165d11a14ec28c644a428f0b
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
> ambari-server/src/main/resources/common-services/AMS/0.1.0/metainfo.xml e62211c2555ba23c768fefc363df89d361f364b7
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 0f43ed5db69efa916f38c6bd576101f309fd8756
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml cbf55e04f2ccee65446960094e23184a77d2a5e4
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/metainfo.xml 9a05b6f004a4c217ac4a6306af40fd054ff920ff
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 31:45.217s
> [INFO] Finished at: Tue Jan 27 14:33:46 PST 2015
> [INFO] Final Memory: 63M/773M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/
-----------------------------------------------------------
(Updated Jan. 27, 2015, 10:33 p.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Bugs: AMBARI-9340
https://issues.apache.org/jira/browse/AMBARI-9340
Repository: ambari
Description
-------
New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java 7924ba8494fbed595e5419bee4eae1c2facbeba6
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java 1ad0a0a261f596dad7271e26dd197ea755c715c9
ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 7944908d4cdb9d10cf27fcd8b5e580471960b48d
ambari-server/src/main/java/org/apache/ambari/server/events/HostComponentVersionEvent.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java f79db0ccc072cd600ba29bcb2f69292baf7dbdf1
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 159408f7d0adadd7e394a03c3d08a36556b325f7
ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java cbf16e3d02ee32d8885cb896810f866d7ca60157
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c8abdf214d3074de165d11a14ec28c644a428f0b
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
ambari-server/src/main/resources/common-services/AMS/0.1.0/metainfo.xml e62211c2555ba23c768fefc363df89d361f364b7
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 0f43ed5db69efa916f38c6bd576101f309fd8756
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml cbf55e04f2ccee65446960094e23184a77d2a5e4
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/metainfo.xml 9a05b6f004a4c217ac4a6306af40fd054ff920ff
Diff: https://reviews.apache.org/r/30306/diff/
Testing (updated)
-------
Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:45.217s
[INFO] Finished at: Tue Jan 27 14:33:46 PST 2015
[INFO] Final Memory: 63M/773M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/
-----------------------------------------------------------
(Updated Jan. 27, 2015, 10:06 p.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Minor tweaks. I just re-tested, turns out that the last patch had a deadlock since the writelock to save the RepoVersion would never get acquired. Running unit tests right now.
Bugs: AMBARI-9340
https://issues.apache.org/jira/browse/AMBARI-9340
Repository: ambari
Description
-------
New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java 7924ba8494fbed595e5419bee4eae1c2facbeba6
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java 1ad0a0a261f596dad7271e26dd197ea755c715c9
ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 7944908d4cdb9d10cf27fcd8b5e580471960b48d
ambari-server/src/main/java/org/apache/ambari/server/events/HostComponentVersionEvent.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java f79db0ccc072cd600ba29bcb2f69292baf7dbdf1
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 159408f7d0adadd7e394a03c3d08a36556b325f7
ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java cbf16e3d02ee32d8885cb896810f866d7ca60157
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c8abdf214d3074de165d11a14ec28c644a428f0b
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
ambari-server/src/main/resources/common-services/AMS/0.1.0/metainfo.xml e62211c2555ba23c768fefc363df89d361f364b7
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 0f43ed5db69efa916f38c6bd576101f309fd8756
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml cbf55e04f2ccee65446960094e23184a77d2a5e4
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/metainfo.xml 9a05b6f004a4c217ac4a6306af40fd054ff920ff
Diff: https://reviews.apache.org/r/30306/diff/
Testing
-------
Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
Unit tests are in progress.
Thanks,
Alejandro Fernandez
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/
-----------------------------------------------------------
(Updated Jan. 27, 2015, 8:49 p.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Used an asynchronous listener to improve performance.
Bugs: AMBARI-9340
https://issues.apache.org/jira/browse/AMBARI-9340
Repository: ambari
Description
-------
New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceComponentResponse.java 7924ba8494fbed595e5419bee4eae1c2facbeba6
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceComponentResourceProvider.java 1ad0a0a261f596dad7271e26dd197ea755c715c9
ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 7944908d4cdb9d10cf27fcd8b5e580471960b48d
ambari-server/src/main/java/org/apache/ambari/server/events/HostComponentVersionEvent.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/stack/ComponentModule.java f79db0ccc072cd600ba29bcb2f69292baf7dbdf1
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 159408f7d0adadd7e394a03c3d08a36556b325f7
ambari-server/src/main/java/org/apache/ambari/server/state/ComponentInfo.java cbf16e3d02ee32d8885cb896810f866d7ca60157
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java c8abdf214d3074de165d11a14ec28c644a428f0b
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
ambari-server/src/main/resources/common-services/AMS/0.1.0/metainfo.xml e62211c2555ba23c768fefc363df89d361f364b7
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 0f43ed5db69efa916f38c6bd576101f309fd8756
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml cbf55e04f2ccee65446960094e23184a77d2a5e4
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/metainfo.xml 9a05b6f004a4c217ac4a6306af40fd054ff920ff
Diff: https://reviews.apache.org/r/30306/diff/
Testing
-------
Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
Unit tests are in progress.
Thanks,
Alejandro Fernandez
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69745
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
<https://reviews.apache.org/r/30306/#comment114535>
Before, we would not have updated the cluster version because a host with only AMS would have caused this to return.
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
<https://reviews.apache.org/r/30306/#comment114536>
I split this out because the original function was even larger.
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
<https://reviews.apache.org/r/30306/#comment114537>
This is its own function because it is wrapped in Transactional.
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
<https://reviews.apache.org/r/30306/#comment114538>
The remaining logic moved into its own function.
- Alejandro Fernandez
On Jan. 27, 2015, 4:10 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2015, 4:10 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> Unit tests are in progress.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 27, 2015, 10:02 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java, lines 491-494
> > <https://reviews.apache.org/r/30306/diff/1/?file=836148#file836148line491>
> >
> > Manual database edits are not something that we should support because they mess with JPA's caching. This work seems dangerous to do on a heartbeat as it involves a lot of work and queries. I'd rather you use an event to do this work asynchronously.
>
> Alejandro Fernandez wrote:
> I wasn't expecting manual edits while the server was running, but if the value in the DB does change and it happens to equal the same value that the agent reports, which could certainly happen with support, then we still need a way for the changes to the hostcomponentstate to bubble up through the host_version and then the cluster_version tables. So this ensures consistency for very little additional overhead.
>
> Yurii Shylov wrote:
> This code is called only when components advertise their version and it doesn't happen often. I'd keep the safer variant until we get to the point when RU always succeeds without manual intervention.
Manual database edits don't really work b/c JPA's caching doesn't detect the changes. It's a bad idea while Ambari Server is running and not something we should code for. Regardless, this code here is fine, but should be done in an event.
> On Jan. 27, 2015, 10:02 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java, line 1411
> > <https://reviews.apache.org/r/30306/diff/1/?file=836150#file836150line1411>
> >
> > Naming; this is confusing and hard to read. Instead, maybe make this compInfo.isVersionAdvertised()
>
> Alejandro Fernandez wrote:
> versionAdvertised is a boolean, so I'm following the convention.
If the boolean is `versionAdvertised` then the method should if isVersionAdvertised/setVersionAdvertised
> On Jan. 27, 2015, 10:02 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java, lines 1451-1456
> > <https://reviews.apache.org/r/30306/diff/1/?file=836150#file836150line1451>
> >
> > Does this code need a lock? You're querying and then adding if absent, but couldn't 2 heartbeats come in while you're doing this work? Locking would probably be good, especially since I'm recommending that you do this work in an async event, not from the heartbeat handler directly.
>
> Alejandro Fernandez wrote:
> Technically yes. I had an earlier patch for the state transitions that used locking code but I ditched it to avoid performance hits. The heartbeats we are expecting can come from any host, so we should lock around creating the RepoVersion object. After that, we only expect a status heartbeat for one component at a time in a host, so the host_version updates may not need it, but the cluster_vesrion updates will also need locking code. I'll fix this in the next patch.
Using events will prevent the performance hit when using a lock on the heartbeat.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69800
-----------------------------------------------------------
On Jan. 26, 2015, 11:10 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Jan. 26, 2015, 11:10 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> Unit tests are in progress.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Jan. 27, 2015, 3:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java, lines 491-494
> > <https://reviews.apache.org/r/30306/diff/1/?file=836148#file836148line491>
> >
> > Manual database edits are not something that we should support because they mess with JPA's caching. This work seems dangerous to do on a heartbeat as it involves a lot of work and queries. I'd rather you use an event to do this work asynchronously.
I wasn't expecting manual edits while the server was running, but if the value in the DB does change and it happens to equal the same value that the agent reports, which could certainly happen with support, then we still need a way for the changes to the hostcomponentstate to bubble up through the host_version and then the cluster_version tables. So this ensures consistency for very little additional overhead.
> On Jan. 27, 2015, 3:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java, line 1411
> > <https://reviews.apache.org/r/30306/diff/1/?file=836150#file836150line1411>
> >
> > Naming; this is confusing and hard to read. Instead, maybe make this compInfo.isVersionAdvertised()
versionAdvertised is a boolean, so I'm following the convention.
> On Jan. 27, 2015, 3:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java, lines 1451-1456
> > <https://reviews.apache.org/r/30306/diff/1/?file=836150#file836150line1451>
> >
> > Does this code need a lock? You're querying and then adding if absent, but couldn't 2 heartbeats come in while you're doing this work? Locking would probably be good, especially since I'm recommending that you do this work in an async event, not from the heartbeat handler directly.
Technically yes. I had an earlier patch for the state transitions that used locking code but I ditched it to avoid performance hits. The heartbeats we are expecting can come from any host, so we should lock around creating the RepoVersion object. After that, we only expect a status heartbeat for one component at a time in a host, so the host_version updates may not need it, but the cluster_vesrion updates will also need locking code. I'll fix this in the next patch.
> On Jan. 27, 2015, 3:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java, lines 1722-1727
> > <https://reviews.apache.org/r/30306/diff/1/?file=836151#file836151line1722>
> >
> > Shouldn't the upgrade logic take care of this?
The new logic is for a brand new cluster install to detect the exact version installed when components advertise the version on a heartbeat. Originally, we were going to bootstrap everything in one shot, but we decided we need the heartbeat handler to recognize the version in order for a RU to also detect when a component has moved to a version. So this logic handles both brand new install and RU.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69800
-----------------------------------------------------------
On Jan. 27, 2015, 4:10 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2015, 4:10 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> Unit tests are in progress.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Yurii Shylov <yu...@gmail.com>.
> On Янв. 27, 2015, 3:02 п.п., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java, lines 491-494
> > <https://reviews.apache.org/r/30306/diff/1/?file=836148#file836148line491>
> >
> > Manual database edits are not something that we should support because they mess with JPA's caching. This work seems dangerous to do on a heartbeat as it involves a lot of work and queries. I'd rather you use an event to do this work asynchronously.
>
> Alejandro Fernandez wrote:
> I wasn't expecting manual edits while the server was running, but if the value in the DB does change and it happens to equal the same value that the agent reports, which could certainly happen with support, then we still need a way for the changes to the hostcomponentstate to bubble up through the host_version and then the cluster_version tables. So this ensures consistency for very little additional overhead.
This code is called only when components advertise their version and it doesn't happen often. I'd keep the safer variant until we get to the point when RU always succeeds without manual intervention.
- Yurii
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69800
-----------------------------------------------------------
On Янв. 27, 2015, 4:10 д.п., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Янв. 27, 2015, 4:10 д.п.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> Unit tests are in progress.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/#review69800
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java
<https://reviews.apache.org/r/30306/#comment114597>
Manual database edits are not something that we should support because they mess with JPA's caching. This work seems dangerous to do on a heartbeat as it involves a lot of work and queries. I'd rather you use an event to do this work asynchronously.
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
<https://reviews.apache.org/r/30306/#comment114599>
Naming; this is confusing and hard to read. Instead, maybe make this compInfo.isVersionAdvertised()
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
<https://reviews.apache.org/r/30306/#comment114600>
Does this code need a lock? You're querying and then adding if absent, but couldn't 2 heartbeats come in while you're doing this work? Locking would probably be good, especially since I'm recommending that you do this work in an async event, not from the heartbeat handler directly.
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
<https://reviews.apache.org/r/30306/#comment114598>
Shouldn't the upgrade logic take care of this?
- Jonathan Hurley
On Jan. 26, 2015, 11:10 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30306/
> -----------------------------------------------------------
>
> (Updated Jan. 26, 2015, 11:10 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9340
> https://issues.apache.org/jira/browse/AMBARI-9340
>
>
> Repository: ambari
>
>
> Description
> -------
>
> New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
> ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
> ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
>
> Diff: https://reviews.apache.org/r/30306/diff/
>
>
> Testing
> -------
>
> Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
> After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
>
> This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
>
> Unit tests are in progress.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30306: RU - Cluster install did not transition
cluster_version to CURRENT because host has only AMS
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30306/
-----------------------------------------------------------
(Updated Jan. 27, 2015, 4:10 a.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Bugs: AMBARI-9340
https://issues.apache.org/jira/browse/AMBARI-9340
Repository: ambari
Description
-------
New cluster install through the Install Wizard correctly transitioned all of the host_versions into CURRENT, but did not propagate the changes to the cluster_version, which stayed stuck on UPGRADING, because one host only had AMS, so it could not create a host_version record.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 092f9d54db36686412d7ccaa3318330688cf67cd
ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java b344a052549bb393028a3d60ea9fdb0c2f70fde0
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 220a5af9923444703eab4cff1875f19aeee08742
ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java 372325df8b1e1191b87441304dc9a994f8b56f72
Diff: https://reviews.apache.org/r/30306/diff/
Testing (updated)
-------
Installed a 3-node cluster with only HDFS, ZK, and AMS. One of the hosts only had the AMS component. When the install finishes, the other 2 hosts correctly had host_version records in CURRENT, and the cluster_version was in CURRENT.
After the install finishes, added a 4th node with all 3 components, and it correctly created a host_version record in the CURRENT state.
This means that the UI needs to be robust enough to handle the case where some hosts don't have host_version records.
Unit tests are in progress.
Thanks,
Alejandro Fernandez