You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Myroslav Papirkovskyy <mp...@hortonworks.com> on 2017/12/06 18:52:09 UTC
Review Request 64388: Tez service check hangs when adding service
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64388/
-----------------------------------------------------------
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle.
Bugs: AMBARI-22605
https://issues.apache.org/jira/browse/AMBARI-22605
Repository: ambari
Description
-------
STR:
Deploy minimal cluster (HDFS, YARN+MR2, ZooKeeper).
Add Tez service.
Service check hangs for a long time.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 85e0c1cf130974377abb7b685f191e20324b090e
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java d5a1ac4fd06377a992a8286d6fca366536a25157
Diff: https://reviews.apache.org/r/64388/diff/1/
Testing
-------
mvn clean test
manual check
Thanks,
Myroslav Papirkovskyy
Re: Review Request 64388: Tez service check hangs when adding service
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Dec. 6, 2017, 2 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java
> > Lines 134-139 (patched)
> > <https://reviews.apache.org/r/64388/diff/1/?file=1909709#file1909709line134>
> >
> > In this problem, you're saying that after upgrading the cluster from BI to HDP, adding a service is looking it up via BigInsights.
> >
> > ```
> > org.apache.ambari.server.StackAccessException: Stack data, stackName=BigInsights, stackVersion=4.2.5, serviceName=TEZ
> > at org.apache.ambari.server.api.services.AmbariMetaInfo.getService(AmbariMetaInfo.java:532)
> > at org.apache.ambari.server.api.services.AmbariMetaInfo.getComponent(AmbariMetaInfo.java:284)
> > at org.apache.ambari.server.events.listeners.upgrade.HostVersionOutOfSyncListener.onServiceEvent(HostVersionOutOfSyncListener.java:263)
> > ```
> >
> > Doesn't this mean that the lookup is using the wrong stack version:
> >
> > ```
> > ComponentInfo component = ami.get().getComponent(repositoryVersion.getStackName(),
> > repositoryVersion.getStackVersion(), serviceName, componentName);
> > ```
> >
> > Instead of skipping the lookup (which could cause problems with reporting component versions), we should figure out why the wrong stack is being used to lookup the information. HDP should be used since it's current...
See my other comment.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64388/#review193018
-----------------------------------------------------------
On Dec. 6, 2017, 1:52 p.m., Myroslav Papirkovskyy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64388/
> -----------------------------------------------------------
>
> (Updated Dec. 6, 2017, 1:52 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle.
>
>
> Bugs: AMBARI-22605
> https://issues.apache.org/jira/browse/AMBARI-22605
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> Deploy minimal cluster (HDFS, YARN+MR2, ZooKeeper).
> Add Tez service.
> Service check hangs for a long time.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 85e0c1cf130974377abb7b685f191e20324b090e
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java d5a1ac4fd06377a992a8286d6fca366536a25157
>
>
> Diff: https://reviews.apache.org/r/64388/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64388: Tez service check hangs when adding service
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64388/#review193018
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java
Lines 134-139 (patched)
<https://reviews.apache.org/r/64388/#comment271506>
In this problem, you're saying that after upgrading the cluster from BI to HDP, adding a service is looking it up via BigInsights.
```
org.apache.ambari.server.StackAccessException: Stack data, stackName=BigInsights, stackVersion=4.2.5, serviceName=TEZ
at org.apache.ambari.server.api.services.AmbariMetaInfo.getService(AmbariMetaInfo.java:532)
at org.apache.ambari.server.api.services.AmbariMetaInfo.getComponent(AmbariMetaInfo.java:284)
at org.apache.ambari.server.events.listeners.upgrade.HostVersionOutOfSyncListener.onServiceEvent(HostVersionOutOfSyncListener.java:263)
```
Doesn't this mean that the lookup is using the wrong stack version:
```
ComponentInfo component = ami.get().getComponent(repositoryVersion.getStackName(),
repositoryVersion.getStackVersion(), serviceName, componentName);
```
Instead of skipping the lookup (which could cause problems with reporting component versions), we should figure out why the wrong stack is being used to lookup the information. HDP should be used since it's current...
- Jonathan Hurley
On Dec. 6, 2017, 1:52 p.m., Myroslav Papirkovskyy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64388/
> -----------------------------------------------------------
>
> (Updated Dec. 6, 2017, 1:52 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle.
>
>
> Bugs: AMBARI-22605
> https://issues.apache.org/jira/browse/AMBARI-22605
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> Deploy minimal cluster (HDFS, YARN+MR2, ZooKeeper).
> Add Tez service.
> Service check hangs for a long time.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 85e0c1cf130974377abb7b685f191e20324b090e
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java d5a1ac4fd06377a992a8286d6fca366536a25157
>
>
> Diff: https://reviews.apache.org/r/64388/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64388: Tez service check hangs when adding service
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64388/#review193048
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java
Lines 132-133 (original), 140-141 (patched)
<https://reviews.apache.org/r/64388/#comment271545>
OK, so here's the issue - we're iterating over all host versions looking to see if any need to be marked as OUT_OF_SYNC.
The case we're trying to cover here is whether FOO_COMPONENT is not advertised in HDP 2.4 and doesn't affect HDP 2.4 host versions. But instead, it affects HDP 2.5+ since it's advertised there.
So I think skipping it with this is OK.
However, you also missed it in the other method onServiceEvent(...)
- Jonathan Hurley
On Dec. 6, 2017, 1:52 p.m., Myroslav Papirkovskyy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64388/
> -----------------------------------------------------------
>
> (Updated Dec. 6, 2017, 1:52 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle.
>
>
> Bugs: AMBARI-22605
> https://issues.apache.org/jira/browse/AMBARI-22605
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> Deploy minimal cluster (HDFS, YARN+MR2, ZooKeeper).
> Add Tez service.
> Service check hangs for a long time.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 85e0c1cf130974377abb7b685f191e20324b090e
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java d5a1ac4fd06377a992a8286d6fca366536a25157
>
>
> Diff: https://reviews.apache.org/r/64388/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64388: Tez service check hangs when adding service
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64388/#review193122
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
Lines 3092-3096 (patched)
<https://reviews.apache.org/r/64388/#comment271624>
One more question - what is the reason for this code? Is it necessary for this fix?
- Jonathan Hurley
On Dec. 7, 2017, 9:43 a.m., Myroslav Papirkovskyy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64388/
> -----------------------------------------------------------
>
> (Updated Dec. 7, 2017, 9:43 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle.
>
>
> Bugs: AMBARI-22605
> https://issues.apache.org/jira/browse/AMBARI-22605
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> Deploy minimal cluster (HDFS, YARN+MR2, ZooKeeper).
> Add Tez service.
> Service check hangs for a long time.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 85e0c1cf130974377abb7b685f191e20324b090e
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java d5a1ac4fd06377a992a8286d6fca366536a25157
>
>
> Diff: https://reviews.apache.org/r/64388/diff/2/
>
>
> Testing
> -------
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64388: Tez service check hangs when adding service
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64388/#review193124
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Dec. 7, 2017, 9:43 a.m., Myroslav Papirkovskyy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64388/
> -----------------------------------------------------------
>
> (Updated Dec. 7, 2017, 9:43 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle.
>
>
> Bugs: AMBARI-22605
> https://issues.apache.org/jira/browse/AMBARI-22605
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
> Deploy minimal cluster (HDFS, YARN+MR2, ZooKeeper).
> Add Tez service.
> Service check hangs for a long time.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 85e0c1cf130974377abb7b685f191e20324b090e
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java d5a1ac4fd06377a992a8286d6fca366536a25157
>
>
> Diff: https://reviews.apache.org/r/64388/diff/2/
>
>
> Testing
> -------
>
> mvn clean test
> manual check
>
>
> Thanks,
>
> Myroslav Papirkovskyy
>
>
Re: Review Request 64388: Tez service check hangs when adding service
Posted by Myroslav Papirkovskyy <mp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64388/
-----------------------------------------------------------
(Updated Гру. 7, 2017, 4:43 після полудня)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Sid Wagle.
Bugs: AMBARI-22605
https://issues.apache.org/jira/browse/AMBARI-22605
Repository: ambari
Description
-------
STR:
Deploy minimal cluster (HDFS, YARN+MR2, ZooKeeper).
Add Tez service.
Service check hangs for a long time.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 85e0c1cf130974377abb7b685f191e20324b090e
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/HostVersionOutOfSyncListener.java d5a1ac4fd06377a992a8286d6fca366536a25157
Diff: https://reviews.apache.org/r/64388/diff/2/
Changes: https://reviews.apache.org/r/64388/diff/1-2/
Testing
-------
mvn clean test
manual check
Thanks,
Myroslav Papirkovskyy