You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Di Li <di...@ca.ibm.com> on 2017/12/04 19:20:28 UTC
Review Request 64311: AMBARI-22574 Failed to restart services on PPC
cluster post Ambari upgrade for IOP/HDP migration
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/
-----------------------------------------------------------
Review request for Ambari, Jonathan Hurley and Tim Thorpe.
Bugs: AMBARI-22574
https://issues.apache.org/jira/browse/AMBARI-22574
Repository: ambari
Description
-------
The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
Ambari server log contains the following error on unable to find redhat-ppc7 os type.
java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
{"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
,
{"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
Diff: https://reviews.apache.org/r/64311/diff/1/
Testing
-------
upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
Thanks,
Di Li
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192899
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
Lines 161 (patched)
<https://reviews.apache.org/r/64311/#comment271243>
You can use LOG.debug's built in support of {}
- Jonathan Hurley
On Dec. 5, 2017, 12:39 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2017, 12:39 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/3/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Tim Thorpe <tt...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192887
-----------------------------------------------------------
Ship it!
Ship It!
- Tim Thorpe
On Dec. 5, 2017, 5:39 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2017, 5:39 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/3/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Di Li <di...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/
-----------------------------------------------------------
(Updated Dec. 5, 2017, 7:11 p.m.)
Review request for Ambari, Jonathan Hurley and Tim Thorpe.
Changes
-------
updated some comments
Bugs: AMBARI-22574
https://issues.apache.org/jira/browse/AMBARI-22574
Repository: ambari
Description
-------
The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
Ambari server log contains the following error on unable to find redhat-ppc7 os type.
java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
{"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
,
{"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
Diff: https://reviews.apache.org/r/64311/diff/4/
Changes: https://reviews.apache.org/r/64311/diff/3-4/
Testing
-------
upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
Thanks,
Di Li
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Di Li <di...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/
-----------------------------------------------------------
(Updated Dec. 5, 2017, 5:39 p.m.)
Review request for Ambari, Jonathan Hurley and Tim Thorpe.
Bugs: AMBARI-22574
https://issues.apache.org/jira/browse/AMBARI-22574
Repository: ambari
Description
-------
The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
Ambari server log contains the following error on unable to find redhat-ppc7 os type.
java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
{"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
,
{"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
Diff: https://reviews.apache.org/r/64311/diff/3/
Changes: https://reviews.apache.org/r/64311/diff/2-3/
Testing
-------
upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
Thanks,
Di Li
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Di Li <di...@ca.ibm.com>.
> On Dec. 5, 2017, 1:01 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> > Lines 154 (patched)
> > <https://reviews.apache.org/r/64311/diff/2/?file=1908245#file1908245line154>
> >
> > We really shouldn't be hardcoding stack names and versions like this.
> >
> > what change caused this problem? there must be a better workaround
>
> Di Li wrote:
> Hello Jonathan,
>
> This is the code change that caused the err >> "AMBARI-22318 repositoryFile entity populating for wrong repository for RU (dgrinenko)"
>
> BigInsights 4.2.5 uses "redhat7" as the os type in the repo_version table instead of "redhat-ppc7" expected by Dmytro's code. Since this blocks the IOP/HDP migration, it has to happen at the ambari-upgrade time and needs to deal with BigInsights 4.2.5 repositories.
>
> Tim Thorpe wrote:
> Since the next release of 2.6 is supposed to be the last that supports migration, this code should only be there for a short time.
>
> Di Li wrote:
> Hello Jonathan,
>
> I provided a more generic fix that handles ALL records in the repo_version table. It also adds a new redhat-ppc7 section instead of modifying the existing redhat7 one. The idea is so that users are able to update repo urls post Ambari upgrade.
>
> Dmytro Grinenko wrote:
> @Di Li it is not my code expects redhat-ppc7, it is change introduced at os_family script, which adding ppc fow any platform on PowerPC CPU. It means that on Ambari prior 2.6 we used redhat7 as platform, same as for x86 arch, however with 2.6 we start to distinguish them.
Hello Dmytro,
yes, I understand the history of redhat-ppc for PowerPC repo as we have been dealing with it since Ambari 2.5.2 (where the migration was added) one incarnation or other.
Restarting services on Ambari 2.6.0 PPC cluser for migration was OK. It stopped working in Ambari 2.6.1 recently. This nontheless is a special case for the migration where IOP uses redhat7. The fix I pushed in doesn't use hardcoded stack name/version anymore and allows users to update repos via Ambari web UI by having a new repo_version entry than the in-place update logic I originally had.
- Di
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192856
-----------------------------------------------------------
On Dec. 5, 2017, 7:11 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2017, 7:11 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/4/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On Dec. 5, 2017, 1:01 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> > Lines 154 (patched)
> > <https://reviews.apache.org/r/64311/diff/2/?file=1908245#file1908245line154>
> >
> > We really shouldn't be hardcoding stack names and versions like this.
> >
> > what change caused this problem? there must be a better workaround
>
> Di Li wrote:
> Hello Jonathan,
>
> This is the code change that caused the err >> "AMBARI-22318 repositoryFile entity populating for wrong repository for RU (dgrinenko)"
>
> BigInsights 4.2.5 uses "redhat7" as the os type in the repo_version table instead of "redhat-ppc7" expected by Dmytro's code. Since this blocks the IOP/HDP migration, it has to happen at the ambari-upgrade time and needs to deal with BigInsights 4.2.5 repositories.
Since the next release of 2.6 is supposed to be the last that supports migration, this code should only be there for a short time.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192856
-----------------------------------------------------------
On Dec. 4, 2017, 8:08 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2017, 8:08 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/2/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Di Li <di...@ca.ibm.com>.
> On Dec. 5, 2017, 1:01 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> > Lines 154 (patched)
> > <https://reviews.apache.org/r/64311/diff/2/?file=1908245#file1908245line154>
> >
> > We really shouldn't be hardcoding stack names and versions like this.
> >
> > what change caused this problem? there must be a better workaround
>
> Di Li wrote:
> Hello Jonathan,
>
> This is the code change that caused the err >> "AMBARI-22318 repositoryFile entity populating for wrong repository for RU (dgrinenko)"
>
> BigInsights 4.2.5 uses "redhat7" as the os type in the repo_version table instead of "redhat-ppc7" expected by Dmytro's code. Since this blocks the IOP/HDP migration, it has to happen at the ambari-upgrade time and needs to deal with BigInsights 4.2.5 repositories.
>
> Tim Thorpe wrote:
> Since the next release of 2.6 is supposed to be the last that supports migration, this code should only be there for a short time.
Hello Jonathan,
I provided a more generic fix that handles ALL records in the repo_version table. It also adds a new redhat-ppc7 section instead of modifying the existing redhat7 one. The idea is so that users are able to update repo urls post Ambari upgrade.
- Di
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192856
-----------------------------------------------------------
On Dec. 5, 2017, 5:39 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2017, 5:39 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/3/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Di Li <di...@ca.ibm.com>.
> On Dec. 5, 2017, 1:01 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> > Lines 154 (patched)
> > <https://reviews.apache.org/r/64311/diff/2/?file=1908245#file1908245line154>
> >
> > We really shouldn't be hardcoding stack names and versions like this.
> >
> > what change caused this problem? there must be a better workaround
Hello Jonathan,
This is the code change that caused the err >> "AMBARI-22318 repositoryFile entity populating for wrong repository for RU (dgrinenko)"
BigInsights 4.2.5 uses "redhat7" as the os type in the repo_version table instead of "redhat-ppc7" expected by Dmytro's code. Since this blocks the IOP/HDP migration, it has to happen at the ambari-upgrade time and needs to deal with BigInsights 4.2.5 repositories.
- Di
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192856
-----------------------------------------------------------
On Dec. 4, 2017, 8:08 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2017, 8:08 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/2/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Dmytro Grinenko <ha...@gmail.com>.
> On Dec. 5, 2017, 1:01 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> > Lines 154 (patched)
> > <https://reviews.apache.org/r/64311/diff/2/?file=1908245#file1908245line154>
> >
> > We really shouldn't be hardcoding stack names and versions like this.
> >
> > what change caused this problem? there must be a better workaround
>
> Di Li wrote:
> Hello Jonathan,
>
> This is the code change that caused the err >> "AMBARI-22318 repositoryFile entity populating for wrong repository for RU (dgrinenko)"
>
> BigInsights 4.2.5 uses "redhat7" as the os type in the repo_version table instead of "redhat-ppc7" expected by Dmytro's code. Since this blocks the IOP/HDP migration, it has to happen at the ambari-upgrade time and needs to deal with BigInsights 4.2.5 repositories.
>
> Tim Thorpe wrote:
> Since the next release of 2.6 is supposed to be the last that supports migration, this code should only be there for a short time.
>
> Di Li wrote:
> Hello Jonathan,
>
> I provided a more generic fix that handles ALL records in the repo_version table. It also adds a new redhat-ppc7 section instead of modifying the existing redhat7 one. The idea is so that users are able to update repo urls post Ambari upgrade.
@Di Li it is not my code expects redhat-ppc7, it is change introduced at os_family script, which adding ppc fow any platform on PowerPC CPU. It means that on Ambari prior 2.6 we used redhat7 as platform, same as for x86 arch, however with 2.6 we start to distinguish them.
- Dmytro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192856
-----------------------------------------------------------
On Dec. 5, 2017, 7:11 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2017, 7:11 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/4/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192856
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
Lines 154 (patched)
<https://reviews.apache.org/r/64311/#comment271178>
We really shouldn't be hardcoding stack names and versions like this.
what change caused this problem? there must be a better workaround
- Jonathan Hurley
On Dec. 4, 2017, 3:08 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2017, 3:08 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/2/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Di Li <di...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/
-----------------------------------------------------------
(Updated Dec. 4, 2017, 8:08 p.m.)
Review request for Ambari, Jonathan Hurley and Tim Thorpe.
Bugs: AMBARI-22574
https://issues.apache.org/jira/browse/AMBARI-22574
Repository: ambari
Description
-------
The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
Ambari server log contains the following error on unable to find redhat-ppc7 os type.
java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
{"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
,
{"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
Diff: https://reviews.apache.org/r/64311/diff/2/
Changes: https://reviews.apache.org/r/64311/diff/1-2/
Testing
-------
upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
Thanks,
Di Li
Re: Review Request 64311: AMBARI-22574 Failed to restart services on
PPC cluster post Ambari upgrade for IOP/HDP migration
Posted by Tim Thorpe <tt...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192750
-----------------------------------------------------------
Ship it!
Ship It!
- Tim Thorpe
On Dec. 4, 2017, 7:20 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------------------------------------------------------
>
> (Updated Dec. 4, 2017, 7:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.
> Ambari server log contains the following error on unable to find redhat-ppc7 os type.
> java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
> at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:
> update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/1/
>
>
> Testing
> -------
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server jar with the one built with the fixes, run ambari-server upgrade, verify ambari database has the repositories info in repo_version is updated from redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>