You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by na...@bt.com on 2021/10/06 11:03:45 UTC
Fw: Bigtop Ambari Mpack Issue
Hi There,
I'm not sure which community fits the issue I'm having best, so apologies if it seems like I am duplicating an email across Bigtop and Ambari.
As you can see in the below email, I am deploying Ambari 2.7.5 with the Bigtop Mpack from the master branch. It all looks fine until I go to install the services from the mpack and get the below error.
I've have retried the deployment again on a fresh cluster, but still getting the same error. When I look at the file '/var/lib/ambari-agent/data/command-13.json' it doesn't have a stack packages object in the cluster-env object. I have also attached a copy of the file. Stack Packages is also not present in the cluster-env.xml on the server.
I'm a bit confused as to where I should look next, so if anyone has any suggestions, I would really appreciate it.
Kind Regards,
Nathan
________________________________
From: English,N,Nathan,VIR R
Sent: 06 October 2021 10:13
To: user@bigtop.apache.org <us...@bigtop.apache.org>
Subject: Bigtop Ambari Mpack Issue
Hi There,
I am deploying Ambari 2.7.5 with the Bigtop Mpack from the master branch. It all looks fine until I go to install the services from the mpack and get the below error.
Looking at stack_select.py[1], It looks like it's searching for the 'stack_packages' in the cluster-env.xml (Path: "/configurations/cluster-env/stack_packages") however when I look at the cluster-env.xml[2] in the bigtop mpack I don't see anything called stack_packages? I could be completely wrong at what it's looking for though!
Does anyone have any ideas or suggestions?
Kind Regards,
Nathan
stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 39, in <module>
AfterInstallHook().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 32, in hook
setup_stack_symlinks(self.stroutfile)
File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py", line 53, in setup_stack_symlinks
stack_packages = stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 185, in get_packages
raise Fail("The stack packages are not defined on the command. Unable to load packages for the stack-select tool")
resource_management.core.exceptions.Fail: The stack packages are not defined on the command. Unable to load packages for the stack-select tool
stdout:
2021-10-06 08:18:24,404 - Stack Feature Version Info: Cluster Stack=1.0, Command Stack=None, Command Version=None -> 1.0
2021-10-06 08:18:24,411 - Skipping creation of User and Group as host is sys prepped or ignore_groupsusers_create flag is on
2021-10-06 08:18:24,412 - Skipping setting dfs cluster admin and tez view acls as host is sys prepped
2021-10-06 08:18:24,412 - FS Type: HDFS
2021-10-06 08:18:24,412 - Directory['/etc/hadoop'] {'mode': 0755}
2021-10-06 08:18:24,464 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2021-10-06 08:18:24,466 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2021-10-06 08:18:24,487 - Repository['BGTP-1.0-repo-4'] {'base_url': 'http://c7905.ambari.apache.org/', 'action': ['prepare'], 'components': [u'BGTP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-bgtp-4', 'mirror_list': None}
2021-10-06 08:18:24,501 - Repository[None] {'action': ['create']}
2021-10-06 08:18:24,503 - File['/tmp/tmpSnZLdC'] {'content': '[BGTP-1.0-repo-4]\nname=BGTP-1.0-repo-4\nbaseurl=http://c7905.ambari.apache.org/\n\npath=/\nenabled=1\ngpgcheck=0'}
2021-10-06 08:18:24,504 - Writing File['/tmp/tmpSnZLdC'] because contents don't match
2021-10-06 08:18:24,505 - Rewriting /etc/yum.repos.d/ambari-bgtp-4.repo since it has changed.
2021-10-06 08:18:24,505 - File['/etc/yum.repos.d/ambari-bgtp-4.repo'] {'content': StaticFile('/tmp/tmpSnZLdC')}
2021-10-06 08:18:24,507 - Writing File['/etc/yum.repos.d/ambari-bgtp-4.repo'] because it doesn't exist
2021-10-06 08:18:24,508 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:24,689 - Skipping installation of existing package unzip
2021-10-06 08:18:24,689 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:24,712 - Skipping installation of existing package curl
2021-10-06 08:18:25,093 - Package['hadoop-yarn'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:25,271 - Skipping installation of existing package hadoop-yarn
2021-10-06 08:18:25,273 - Package['hadoop-hdfs'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:25,297 - Skipping installation of existing package hadoop-hdfs
2021-10-06 08:18:25,298 - Package['hadoop-mapreduce'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:25,321 - Skipping installation of existing package hadoop-mapreduce
1. https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
2. https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml
[https://opengraph.githubassets.com/8810b73a2c2c4b7cab46ecfe0397fc64e5b171d1a2679752c656e7c8f03060e5/apache/bigtop]<https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml>
bigtop/cluster-env.xml at release-1.5.0 · apache/bigtop<https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml>
Mirror of Apache Bigtop. Contribute to apache/bigtop development by creating an account on GitHub.
github.com
[https://opengraph.githubassets.com/9302541dd14e06f69f6b955eb7f20a4e495240e95027b33d65cf7da9995ea5e3/apache/ambari]<https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py>
ambari/stack_select.py at release-2.7.5 · apache/ambari<https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py>
Mirror of Apache Ambari. Contribute to apache/ambari development by creating an account on GitHub.
github.com
3.
Command failed after 1 tries
Re: Bigtop Ambari Mpack Issue
Posted by na...@bt.com.
Hi Matt,
That explains it! I had to bump the build due to various packages sitting behind the hortonworks paywall.
I’ll certainly sign up for the Dev mailing list, thanks for the reminder of it.
I love the bleeding edge, and I’ll certainly take a look at the documentation.
Thanks for the information, I appreciate it!
Kind Regards,
Nathan
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Matt Andruff <ma...@gmail.com>
Sent: Wednesday, October 6, 2021 4:07 pm
To: user@ambari.apache.org
Subject: Re: Fw: Bigtop Ambari Mpack Issue
You don't often get email from matt.andruff@gmail.com. Learn why this is important<http://aka.ms/LearnAboutSenderIdentification>
Hey Nathan,
This Bigtop MPAC was only tested on the 2.6.2 of Ambari. So you are likely in new territory! Welcome to the bleeding edge!
Ambari MPack Documentation.<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FManagement%2BPacks&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604250711%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vHkbB58WJsGRCD%2BZzWiZh5EYMSgIbvPpJQSlGOpqXUI%3D&reserved=0>
You can try the following paths
Watch out for BigTop/Ambari gotchas. (Big Top supports OS's that aren't supported by Ambari. Make sure you are on a supported platform for both BIg Top and Ambari)
Look at the Stack for 2.7.5. You can borrow from it to find what's missing/needed to help update the MPack from 2.6.2 to work with 2.7.5
You may be able to also ask for help on the Ambari Dev forum. (Users@ambari.apache.org<ma...@ambari.apache.org> is likely not your target audience.) I haven't tried it but you might get lucky.
On Wed, Oct 6, 2021 at 7:04 AM <na...@bt.com>> wrote:
Hi There,
I'm not sure which community fits the issue I'm having best, so apologies if it seems like I am duplicating an email across Bigtop and Ambari.
As you can see in the below email, I am deploying Ambari 2.7.5 with the Bigtop Mpack from the master branch. It all looks fine until I go to install the services from the mpack and get the below error.
I've have retried the deployment again on a fresh cluster, but still getting the same error. When I look at the file '/var/lib/ambari-agent/data/command-13.json' it doesn't have a stack packages object in the cluster-env object. I have also attached a copy of the file. Stack Packages is also not present in the cluster-env.xml on the server.
I'm a bit confused as to where I should look next, so if anyone has any suggestions, I would really appreciate it.
Kind Regards,
Nathan
________________________________
From: English,N,Nathan,VIR R
Sent: 06 October 2021 10:13
To: user@bigtop.apache.org<ma...@bigtop.apache.org> <us...@bigtop.apache.org>>
Subject: Bigtop Ambari Mpack Issue
Hi There,
I am deploying Ambari 2.7.5 with the Bigtop Mpack from the master branch. It all looks fine until I go to install the services from the mpack and get the below error.
Looking at stack_select.py[1], It looks like it's searching for the 'stack_packages' in the cluster-env.xml (Path: "/configurations/cluster-env/stack_packages") however when I look at the cluster-env.xml[2] in the bigtop mpack I don't see anything called stack_packages? I could be completely wrong at what it's looking for though!
Does anyone have any ideas or suggestions?
Kind Regards,
Nathan
stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 39, in <module>
AfterInstallHook().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", line 32, in hook
setup_stack_symlinks(self.stroutfile)
File "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py", line 53, in setup_stack_symlinks
stack_packages = stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 185, in get_packages
raise Fail("The stack packages are not defined on the command. Unable to load packages for the stack-select tool")
resource_management.core.exceptions.Fail: The stack packages are not defined on the command. Unable to load packages for the stack-select tool
stdout:
2021-10-06 08:18:24,404 - Stack Feature Version Info: Cluster Stack=1.0, Command Stack=None, Command Version=None -> 1.0
2021-10-06 08:18:24,411 - Skipping creation of User and Group as host is sys prepped or ignore_groupsusers_create flag is on
2021-10-06 08:18:24,412 - Skipping setting dfs cluster admin and tez view acls as host is sys prepped
2021-10-06 08:18:24,412 - FS Type: HDFS
2021-10-06 08:18:24,412 - Directory['/etc/hadoop'] {'mode': 0755}
2021-10-06 08:18:24,464 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2021-10-06 08:18:24,466 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2021-10-06 08:18:24,487 - Repository['BGTP-1.0-repo-4'] {'base_url': 'http://c7905.ambari.apache.org/<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fc7905.ambari.apache.org%2F&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604260706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PhXohJ1GgI%2FQMqg4IkkybyE273nz8Suv87PjNpQ%2FD3A%3D&reserved=0>', 'action': ['prepare'], 'components': [u'BGTP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-bgtp-4', 'mirror_list': None}
2021-10-06 08:18:24,501 - Repository[None] {'action': ['create']}
2021-10-06 08:18:24,503 - File['/tmp/tmpSnZLdC'] {'content': '[BGTP-1.0-repo-4]\nname=BGTP-1.0-repo-4\nbaseurl=http://c7905.ambari.apache.org/\n\npath=/\nenabled=1\ngpgcheck=0<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fc7905.ambari.apache.org%2F%255Cn%255Cnpath%3D%2F%255Cnenabled%3D1%255Cngpgcheck%3D0&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604260706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PCe6hzAAiOY%2FxPfJe5T0kF%2FE7%2FwWJlRcWdfBsSar6KI%3D&reserved=0>'}
2021-10-06 08:18:24,504 - Writing File['/tmp/tmpSnZLdC'] because contents don't match
2021-10-06 08:18:24,505 - Rewriting /etc/yum.repos.d/ambari-bgtp-4.repo since it has changed.
2021-10-06 08:18:24,505 - File['/etc/yum.repos.d/ambari-bgtp-4.repo'] {'content': StaticFile('/tmp/tmpSnZLdC')}
2021-10-06 08:18:24,507 - Writing File['/etc/yum.repos.d/ambari-bgtp-4.repo'] because it doesn't exist
2021-10-06 08:18:24,508 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:24,689 - Skipping installation of existing package unzip
2021-10-06 08:18:24,689 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:24,712 - Skipping installation of existing package curl
2021-10-06 08:18:25,093 - Package['hadoop-yarn'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:25,271 - Skipping installation of existing package hadoop-yarn
2021-10-06 08:18:25,273 - Package['hadoop-hdfs'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:25,297 - Skipping installation of existing package hadoop-hdfs
2021-10-06 08:18:25,298 - Package['hadoop-mapreduce'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2021-10-06 08:18:25,321 - Skipping installation of existing package hadoop-mapreduce
1. https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.5%2Fambari-common%2Fsrc%2Fmain%2Fpython%2Fresource_management%2Flibraries%2Ffunctions%2Fstack_select.py&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604270702%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0IDfdIoKBzV6y8e7Kw%2BLz%2Bz1fIPn0CrP%2BqjYm8qHDiE%3D&reserved=0>
2. https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fbigtop%2Fblob%2Frelease-1.5.0%2Fbigtop-packages%2Fsrc%2Fcommon%2Fbigtop-ambari-mpack%2Fbgtp-ambari-mpack%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBGTP%2F1.0%2Fconfiguration%2Fcluster-env.xml&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604270702%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=z6eHQfV0ecZ4zXi1%2BlzR%2FwDDGkBA9NZn7x2EmKT69CU%3D&reserved=0>
[https://opengraph.githubassets.com/8810b73a2c2c4b7cab46ecfe0397fc64e5b171d1a2679752c656e7c8f03060e5/apache/bigtop]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fbigtop%2Fblob%2Frelease-1.5.0%2Fbigtop-packages%2Fsrc%2Fcommon%2Fbigtop-ambari-mpack%2Fbgtp-ambari-mpack%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBGTP%2F1.0%2Fconfiguration%2Fcluster-env.xml&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604280695%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mckvbaF49bJdtL6vpCxyYeM9YBdGb3%2BPe%2BGDOi2KBQ4%3D&reserved=0>
bigtop/cluster-env.xml at release-1.5.0 · apache/bigtop<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fbigtop%2Fblob%2Frelease-1.5.0%2Fbigtop-packages%2Fsrc%2Fcommon%2Fbigtop-ambari-mpack%2Fbgtp-ambari-mpack%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBGTP%2F1.0%2Fconfiguration%2Fcluster-env.xml&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604290689%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=AUf2GzVJ8rimYCkSPZjzq3dl0kn6kAzYyP1Fg83Fi2Y%3D&reserved=0>
Mirror of Apache Bigtop. Contribute to apache/bigtop development by creating an account on GitHub.
github.com<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604290689%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yONUuRhzW72HQQuWKt1GMLQQ1%2FHZI3w0Gmu5JqiOCNs%3D&reserved=0>
[https://opengraph.githubassets.com/9302541dd14e06f69f6b955eb7f20a4e495240e95027b33d65cf7da9995ea5e3/apache/ambari]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.5%2Fambari-common%2Fsrc%2Fmain%2Fpython%2Fresource_management%2Flibraries%2Ffunctions%2Fstack_select.py&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604300686%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RtSM7PuheRmWL5%2FdQO5DXKVjwq0tv2YyX4RevnqbqKw%3D&reserved=0>
ambari/stack_select.py at release-2.7.5 · apache/ambari<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.5%2Fambari-common%2Fsrc%2Fmain%2Fpython%2Fresource_management%2Flibraries%2Ffunctions%2Fstack_select.py&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604300686%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RtSM7PuheRmWL5%2FdQO5DXKVjwq0tv2YyX4RevnqbqKw%3D&reserved=0>
Mirror of Apache Ambari. Contribute to apache/ambari development by creating an account on GitHub.
github.com<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2F&data=04%7C01%7Cnathan.english%40bt.com%7Cb54e5731e6a742de760608d988ca3d1c%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637691224604310678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=iZ2Y%2FCdqbQnwVriXma%2Fd%2FDm4cFP0eJ5nJ515QiiwBDk%3D&reserved=0>
3.
Command failed after 1 tries
--
Live life
Laugh Often
Re: Fw: Bigtop Ambari Mpack Issue
Posted by Matt Andruff <ma...@gmail.com>.
Hey Nathan,
This Bigtop MPAC was only tested on the 2.6.2 of Ambari. So you are likely
in new territory! Welcome to the bleeding edge!
Ambari MPack Documentation.
<https://cwiki.apache.org/confluence/display/AMBARI/Management+Packs>
You can try the following paths
Watch out for BigTop/Ambari gotchas. (Big Top supports OS's that
aren't supported by Ambari. Make sure you are on a supported platform for
both BIg Top and Ambari)
Look at the Stack for 2.7.5. You can borrow from it to find what's
missing/needed to help update the MPack from 2.6.2 to work with 2.7.5
You may be able to also ask for help on the Ambari Dev forum. (
Users@ambari.apache.org is likely not your target audience.) I haven't
tried it but you might get lucky.
On Wed, Oct 6, 2021 at 7:04 AM <na...@bt.com> wrote:
> Hi There,
>
> I'm not sure which community fits the issue I'm having best, so apologies
> if it seems like I am duplicating an email across Bigtop and Ambari.
>
> As you can see in the below email, I am deploying Ambari 2.7.5 with the
> Bigtop Mpack from the master branch. It all looks fine until I go to
> install the services from the mpack and get the below error.
>
> I've have retried the deployment again on a fresh cluster, but still
> getting the same error. When I look at the file
> '/var/lib/ambari-agent/data/command-13.json' it doesn't have a stack
> packages object in the cluster-env object. I have also attached a copy of
> the file. Stack Packages is also not present in the cluster-env.xml on the
> server.
>
> I'm a bit confused as to where I should look next, so if anyone has any
> suggestions, I would really appreciate it.
>
> Kind Regards,
>
> Nathan
> ------------------------------
> *From:* English,N,Nathan,VIR R
> *Sent:* 06 October 2021 10:13
> *To:* user@bigtop.apache.org <us...@bigtop.apache.org>
> *Subject:* Bigtop Ambari Mpack Issue
>
> Hi There,
>
> I am deploying Ambari 2.7.5 with the Bigtop Mpack from the master branch.
> It all looks fine until I go to install the services from the mpack and get
> the below error.
>
> Looking at stack_select.py[1], It looks like it's searching for the
> 'stack_packages' in the cluster-env.xml (Path:
> "/configurations/cluster-env/stack_packages") however when I look at the
> cluster-env.xml[2] in the bigtop mpack I don't see anything called
> stack_packages? I could be completely wrong at what it's looking for though!
>
> Does anyone have any ideas or suggestions?
>
> Kind Regards,
>
> Nathan
>
> *stderr: *
> *Traceback (most recent call last):*
> * File
> "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py",
> line 39, in <module>*
> * AfterInstallHook().execute()*
> * File
> "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py",
> line 352, in execute*
> * method(env)*
> * File
> "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py",
> line 32, in hook*
> * setup_stack_symlinks(self.stroutfile)*
> * File
> "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py",
> line 53, in setup_stack_symlinks*
> * stack_packages =
> stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL)*
> * File
> "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py",
> line 185, in get_packages*
> * raise Fail("The stack packages are not defined on the command. Unable
> to load packages for the stack-select tool")*
> *resource_management.core.exceptions.Fail: The stack packages are not
> defined on the command. Unable to load packages for the stack-select tool*
>
> * stdout:*
> *2021-10-06 08:18:24,404 - Stack Feature Version Info: Cluster Stack=1.0,
> Command Stack=None, Command Version=None -> 1.0*
> *2021-10-06 08:18:24,411 - Skipping creation of User and Group as host is
> sys prepped or ignore_groupsusers_create flag is on*
> *2021-10-06 08:18:24,412 - Skipping setting dfs cluster admin and tez view
> acls as host is sys prepped*
> *2021-10-06 08:18:24,412 - FS Type: HDFS*
> *2021-10-06 08:18:24,412 - Directory['/etc/hadoop'] {'mode': 0755}*
> *2021-10-06 08:18:24,464 - File['/etc/hadoop/conf/hadoop-env.sh']
> {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}*
> *2021-10-06 08:18:24,466 -
> Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner':
> 'hdfs', 'group': 'hadoop', 'mode': 01777}*
> *2021-10-06 08:18:24,487 - Repository['BGTP-1.0-repo-4'] {'base_url':
> 'http://c7905.ambari.apache.org/ <http://c7905.ambari.apache.org/>',
> 'action': ['prepare'], 'components': [u'BGTP', 'main'], 'repo_template':
> '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list
> %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif
> %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-bgtp-4',
> 'mirror_list': None}*
> *2021-10-06 08:18:24,501 - Repository[None] {'action': ['create']}*
> *2021-10-06 08:18:24,503 - File['/tmp/tmpSnZLdC'] {'content':
> '[BGTP-1.0-repo-4]\nname=BGTP-1.0-repo-4\nbaseurl=http://c7905.ambari.apache.org/\n\npath=/\nenabled=1\ngpgcheck=0
> <http://c7905.ambari.apache.org/%5Cn%5Cnpath=/%5Cnenabled=1%5Cngpgcheck=0>'}*
> *2021-10-06 08:18:24,504 - Writing File['/tmp/tmpSnZLdC'] because contents
> don't match*
> *2021-10-06 08:18:24,505 - Rewriting /etc/yum.repos.d/ambari-bgtp-4.repo
> since it has changed.*
> *2021-10-06 08:18:24,505 - File['/etc/yum.repos.d/ambari-bgtp-4.repo']
> {'content': StaticFile('/tmp/tmpSnZLdC')}*
> *2021-10-06 08:18:24,507 - Writing
> File['/etc/yum.repos.d/ambari-bgtp-4.repo'] because it doesn't exist*
> *2021-10-06 08:18:24,508 - Package['unzip']
> {'retry_on_repo_unavailability': False, 'retry_count': 5}*
> *2021-10-06 08:18:24,689 - Skipping installation of existing package unzip*
> *2021-10-06 08:18:24,689 - Package['curl']
> {'retry_on_repo_unavailability': False, 'retry_count': 5}*
> *2021-10-06 08:18:24,712 - Skipping installation of existing package curl*
> *2021-10-06 08:18:25,093 - Package['hadoop-yarn']
> {'retry_on_repo_unavailability': False, 'retry_count': 5}*
> *2021-10-06 08:18:25,271 - Skipping installation of existing package
> hadoop-yarn*
> *2021-10-06 08:18:25,273 - Package['hadoop-hdfs']
> {'retry_on_repo_unavailability': False, 'retry_count': 5}*
> *2021-10-06 08:18:25,297 - Skipping installation of existing package
> hadoop-hdfs*
> *2021-10-06 08:18:25,298 - Package['hadoop-mapreduce']
> {'retry_on_repo_unavailability': False, 'retry_count': 5}*
> *2021-10-06 08:18:25,321 - Skipping installation of existing package
> hadoop-mapreduce*
>
>
> 1.
> https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
> 2.
> https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml
>
> <https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml>
> bigtop/cluster-env.xml at release-1.5.0 · apache/bigtop
> <https://github.com/apache/bigtop/blob/release-1.5.0/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/configuration/cluster-env.xml>
> Mirror of Apache Bigtop. Contribute to apache/bigtop development by
> creating an account on GitHub.
> github.com
>
>
> <https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py>
> ambari/stack_select.py at release-2.7.5 · apache/ambari
> <https://github.com/apache/ambari/blob/release-2.7.5/ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py>
> Mirror of Apache Ambari. Contribute to apache/ambari development by
> creating an account on GitHub.
> github.com
> **
> **
> **
> 3.
>
>
> Command failed after 1 tries
>
--
Live life
Laugh Often