You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2017/07/20 13:04:26 UTC
Review Request 60998: ACTIVITY_ANALYZER Install failed: Error: Unable
to run the custom hook script
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60998/
-----------------------------------------------------------
Review request for Ambari and Eugene Chekanskiy.
Bugs: AMBARI-21535
https://issues.apache.org/jira/browse/AMBARI-21535
Repository: ambari
Description
-------
STR:
* Create ambari-qa and hbase user with UIDs less than 1000
* Navigate through UI install wizard. At customize servicespage Set the checkbox for "Misc->Have Ambari manage UIDs" to true/checked so that after deployment above created users will have UIDs >= 1000
* Go through install wizard to finish off deployment. But it fails at Activity analyser install with the below error
{
"href" : "http://172.27.25.210:8080/api/v1/clusters/cl1/requests/4/tasks/29",
"Tasks" : {
"attempt_cnt" : 1,
"cluster_name" : "cl1",
"command" : "INSTALL",
"command_detail" : "ACTIVITY_ANALYZER INSTALL",
"end_time" : 1500427251810,
"error_log" : "/var/lib/ambari-agent/data/errors-29.txt",
"exit_code" : 1,
"host_name" : "ctr-e134-1499953498516-19756-01-000005.hwx.site",
"id" : 29,
"output_log" : "/var/lib/ambari-agent/data/output-29.txt",
"request_id" : 4,
"role" : "ACTIVITY_ANALYZER",
"stage_id" : 0,
"start_time" : 1500427242346,
"status" : "FAILED",
"stderr" : "Traceback (most recent call last):\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py\", line 35, in <module>\n BeforeAnyHook().execute()\n File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 329, in execute\n method(env)\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py\", line 29, in hook\n setup_users()\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py\", line 60, in setup_users\n set_uid(params.smoke_user, params.smoke_user_dirs)\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py\", line 149, in set_uid\n not_if = format(\"(test $(id -u {user}) -gt 1000) || ({ignore_groupsusers_create_str})\"))\n File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 155, in __init__\n self.env.run()\n File
\"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 160, in run\n self.run_action(resource, action)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 124, in run_action\n provider_action()\n File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 262, in action_run\n tries=self.resource.tries, try_sleep=self.resource.try_sleep)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 72, in inner\n result = function(command, **kwargs)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 102, in checked_call\n tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 150, in _call_wrapper\n result = _call(command, **kwargs_copy)\n File \"/usr/lib/python2.6/site-packages/resource_management/core
/shell.py\", line 303, in _call\n raise ExecutionFailed(err_msg, code, out, err)\nresource_management.core.exceptions.ExecutionFailed: Execution of '/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0' returned 1. Failed to find Uid between 1000 and 2000\nError: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-29.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-29.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']",
"stdout" : "2017-07-19 01:20:49,237 - Stack Feature Version Info: Cluster Stack=2.6, Cluster Current Version=None, Command Stack=None, Command Version=None-> 2.6\n2017-07-19 01:20:49,286 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf\nUser Group mapping (user_group) is missing in the hostLevelParams\n2017-07-19 01:20:49,288 - Group['hadoop'] {}\n2017-07-19 01:20:49,292 - Group['users'] {}\n2017-07-19 01:20:49,293 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:49,297 - Writing File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't exist\n2017-07-19 01:20:49,297 - Changing permission for /var/lib/ambari-agent/tmp/changeUid.sh from 640 to 555\n2017-07-19 01:20:49,298 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'zookeeper')] {'sudo': True}\n2017-07-19 01:20:49,350 - call returned (0, '1042')\n2017-07-19 01:20:49,351 - User['zookeeper'] {'gid': 'hadoop', 'uid': '1042',
'groups': [u'hadoop']}\n2017-07-19 01:20:49,351 - Modifying user zookeeper\n2017-07-19 01:20:49,593 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:49,595 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'ams')] {'sudo': True}\n2017-07-19 01:20:49,765 - call returned (0, '1072')\n2017-07-19 01:20:49,766 - User['ams'] {'gid': 'hadoop', 'uid': '1072', 'groups': [u'hadoop']}\n2017-07-19 01:20:49,766 - Adding user User['ams']\n2017-07-19 01:20:50,173 - User['ambari-qa'] {'gid': 'hadoop', 'uid': 0, 'groups': [u'users']}\n2017-07-19 01:20:50,174 - Modifying user ambari-qa\n2017-07-19 01:20:50,337 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,338 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'hdfs')] {'sudo': True}\n2017-07-19 01:20:50,376 - call returned (0, '1043')\n2017-07-19 01:20:50,377 - User['hdfs'] {'gid': 'ha
doop', 'uid': '1043', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,377 - Modifying user hdfs\n2017-07-19 01:20:50,457 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,457 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'yarn')] {'sudo': True}\n2017-07-19 01:20:50,489 - call returned (0, '1045')\n2017-07-19 01:20:50,490 - User['yarn'] {'gid': 'hadoop', 'uid': '1045', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,490 - Modifying user yarn\n2017-07-19 01:20:50,564 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,565 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'mapred')] {'sudo': True}\n2017-07-19 01:20:50,597 - call returned (0, '1046')\n2017-07-19 01:20:50,597 - User['mapred'] {'gid': 'hadoop', 'uid': '1046', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,598 - Modifying user mapred\n2017-07-19 01:20:50,665 - File['/v
ar/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,666 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'hbase')] {'sudo': True}\n2017-07-19 01:20:50,698 - call returned (0, '999')\n2017-07-19 01:20:50,698 - User['hbase'] {'gid': 'hadoop', 'uid': '999', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,698 - Modifying user hbase\n2017-07-19 01:20:50,823 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,824 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}\nError: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-29.json', '/var/li
b/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-29.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']\n\nCommand failed after 1 tries\n",
"structured_out" : { }
Artifacts for this run can be found [here](http://qelog.hortonworks.com/log
/nat-yc-u16-foss-ambari-cstmdeploys-9/test-logs/ambari-cstmdeploys/artifacts/s
creenshots/com.hw.ambari.ui.tests.installer.customize_services_page.TestResett
ingUserUIDs/testResettingUserUIDsChanging/_19_1_26_35_Page_with_URL__http_172_
27_25_2108080_installer_step10__has_not_been_loaded_within_300_s/)
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/files/changeToSecureUid.sh 4663f10
Diff: https://reviews.apache.org/r/60998/diff/1/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 60998: ACTIVITY_ANALYZER Install failed: Error:
Unable to run the custom hook script
Posted by Eugene Chekanskiy <ec...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60998/#review181036
-----------------------------------------------------------
Ship it!
Ship It!
- Eugene Chekanskiy
On Липень 20, 2017, 1:04 після полудня, Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60998/
> -----------------------------------------------------------
>
> (Updated Липень 20, 2017, 1:04 після полудня)
>
>
> Review request for Ambari and Eugene Chekanskiy.
>
>
> Bugs: AMBARI-21535
> https://issues.apache.org/jira/browse/AMBARI-21535
>
>
> Repository: ambari
>
>
> Description
> -------
>
> STR:
>
> * Create ambari-qa and hbase user with UIDs less than 1000
> * Navigate through UI install wizard. At customize servicespage Set the checkbox for "Misc->Have Ambari manage UIDs" to true/checked so that after deployment above created users will have UIDs >= 1000
> * Go through install wizard to finish off deployment. But it fails at Activity analyser install with the below error
>
>
> {
> "href" : "http://172.27.25.210:8080/api/v1/clusters/cl1/requests/4/tasks/29",
> "Tasks" : {
> "attempt_cnt" : 1,
> "cluster_name" : "cl1",
> "command" : "INSTALL",
> "command_detail" : "ACTIVITY_ANALYZER INSTALL",
> "end_time" : 1500427251810,
> "error_log" : "/var/lib/ambari-agent/data/errors-29.txt",
> "exit_code" : 1,
> "host_name" : "ctr-e134-1499953498516-19756-01-000005.hwx.site",
> "id" : 29,
> "output_log" : "/var/lib/ambari-agent/data/output-29.txt",
> "request_id" : 4,
> "role" : "ACTIVITY_ANALYZER",
> "stage_id" : 0,
> "start_time" : 1500427242346,
> "status" : "FAILED",
> "stderr" : "Traceback (most recent call last):\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py\", line 35, in <module>\n BeforeAnyHook().execute()\n File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 329, in execute\n method(env)\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py\", line 29, in hook\n setup_users()\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py\", line 60, in setup_users\n set_uid(params.smoke_user, params.smoke_user_dirs)\n File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py\", line 149, in set_uid\n not_if = format(\"(test $(id -u {user}) -gt 1000) || ({ignore_groupsusers_create_str})\"))\n File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 155, in __init__\n self.env.run()\n Fil
e \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 160, in run\n self.run_action(resource, action)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 124, in run_action\n provider_action()\n File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 262, in action_run\n tries=self.resource.tries, try_sleep=self.resource.try_sleep)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 72, in inner\n result = function(command, **kwargs)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 102, in checked_call\n tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)\n File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 150, in _call_wrapper\n result = _call(command, **kwargs_copy)\n File \"/usr/lib/python2.6/site-packages/resource_management/co
re/shell.py\", line 303, in _call\n raise ExecutionFailed(err_msg, code, out, err)\nresource_management.core.exceptions.ExecutionFailed: Execution of '/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0' returned 1. Failed to find Uid between 1000 and 2000\nError: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-29.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-29.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']",
> "stdout" : "2017-07-19 01:20:49,237 - Stack Feature Version Info: Cluster Stack=2.6, Cluster Current Version=None, Command Stack=None, Command Version=None-> 2.6\n2017-07-19 01:20:49,286 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf\nUser Group mapping (user_group) is missing in the hostLevelParams\n2017-07-19 01:20:49,288 - Group['hadoop'] {}\n2017-07-19 01:20:49,292 - Group['users'] {}\n2017-07-19 01:20:49,293 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:49,297 - Writing File['/var/lib/ambari-agent/tmp/changeUid.sh'] because it doesn't exist\n2017-07-19 01:20:49,297 - Changing permission for /var/lib/ambari-agent/tmp/changeUid.sh from 640 to 555\n2017-07-19 01:20:49,298 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'zookeeper')] {'sudo': True}\n2017-07-19 01:20:49,350 - call returned (0, '1042')\n2017-07-19 01:20:49,351 - User['zookeeper'] {'gid': 'hadoop', 'uid': '1042
', 'groups': [u'hadoop']}\n2017-07-19 01:20:49,351 - Modifying user zookeeper\n2017-07-19 01:20:49,593 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:49,595 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'ams')] {'sudo': True}\n2017-07-19 01:20:49,765 - call returned (0, '1072')\n2017-07-19 01:20:49,766 - User['ams'] {'gid': 'hadoop', 'uid': '1072', 'groups': [u'hadoop']}\n2017-07-19 01:20:49,766 - Adding user User['ams']\n2017-07-19 01:20:50,173 - User['ambari-qa'] {'gid': 'hadoop', 'uid': 0, 'groups': [u'users']}\n2017-07-19 01:20:50,174 - Modifying user ambari-qa\n2017-07-19 01:20:50,337 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,338 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'hdfs')] {'sudo': True}\n2017-07-19 01:20:50,376 - call returned (0, '1043')\n2017-07-19 01:20:50,377 - User['hdfs'] {'gid': '
hadoop', 'uid': '1043', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,377 - Modifying user hdfs\n2017-07-19 01:20:50,457 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,457 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'yarn')] {'sudo': True}\n2017-07-19 01:20:50,489 - call returned (0, '1045')\n2017-07-19 01:20:50,490 - User['yarn'] {'gid': 'hadoop', 'uid': '1045', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,490 - Modifying user yarn\n2017-07-19 01:20:50,564 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,565 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'mapred')] {'sudo': True}\n2017-07-19 01:20:50,597 - call returned (0, '1046')\n2017-07-19 01:20:50,597 - User['mapred'] {'gid': 'hadoop', 'uid': '1046', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,598 - Modifying user mapred\n2017-07-19 01:20:50,665 - File['
/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,666 - call[('/var/lib/ambari-agent/tmp/changeUid.sh', u'hbase')] {'sudo': True}\n2017-07-19 01:20:50,698 - call returned (0, '999')\n2017-07-19 01:20:50,698 - User['hbase'] {'gid': 'hadoop', 'uid': '999', 'groups': [u'hadoop']}\n2017-07-19 01:20:50,698 - Modifying user hbase\n2017-07-19 01:20:50,823 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}\n2017-07-19 01:20:50,824 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}\nError: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-29.json', '/var/
lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-29.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1', '']\n\nCommand failed after 1 tries\n",
> "structured_out" : { }
>
>
> Artifacts for this run can be found [here](http://qelog.hortonworks.com/log
> /nat-yc-u16-foss-ambari-cstmdeploys-9/test-logs/ambari-cstmdeploys/artifacts/s
> creenshots/com.hw.ambari.ui.tests.installer.customize_services_page.TestResett
> ingUserUIDs/testResettingUserUIDsChanging/_19_1_26_35_Page_with_URL__http_172_
> 27_25_2108080_installer_step10__has_not_been_loaded_within_300_s/)
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/files/changeToSecureUid.sh 4663f10
>
>
> Diff: https://reviews.apache.org/r/60998/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>