You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2015/11/11 13:34:13 UTC
Review Request 40183: HBase Master Stop is Failed when Use 'Stop ALL'
(Upgrade from 2.1.1 to 2.1.3.0 )
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40183/
-----------------------------------------------------------
Review request for Ambari and Dmytro Sen.
Bugs: AMBARI-13832
https://issues.apache.org/jira/browse/AMBARI-13832
Repository: ambari
Description
-------
1)Deploy old version 2.1.1
2) Add host, ZK server at added host (Optional step)
3)Enable NN, RM HA (Optional step)
3)Make Ambari only upgrade to 2.1.3.0
4)Enable security (MIT)
Actual results:
HBase Master Stop is Failed when Use 'Stop ALL'
**But if I stop HBASE as separate operation, everything works.**
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py", line 149, in <module>
HbaseMaster().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py", line 90, in stop
hbase_service('master', action = 'stop')
File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py", line 50, in hbase_service
on_timeout = format("! ( {no_op_test} ) || {sudo} -H -E kill -9 `cat {pid_file}`"),
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 156, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 119, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 145, in _call_wrapper
result = checked_call(on_timeout)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '! ( ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1 ) || ambari-sudo.sh -H -E kill -9 `cat /var/run/hbase/hbase-hbase-master.pid`' returned 1. cat: /var/run/hbase/hbase-hbase-master.pid: Permission denied
usage: kill [ -s signal | -p ] [ -a ] pid ...
kill -l [ signal ]
stdout: /var/lib/ambari-agent/data/output-2285.txt
2015-11-09 14:09:44,480 - Execute['/usr/hdp/current/hbase-master/bin/hbase-daemon.sh --config /usr/hdp/current/hbase-master/conf stop master'] {'only_if': 'ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1', 'on_timeout': '! ( ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1 ) || ambari-sudo.sh -H -E kill -9 `cat /var/run/hbase/hbase-hbase-master.pid`', 'timeout': 30, 'user': 'hbase'}
2015-11-09 14:10:15,772 - Executing '! ( ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1 ) || ambari-sudo.sh -H -E kill -9 `cat /var/run/hbase/hbase-hbase-master.pid`'. Reason: Execution of 'ambari-sudo.sh su hbase -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/agslava/bin:/var/lib/ambari-agent'"'"' ; /usr/hdp/current/hbase-master/bin/hbase-daemon.sh --config /usr/hdp/current/hbase-master/conf stop master'' was killed due timeout after 30 seconds
Diffs
-----
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py f322aa4
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 3206e1c
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 84e5753
Diff: https://reviews.apache.org/r/40183/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 40183: HBase Master Stop is Failed when Use 'Stop
ALL' (Upgrade from 2.1.1 to 2.1.3.0 )
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40183/#review106041
-----------------------------------------------------------
Ship it!
Ship It!
- Dmytro Sen
On Ноя. 11, 2015, 12:34 п.п., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40183/
> -----------------------------------------------------------
>
> (Updated Ноя. 11, 2015, 12:34 п.п.)
>
>
> Review request for Ambari and Dmytro Sen.
>
>
> Bugs: AMBARI-13832
> https://issues.apache.org/jira/browse/AMBARI-13832
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1)Deploy old version 2.1.1
> 2) Add host, ZK server at added host (Optional step)
> 3)Enable NN, RM HA (Optional step)
> 3)Make Ambari only upgrade to 2.1.3.0
> 4)Enable security (MIT)
>
> Actual results:
> HBase Master Stop is Failed when Use 'Stop ALL'
> **But if I stop HBASE as separate operation, everything works.**
>
>
>
> Traceback (most recent call last):
> File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py", line 149, in <module>
> HbaseMaster().execute()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py", line 90, in stop
> hbase_service('master', action = 'stop')
> File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py", line 50, in hbase_service
> on_timeout = format("! ( {no_op_test} ) || {sudo} -H -E kill -9 `cat {pid_file}`"),
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 156, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 119, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 145, in _call_wrapper
> result = checked_call(on_timeout)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '! ( ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1 ) || ambari-sudo.sh -H -E kill -9 `cat /var/run/hbase/hbase-hbase-master.pid`' returned 1. cat: /var/run/hbase/hbase-hbase-master.pid: Permission denied
> usage: kill [ -s signal | -p ] [ -a ] pid ...
> kill -l [ signal ]
> stdout: /var/lib/ambari-agent/data/output-2285.txt
>
> 2015-11-09 14:09:44,480 - Execute['/usr/hdp/current/hbase-master/bin/hbase-daemon.sh --config /usr/hdp/current/hbase-master/conf stop master'] {'only_if': 'ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1', 'on_timeout': '! ( ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1 ) || ambari-sudo.sh -H -E kill -9 `cat /var/run/hbase/hbase-hbase-master.pid`', 'timeout': 30, 'user': 'hbase'}
> 2015-11-09 14:10:15,772 - Executing '! ( ambari-sudo.sh -H -E test -f /var/run/hbase/hbase-hbase-master.pid && ps -p `ambari-sudo.sh -H -E cat /var/run/hbase/hbase-hbase-master.pid` >/dev/null 2>&1 ) || ambari-sudo.sh -H -E kill -9 `cat /var/run/hbase/hbase-hbase-master.pid`'. Reason: Execution of 'ambari-sudo.sh su hbase -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/agslava/bin:/var/lib/ambari-agent'"'"' ; /usr/hdp/current/hbase-master/bin/hbase-daemon.sh --config /usr/hdp/current/hbase-master/conf stop master'' was killed due timeout after 30 seconds
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py f322aa4
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 3206e1c
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 84e5753
>
> Diff: https://reviews.apache.org/r/40183/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>