You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmitro Lisnichenko <dl...@hortonworks.com> on 2014/10/23 19:11:12 UTC

Review Request 27081: Namenode can not leave safe mode on hdp-2.2 stack

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27081/
-----------------------------------------------------------

Review request for Ambari, Andrew Onischuk and Myroslav Papirkovskyy.


Bugs: AMBARI-7925
    https://issues.apache.org/jira/browse/AMBARI-7925


Repository: ambari


Description
-------

Initial analysis:
First namenode start attempt failed for me. Probably, that caused NN to enter safe mode. After fixing perm gen memory size value, I've tried to start NN again. Our stack scripts seem to move NN out of safe mode on start if needed. But on HDP-2.2 stack, command for doing this is deprecated and fails with a non-zero exit code.
2014-10-21 16:33:25,973 - Error while executing command 'start':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py", line 46, in start
namenode(action="start")
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py", line 60, in namenode
only_if=dfs_check_nn_status_cmd #skip when HA not active
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in _init_
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 237, in action_run
raise ex
Fail: Execution of 'su -s /bin/bash - hdfs -c 'export PATH=$PATH:/usr/hdp/current/hadoop-client/bin ; hadoop --config /etc/hadoop/conf dfsadmin -safemode get' | grep 'Safe mode is OFF'' returned 1. DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
safemode: Call From 09f.vm/10.77.64.159 to 09f.vm:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused


Diffs
-----

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/files/checkForFormat.sh eb48858 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/files/checkForFormat.sh eb48858 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/checkForFormat.sh 82dbda1 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/files/checkForFormat.sh be8c75f 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py 31fc2f1 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py a0b07aa 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2033a53 

Diff: https://reviews.apache.org/r/27081/diff/


Testing
-------

----------------------------------------------------------------------
Ran 236 tests in 4.344s

OK
----------------------------------------------------------------------
Total run:669
Total errors:0
Total failures:0
OK


Thanks,

Dmitro Lisnichenko

Re: Review Request 27081: Namenode can not leave safe mode on hdp-2.2 stack

Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27081/#review58032
-----------------------------------------------------------

Ship it!


Ship It!

- Andrew Onischuk


On Oct. 23, 2014, 5:11 p.m., Dmitro Lisnichenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27081/
> -----------------------------------------------------------
> 
> (Updated Oct. 23, 2014, 5:11 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Myroslav Papirkovskyy.
> 
> 
> Bugs: AMBARI-7925
>     https://issues.apache.org/jira/browse/AMBARI-7925
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Initial analysis:
> First namenode start attempt failed for me. Probably, that caused NN to enter safe mode. After fixing perm gen memory size value, I've tried to start NN again. Our stack scripts seem to move NN out of safe mode on start if needed. But on HDP-2.2 stack, command for doing this is deprecated and fails with a non-zero exit code.
> 2014-10-21 16:33:25,973 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py", line 46, in start
> namenode(action="start")
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py", line 60, in namenode
> only_if=dfs_check_nn_status_cmd #skip when HA not active
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in _init_
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 237, in action_run
> raise ex
> Fail: Execution of 'su -s /bin/bash - hdfs -c 'export PATH=$PATH:/usr/hdp/current/hadoop-client/bin ; hadoop --config /etc/hadoop/conf dfsadmin -safemode get' | grep 'Safe mode is OFF'' returned 1. DEPRECATED: Use of this script to execute hdfs command is deprecated.
> Instead use the hdfs command for it.
> safemode: Call From 09f.vm/10.77.64.159 to 09f.vm:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/files/checkForFormat.sh eb48858 
>   ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/files/checkForFormat.sh eb48858 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/checkForFormat.sh 82dbda1 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/files/checkForFormat.sh be8c75f 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py 31fc2f1 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py a0b07aa 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2033a53 
> 
> Diff: https://reviews.apache.org/r/27081/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Ran 236 tests in 4.344s
> 
> OK
> ----------------------------------------------------------------------
> Total run:669
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 27081: Namenode can not leave safe mode on hdp-2.2 stack

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27081/#review58258
-----------------------------------------------------------

Ship it!


Ship It!

- Vitalyi Brodetskyi


On Жов. 24, 2014, 9:04 до полудня, Dmitro Lisnichenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27081/
> -----------------------------------------------------------
> 
> (Updated Жов. 24, 2014, 9:04 до полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-7925
>     https://issues.apache.org/jira/browse/AMBARI-7925
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Initial analysis:
> First namenode start attempt failed for me. Probably, that caused NN to enter safe mode. After fixing perm gen memory size value, I've tried to start NN again. Our stack scripts seem to move NN out of safe mode on start if needed. But on HDP-2.2 stack, command for doing this is deprecated and fails with a non-zero exit code.
> 2014-10-21 16:33:25,973 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py", line 46, in start
> namenode(action="start")
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py", line 60, in namenode
> only_if=dfs_check_nn_status_cmd #skip when HA not active
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in _init_
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 237, in action_run
> raise ex
> Fail: Execution of 'su -s /bin/bash - hdfs -c 'export PATH=$PATH:/usr/hdp/current/hadoop-client/bin ; hadoop --config /etc/hadoop/conf dfsadmin -safemode get' | grep 'Safe mode is OFF'' returned 1. DEPRECATED: Use of this script to execute hdfs command is deprecated.
> Instead use the hdfs command for it.
> safemode: Call From 09f.vm/10.77.64.159 to 09f.vm:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/files/checkForFormat.sh eb48858 
>   ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/files/checkForFormat.sh eb48858 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/checkForFormat.sh 82dbda1 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/files/checkForFormat.sh be8c75f 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py 31fc2f1 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py a0b07aa 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2033a53 
> 
> Diff: https://reviews.apache.org/r/27081/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Ran 236 tests in 4.344s
> 
> OK
> ----------------------------------------------------------------------
> Total run:669
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 27081: Namenode can not leave safe mode on hdp-2.2 stack

Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27081/
-----------------------------------------------------------

(Updated Oct. 24, 2014, 9:04 a.m.)


Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.


Bugs: AMBARI-7925
    https://issues.apache.org/jira/browse/AMBARI-7925


Repository: ambari


Description
-------

Initial analysis:
First namenode start attempt failed for me. Probably, that caused NN to enter safe mode. After fixing perm gen memory size value, I've tried to start NN again. Our stack scripts seem to move NN out of safe mode on start if needed. But on HDP-2.2 stack, command for doing this is deprecated and fails with a non-zero exit code.
2014-10-21 16:33:25,973 - Error while executing command 'start':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py", line 46, in start
namenode(action="start")
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py", line 60, in namenode
only_if=dfs_check_nn_status_cmd #skip when HA not active
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in _init_
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 237, in action_run
raise ex
Fail: Execution of 'su -s /bin/bash - hdfs -c 'export PATH=$PATH:/usr/hdp/current/hadoop-client/bin ; hadoop --config /etc/hadoop/conf dfsadmin -safemode get' | grep 'Safe mode is OFF'' returned 1. DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
safemode: Call From 09f.vm/10.77.64.159 to 09f.vm:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused


Diffs
-----

  ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/files/checkForFormat.sh eb48858 
  ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/files/checkForFormat.sh eb48858 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/checkForFormat.sh 82dbda1 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/files/checkForFormat.sh be8c75f 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_namenode.py 31fc2f1 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py a0b07aa 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2033a53 

Diff: https://reviews.apache.org/r/27081/diff/


Testing
-------

----------------------------------------------------------------------
Ran 236 tests in 4.344s

OK
----------------------------------------------------------------------
Total run:669
Total errors:0
Total failures:0
OK


Thanks,

Dmitro Lisnichenko