You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2014/09/22 15:10:21 UTC
Review Request 25892: HDP-2.2.1 install results in Nimbus component
failing to start
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25892/
-----------------------------------------------------------
Review request for Ambari and Vitalyi Brodetskyi.
Bugs: AMBARI-7431
https://issues.apache.org/jira/browse/AMBARI-7431
Repository: ambari
Description
-------
Installed HDP-2.2.1 from latest build, and Nimbus component failed to start.
Folder `/var/run/storm` was there but had no children.
stderr:
2014-09-18 19:26:31,460 - Error while executing command 'start':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/nimbus.py", line 43, in start
service("nimbus", action="start")
File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/service.py", line 64, in service
try_sleep=10
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
raise ex
Fail: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
stdout:
2014-09-18 19:25:40,652 - Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://hill-1.c.pramod-thangali.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] {'environment': ..., 'not_if': 'test -e /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip', 'ignore_failures': True, 'path': ['/bin', '/usr/bin/']}
2014-09-18 19:25:40,681 - Skipping Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://hill-1.c.pramod-thangali.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] due to not_if
2014-09-18 19:25:40,838 - Directory['/etc/hadoop/conf.empty'] {'owner': 'root', 'group': 'root', 'recursive': True}
2014-09-18 19:25:40,840 - Link['/etc/hadoop/conf'] {'not_if': 'ls /etc/hadoop/conf', 'to': '/etc/hadoop/conf.empty'}
2014-09-18 19:25:40,852 - Skipping Link['/etc/hadoop/conf'] due to not_if
2014-09-18 19:25:40,865 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs'}
2014-09-18 19:25:40,866 - XmlConfig['core-site.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': ..., 'configurations': ...}
2014-09-18 19:25:40,875 - Generating config: /etc/hadoop/conf/core-site.xml
2014-09-18 19:25:40,877 - File['/etc/hadoop/conf/core-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2014-09-18 19:25:40,878 - Writing File['/etc/hadoop/conf/core-site.xml'] because contents don't match
2014-09-18 19:25:40,890 - Execute['/bin/echo 0 > /selinux/enforce'] {'only_if': 'test -f /selinux/enforce'}
2014-09-18 19:25:40,916 - Execute['mkdir -p /usr/lib/hadoop/lib/native/Linux-i386-32; ln -sf /usr/lib/libsnappy.so /usr/lib/hadoop/lib/native/Linux-i386-32/libsnappy.so'] {}
2014-09-18 19:25:40,930 - Execute['mkdir -p /usr/lib/hadoop/lib/native/Linux-amd64-64; ln -sf /usr/lib64/libsnappy.so /usr/lib/hadoop/lib/native/Linux-amd64-64/libsnappy.so'] {}
2014-09-18 19:25:40,942 - Directory['/var/log/hadoop'] {'owner': 'root', 'group': 'hadoop', 'mode': 0775, 'recursive': True}
2014-09-18 19:25:40,943 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True}
2014-09-18 19:25:40,943 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True}
2014-09-18 19:25:40,948 - File['/etc/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2014-09-18 19:25:40,950 - File['/etc/hadoop/conf/health_check'] {'content': Template('health_check-v2.j2'), 'owner': 'hdfs'}
2014-09-18 19:25:40,951 - File['/etc/hadoop/conf/log4j.properties'] {'content': '...', 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2014-09-18 19:25:40,955 - File['/etc/hadoop/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2014-09-18 19:25:40,956 - File['/etc/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2014-09-18 19:25:40,956 - File['/etc/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2014-09-18 19:25:41,085 - Directory['/var/log/storm'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
2014-09-18 19:25:41,086 - Directory['/var/run/storm'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
2014-09-18 19:25:41,087 - Directory['/hadoop/storm'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
2014-09-18 19:25:41,087 - Directory['/etc/storm/conf'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
2014-09-18 19:25:41,093 - File['/etc/storm/conf/config.yaml'] {'owner': 'storm', 'content': Template('config.yaml.j2'), 'group': 'hadoop'}
2014-09-18 19:25:41,097 - File['/etc/storm/conf/storm.yaml'] {'owner': 'storm', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None}
2014-09-18 19:25:41,100 - File['/etc/storm/conf/storm-env.sh'] {'content': InlineTemplate(...), 'owner': 'storm'}
2014-09-18 19:25:41,101 - Execute['env JAVA_HOME=/usr/jdk64/jdk1.7.0_45 PATH=$PATH:/usr/jdk64/jdk1.7.0_45/bin /usr/bin/storm nimbus > /var/log/storm/nimbus.out 2>&1'] {'wait_for_finish': False, 'not_if': 'ls /var/run/storm/nimbus.pid >/dev/null 2>&1 && ps `cat /var/run/storm/nimbus.pid` >/dev/null 2>&1', 'user': 'storm'}
2014-09-18 19:25:41,115 - Execute['pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid'] {'logoutput': True, 'tries': 6, 'user': 'storm', 'try_sleep': 10}
2014-09-18 19:25:41,164 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
2014-09-18 19:25:51,322 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
2014-09-18 19:26:01,352 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
2014-09-18 19:26:11,383 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
2014-09-18 19:26:21,428 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
2014-09-18 19:26:31,460 - Error while executing command 'start':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/nimbus.py", line 43, in start
service("nimbus", action="start")
File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/service.py", line 64, in service
try_sleep=10
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
raise ex
Fail: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/configuration/storm-site.xml b02d23a
ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/configuration/storm-site.xml 57a4889
Diff: https://reviews.apache.org/r/25892/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 25892: HDP-2.2.1 install results in Nimbus
component failing to start
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25892/#review54124
-----------------------------------------------------------
Ship it!
Ship It!
- Vitalyi Brodetskyi
On Вер. 22, 2014, 1:10 після полудня, Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25892/
> -----------------------------------------------------------
>
> (Updated Вер. 22, 2014, 1:10 після полудня)
>
>
> Review request for Ambari and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-7431
> https://issues.apache.org/jira/browse/AMBARI-7431
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Installed HDP-2.2.1 from latest build, and Nimbus component failed to start.
> Folder `/var/run/storm` was there but had no children.
>
>
>
>
> stderr:
>
> 2014-09-18 19:26:31,460 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/nimbus.py", line 43, in start
> service("nimbus", action="start")
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/service.py", line 64, in service
> try_sleep=10
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
> raise ex
> Fail: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
> stdout:
>
> 2014-09-18 19:25:40,652 - Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://hill-1.c.pramod-thangali.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] {'environment': ..., 'not_if': 'test -e /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip', 'ignore_failures': True, 'path': ['/bin', '/usr/bin/']}
> 2014-09-18 19:25:40,681 - Skipping Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://hill-1.c.pramod-thangali.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] due to not_if
> 2014-09-18 19:25:40,838 - Directory['/etc/hadoop/conf.empty'] {'owner': 'root', 'group': 'root', 'recursive': True}
> 2014-09-18 19:25:40,840 - Link['/etc/hadoop/conf'] {'not_if': 'ls /etc/hadoop/conf', 'to': '/etc/hadoop/conf.empty'}
> 2014-09-18 19:25:40,852 - Skipping Link['/etc/hadoop/conf'] due to not_if
> 2014-09-18 19:25:40,865 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs'}
> 2014-09-18 19:25:40,866 - XmlConfig['core-site.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': ..., 'configurations': ...}
> 2014-09-18 19:25:40,875 - Generating config: /etc/hadoop/conf/core-site.xml
> 2014-09-18 19:25:40,877 - File['/etc/hadoop/conf/core-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
> 2014-09-18 19:25:40,878 - Writing File['/etc/hadoop/conf/core-site.xml'] because contents don't match
> 2014-09-18 19:25:40,890 - Execute['/bin/echo 0 > /selinux/enforce'] {'only_if': 'test -f /selinux/enforce'}
> 2014-09-18 19:25:40,916 - Execute['mkdir -p /usr/lib/hadoop/lib/native/Linux-i386-32; ln -sf /usr/lib/libsnappy.so /usr/lib/hadoop/lib/native/Linux-i386-32/libsnappy.so'] {}
> 2014-09-18 19:25:40,930 - Execute['mkdir -p /usr/lib/hadoop/lib/native/Linux-amd64-64; ln -sf /usr/lib64/libsnappy.so /usr/lib/hadoop/lib/native/Linux-amd64-64/libsnappy.so'] {}
> 2014-09-18 19:25:40,942 - Directory['/var/log/hadoop'] {'owner': 'root', 'group': 'hadoop', 'mode': 0775, 'recursive': True}
> 2014-09-18 19:25:40,943 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True}
> 2014-09-18 19:25:40,943 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True}
> 2014-09-18 19:25:40,948 - File['/etc/hadoop/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
> 2014-09-18 19:25:40,950 - File['/etc/hadoop/conf/health_check'] {'content': Template('health_check-v2.j2'), 'owner': 'hdfs'}
> 2014-09-18 19:25:40,951 - File['/etc/hadoop/conf/log4j.properties'] {'content': '...', 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
> 2014-09-18 19:25:40,955 - File['/etc/hadoop/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
> 2014-09-18 19:25:40,956 - File['/etc/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
> 2014-09-18 19:25:40,956 - File['/etc/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
> 2014-09-18 19:25:41,085 - Directory['/var/log/storm'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
> 2014-09-18 19:25:41,086 - Directory['/var/run/storm'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
> 2014-09-18 19:25:41,087 - Directory['/hadoop/storm'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
> 2014-09-18 19:25:41,087 - Directory['/etc/storm/conf'] {'owner': 'storm', 'group': 'hadoop', 'recursive': True}
> 2014-09-18 19:25:41,093 - File['/etc/storm/conf/config.yaml'] {'owner': 'storm', 'content': Template('config.yaml.j2'), 'group': 'hadoop'}
> 2014-09-18 19:25:41,097 - File['/etc/storm/conf/storm.yaml'] {'owner': 'storm', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None}
> 2014-09-18 19:25:41,100 - File['/etc/storm/conf/storm-env.sh'] {'content': InlineTemplate(...), 'owner': 'storm'}
> 2014-09-18 19:25:41,101 - Execute['env JAVA_HOME=/usr/jdk64/jdk1.7.0_45 PATH=$PATH:/usr/jdk64/jdk1.7.0_45/bin /usr/bin/storm nimbus > /var/log/storm/nimbus.out 2>&1'] {'wait_for_finish': False, 'not_if': 'ls /var/run/storm/nimbus.pid >/dev/null 2>&1 && ps `cat /var/run/storm/nimbus.pid` >/dev/null 2>&1', 'user': 'storm'}
> 2014-09-18 19:25:41,115 - Execute['pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid'] {'logoutput': True, 'tries': 6, 'user': 'storm', 'try_sleep': 10}
> 2014-09-18 19:25:41,164 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
> 2014-09-18 19:25:51,322 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
> 2014-09-18 19:26:01,352 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
> 2014-09-18 19:26:11,383 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
> 2014-09-18 19:26:21,428 - Retrying after 10 seconds. Reason: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
> 2014-09-18 19:26:31,460 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/nimbus.py", line 43, in start
> service("nimbus", action="start")
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/service.py", line 64, in service
> try_sleep=10
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
> raise ex
> Fail: Execution of 'pgrep -f "^java.+backtype.storm.daemon.nimbus$" && pgrep -f "^java.+backtype.storm.daemon.nimbus$" > /var/run/storm/nimbus.pid' returned 1.
>
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/configuration/storm-site.xml b02d23a
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/configuration/storm-site.xml 57a4889
>
> Diff: https://reviews.apache.org/r/25892/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>