You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Jayush Luniya (JIRA)" <ji...@apache.org> on 2015/12/10 02:09:11 UTC
[jira] [Commented] (AMBARI-14312) Express Upgrade: Failure during
restart of all Node Managers during EU from 2.1 to 2.3.4 with timeout error
[ https://issues.apache.org/jira/browse/AMBARI-14312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15049783#comment-15049783 ]
Jayush Luniya commented on AMBARI-14312:
----------------------------------------
Actual issue is that ResourceManager fails to start. because yarn configs on upgrade are settting yarn.resourcemanager.zk-address and hadoop.registry.zk.quorum to localhost and the host where ResourceManager is present doesnt have ZK server. If RM and ZK servers are colocated this issue would be masked.
{code}
yarn.resourcemanager.zk-address=localhost:2181
hadoop.registry.zk.quorum=localhost:2181
{code}
> Express Upgrade: Failure during restart of all Node Managers during EU from 2.1 to 2.3.4 with timeout error
> -----------------------------------------------------------------------------------------------------------
>
> Key: AMBARI-14312
> URL: https://issues.apache.org/jira/browse/AMBARI-14312
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.2.0
> Reporter: Jayush Luniya
> Assignee: Jayush Luniya
> Priority: Blocker
> Fix For: 2.2.0
>
>
> *Steps:*
> Setup HDP 2.1 cluster with Ambari 2.2.0
> Modify the relevant DB tables
> Start Express Upgrade to 2.3.4
> Result:
> EU reported failures during restart of all NameNodes
> *stderr*
> {code}
> Python script has been killed due to timeout after waiting 1200 secs
> {code}
> *stdout*
> {code}
> 2015-12-09 10:58:32,783 - File['/var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid'] {'action': ['delete'], 'not_if': "ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'"}
> 2015-12-09 10:58:32,844 - Execute['ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/2.3.4.0-3426/hadoop/libexec && /usr/hdp/current/hadoop-yarn-nodemanager/sbin/yarn-daemon.sh --config /usr/hdp/2.3.4.0-3426/hadoop/conf start nodemanager'] {'not_if': "ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'", 'user': 'yarn'}
> 2015-12-09 10:58:34,031 - Execute['ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`''] {'not_if': "ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`'", 'tries': 5, 'try_sleep': 1}
> 2015-12-09 10:58:34,081 - Skipping Execute['ambari-sudo.sh su yarn -l -s /bin/bash -c 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-nodemanager.pid`''] due to not_if
> 2015-12-09 10:58:34,081 - Executing NodeManager Stack Upgrade post-restart
> 2015-12-09 10:58:34,083 - NodeManager executing "yarn node -list -states=RUNNING" to verify the node has rejoined the cluster...
> 2015-12-09 10:58:34,083 - call['yarn node -list -states=RUNNING'] {'user': 'yarn'}
> {code}
> Attachments: stdout and screenshots
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)