You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Michael Howard (JIRA)" <ji...@apache.org> on 2017/07/24 20:26:00 UTC
[jira] [Updated] (AMBARI-21567) Ambari reports GREEN zookeeper when
out of disk space prevents quorum
[ https://issues.apache.org/jira/browse/AMBARI-21567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Howard updated AMBARI-21567:
------------------------------------
Description:
* Ambari reports Zookeeper status as GREEN, even though ZK is not happy and has not reached a quorum.
* Ambari also shows YARN ResourceManager as GREEN, even though it doesn't fully start ... because it is waiting for Zookeeper
* YARN NodeManagers will try to start ... Ambari will report as GREEN and started for a few seconds, but then shows them as stopped.
* YARN NodeManagers are failing to start because ResourceManager is not listening on port 8141 ... because Zookeeper is still waiting for quorum.
* root cause is that one of the Zookeeper nodes ran out of disk space on /opt/hadoop/zookeeper volume ... which prevented a quorum from being formed
Seems like Ambari dashboard should catch the fact that Zookeeper was not in a good state.
In our case, it would have saved significant time in diagnosing the problem.
was:
* Ambari reports Zookeeper status as GREEN, even though ZK is not happy and has not reached a quorum.
* Ambari also shows YARN ResourceManager as GREEN, even though it doesn't start ... because it is waiting for Zookeeper
* YARN NodeManagers will try to start ... Ambari will report as GREEN and started for a few seconds, but then shows them as stopped.
* root cause is that one of the Zookeeper nodes ran out of disk space on /opt/hadoop/zookeeper volume ... which prevented a quorum from being formed
Seems like Ambari dashboard should catch the fact that Zookeeper was not in a good state.
In our case, it would have saved significant time in diagnosing the problem.
> Ambari reports GREEN zookeeper when out of disk space prevents quorum
> ---------------------------------------------------------------------
>
> Key: AMBARI-21567
> URL: https://issues.apache.org/jira/browse/AMBARI-21567
> Project: Ambari
> Issue Type: Bug
> Affects Versions: 2.2.0
> Environment: Ambari 2.2.0
> IBM Open Platform 4.2.0.0
> CentOS 6.8
> Reporter: Michael Howard
>
> * Ambari reports Zookeeper status as GREEN, even though ZK is not happy and has not reached a quorum.
> * Ambari also shows YARN ResourceManager as GREEN, even though it doesn't fully start ... because it is waiting for Zookeeper
> * YARN NodeManagers will try to start ... Ambari will report as GREEN and started for a few seconds, but then shows them as stopped.
> * YARN NodeManagers are failing to start because ResourceManager is not listening on port 8141 ... because Zookeeper is still waiting for quorum.
> * root cause is that one of the Zookeeper nodes ran out of disk space on /opt/hadoop/zookeeper volume ... which prevented a quorum from being formed
> Seems like Ambari dashboard should catch the fact that Zookeeper was not in a good state.
> In our case, it would have saved significant time in diagnosing the problem.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)