You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Vincent.He (JIRA)" <ji...@apache.org> on 2015/08/03 15:15:05 UTC
[jira] [Commented] (AMBARI-12374) Unable to Start NameNode in HA
Mode On HDP 2.0
[ https://issues.apache.org/jira/browse/AMBARI-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651845#comment-14651845 ]
Vincent.He commented on AMBARI-12374:
-------------------------------------
I meet the same issues with Ambari 2.1 and HDP2.3, is there a patch or workaround on this? it is a blocking issue for me.
> Unable to Start NameNode in HA Mode On HDP 2.0
> ----------------------------------------------
>
> Key: AMBARI-12374
> URL: https://issues.apache.org/jira/browse/AMBARI-12374
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Reporter: Dmitry Lysnichenko
> Assignee: Dmitry Lysnichenko
> Fix For: 2.1.1
>
> Attachments: AMBARI-12374.patch
>
>
> When starting an HA NameNode cluster on HDP 2.0, the following error is seen:
> {code}
> 2015-07-07 16:02:56,371 - Getting jmx metrics from NN failed. URL: http://c6401.ambari.apache.org:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py", line 41, in get_value_from_jmx
> return data_dict["beans"][0][property]
> IndexError: list index out of range
> 2015-07-07 16:02:56,396 - Getting jmx metrics from NN failed. URL: http://c6402.ambari.apache.org:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py", line 41, in get_value_from_jmx
> return data_dict["beans"][0][property]
> IndexError: list index out of range
> Traceback (most recent call last):
> File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py", line 316, in <module>
> NameNode().execute()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 216, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py", line 81, in start
> namenode(action="start", rolling_restart=rolling_restart, env=env)
> File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
> return fn(*args, **kwargs)
> File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py", line 141, in namenode
> create_hdfs_directories(is_active_namenode_cmd)
> File "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py", line 198, in create_hdfs_directories
> only_if=check
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 390, in action_create_on_execute
> self.action_delayed("create")
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 387, in action_delayed
> self.get_hdfs_resource_executor().action_delayed(action_name, self)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 239, in action_delayed
> main_resource.resource.security_enabled, main_resource.resource.logoutput)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 126, in __init__
> security_enabled, run_user)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/namenode_ha_utils.py", line 113, in get_property_for_active_namenode
> raise Fail("There is no active namenodes.")
> resource_management.core.exceptions.Fail: There is no active namenodes.
> {code}
> Although the NameNode does actually start, a failure is recorded in the request, stopping the rest of the cluster from coming up. This probably because the JMX properties for Active and Standby NameNode are different in HDP 2.0 vs HDP 2.1+:
> {code:title=active jmx}
> {
> "name" : "Hadoop:service=NameNode,name=FSNamesystem",
> "modelerType" : "FSNamesystem",
> "tag.Context" : "dfs",
> "tag.HAState" : "active",
> {code}
> {code:title=standby jmx}
> {
> "name" : "Hadoop:service=NameNode,name=FSNamesystem",
> "modelerType" : "FSNamesystem",
> "tag.Context" : "dfs",
> "tag.HAState" : "standby",
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)