You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by F21 <f2...@gmail.com> on 2015/12/10 05:07:50 UTC
How do I run ZKFC on a different node that is not the name node?
I am receiving this error when trying to format zookeeper using the zkfc:
Exception in thread "main"
org.apache.hadoop.HadoopIllegalArgumentException: Could not get the
namenode ID of this node. You may run zkfc on the node other than namenode.
at
org.apache.hadoop.hdfs.tools.DFSZKFailoverController.create(DFSZKFailoverController.java:137)
at
org.apache.hadoop.hdfs.tools.DFSZKFailoverController.main(DFSZKFailoverController.java:186)
There wasn't any information regarding this on the web. I am running the
namenode, journalnode, datanode and zkfc within separate docker containers.
This is the configuration in the zkfc container (I have launched 3 of
these):
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>dfs.permissions.superusergroup</name>
<value>hadoop</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.cluster</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>shell(/bin/true)</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>zookeeper:2181</value>
</property>
<property>
<name>dfs.nameservices</name>
<value>cluster</value>
</property>
<property>
<name>dfs.ha.namenodescluster</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.cluster.nn1</name>
<value>namenode1:8020</value>
</property>
<property>
<name>dfs.namenode.rpc-address.cluster.nn2</name>
<value>namenode2:8020</value>
</property>
<property>
<name>dfs.namenode.http-address.cluster.nn1</name>
<value>namenode1:50070</value>
</property>
<property>
<name>dfs.namenode.http-address.cluster.nn2</name>
<value>namenode2:50070</value>
</property>
</configuration>
Any ideas how I can get this working?
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: How do I run ZKFC on a different node that is not the name
node?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello,
I can see 2 potential problems.
1. I see property "dfs.ha.namenodescluster" below. If that is really what
you have in your hdfs-site.xml, and not just a copy-paste error, then it
looks like there is a missing period. "dfs.ha.namenodes.cluster" would be
the correct property name.
2. Standard deployment is to run ZKFC on the same host as the NameNode
that it watches. This eliminates a potential failure mode of network
partition between the ZKFC running on one host and the NameNode running on
another host. The ZKFC startup logic tries to pair up automatically with
the correct NameNode by scanning the configured NameNode addresses to see
which one is a network address on the same host. If you aren't running
the ZKFC on the same host as the NameNode, then this automatic detection
logic won't work and it will abort. If you can't arrange to run the ZKFC
on the same host as the NameNode, then it's possible to override this
automatic detection by explicitly pointing the ZKFC to one of the
NameNodes in hdfs-site.xml. You would need to set property
dfs.ha.namenode.id to either nn1 or nn2.
<property>
<name>dfs.ha.namenode.id</name>
<value></value>
<description>
The ID of this namenode. If the namenode ID is not configured it
is determined automatically by matching the local node's address
with the configured address.
</description>
</property>
I hope this helps.
--Chris Nauroth
On 12/9/15, 8:07 PM, "F21" <f2...@gmail.com> wrote:
>I am receiving this error when trying to format zookeeper using the zkfc:
>
>Exception in thread "main"
>org.apache.hadoop.HadoopIllegalArgumentException: Could not get the
>namenode ID of this node. You may run zkfc on the node other than
>namenode.
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.create(DFSZKFailoverC
>ontroller.java:137)
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.main(DFSZKFailoverCon
>troller.java:186)
>
>There wasn't any information regarding this on the web. I am running the
>namenode, journalnode, datanode and zkfc within separate docker
>containers.
>
>This is the configuration in the zkfc container (I have launched 3 of
>these):
>
><?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
><configuration>
> <property>
> <name>dfs.permissions.superusergroup</name>
> <value>hadoop</value>
> </property>
> <property>
><name>dfs.client.failover.proxy.provider.cluster</name>
><value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyPr
>ovider</value>
> </property>
> <property>
> <name>dfs.ha.fencing.methods</name>
> <value>shell(/bin/true)</value>
> </property>
> <property>
> <name>ha.zookeeper.quorum</name>
> <value>zookeeper:2181</value>
> </property>
> <property>
> <name>dfs.nameservices</name>
> <value>cluster</value>
> </property>
> <property>
> <name>dfs.ha.namenodescluster</name>
> <value>nn1,nn2</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn1</name>
> <value>namenode1:8020</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn2</name>
> <value>namenode2:8020</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn1</name>
> <value>namenode1:50070</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn2</name>
> <value>namenode2:50070</value>
> </property>
></configuration>
>
>Any ideas how I can get this working?
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>For additional commands, e-mail: user-help@hadoop.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: How do I run ZKFC on a different node that is not the name
node?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello,
I can see 2 potential problems.
1. I see property "dfs.ha.namenodescluster" below. If that is really what
you have in your hdfs-site.xml, and not just a copy-paste error, then it
looks like there is a missing period. "dfs.ha.namenodes.cluster" would be
the correct property name.
2. Standard deployment is to run ZKFC on the same host as the NameNode
that it watches. This eliminates a potential failure mode of network
partition between the ZKFC running on one host and the NameNode running on
another host. The ZKFC startup logic tries to pair up automatically with
the correct NameNode by scanning the configured NameNode addresses to see
which one is a network address on the same host. If you aren't running
the ZKFC on the same host as the NameNode, then this automatic detection
logic won't work and it will abort. If you can't arrange to run the ZKFC
on the same host as the NameNode, then it's possible to override this
automatic detection by explicitly pointing the ZKFC to one of the
NameNodes in hdfs-site.xml. You would need to set property
dfs.ha.namenode.id to either nn1 or nn2.
<property>
<name>dfs.ha.namenode.id</name>
<value></value>
<description>
The ID of this namenode. If the namenode ID is not configured it
is determined automatically by matching the local node's address
with the configured address.
</description>
</property>
I hope this helps.
--Chris Nauroth
On 12/9/15, 8:07 PM, "F21" <f2...@gmail.com> wrote:
>I am receiving this error when trying to format zookeeper using the zkfc:
>
>Exception in thread "main"
>org.apache.hadoop.HadoopIllegalArgumentException: Could not get the
>namenode ID of this node. You may run zkfc on the node other than
>namenode.
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.create(DFSZKFailoverC
>ontroller.java:137)
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.main(DFSZKFailoverCon
>troller.java:186)
>
>There wasn't any information regarding this on the web. I am running the
>namenode, journalnode, datanode and zkfc within separate docker
>containers.
>
>This is the configuration in the zkfc container (I have launched 3 of
>these):
>
><?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
><configuration>
> <property>
> <name>dfs.permissions.superusergroup</name>
> <value>hadoop</value>
> </property>
> <property>
><name>dfs.client.failover.proxy.provider.cluster</name>
><value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyPr
>ovider</value>
> </property>
> <property>
> <name>dfs.ha.fencing.methods</name>
> <value>shell(/bin/true)</value>
> </property>
> <property>
> <name>ha.zookeeper.quorum</name>
> <value>zookeeper:2181</value>
> </property>
> <property>
> <name>dfs.nameservices</name>
> <value>cluster</value>
> </property>
> <property>
> <name>dfs.ha.namenodescluster</name>
> <value>nn1,nn2</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn1</name>
> <value>namenode1:8020</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn2</name>
> <value>namenode2:8020</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn1</name>
> <value>namenode1:50070</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn2</name>
> <value>namenode2:50070</value>
> </property>
></configuration>
>
>Any ideas how I can get this working?
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>For additional commands, e-mail: user-help@hadoop.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: How do I run ZKFC on a different node that is not the name
node?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello,
I can see 2 potential problems.
1. I see property "dfs.ha.namenodescluster" below. If that is really what
you have in your hdfs-site.xml, and not just a copy-paste error, then it
looks like there is a missing period. "dfs.ha.namenodes.cluster" would be
the correct property name.
2. Standard deployment is to run ZKFC on the same host as the NameNode
that it watches. This eliminates a potential failure mode of network
partition between the ZKFC running on one host and the NameNode running on
another host. The ZKFC startup logic tries to pair up automatically with
the correct NameNode by scanning the configured NameNode addresses to see
which one is a network address on the same host. If you aren't running
the ZKFC on the same host as the NameNode, then this automatic detection
logic won't work and it will abort. If you can't arrange to run the ZKFC
on the same host as the NameNode, then it's possible to override this
automatic detection by explicitly pointing the ZKFC to one of the
NameNodes in hdfs-site.xml. You would need to set property
dfs.ha.namenode.id to either nn1 or nn2.
<property>
<name>dfs.ha.namenode.id</name>
<value></value>
<description>
The ID of this namenode. If the namenode ID is not configured it
is determined automatically by matching the local node's address
with the configured address.
</description>
</property>
I hope this helps.
--Chris Nauroth
On 12/9/15, 8:07 PM, "F21" <f2...@gmail.com> wrote:
>I am receiving this error when trying to format zookeeper using the zkfc:
>
>Exception in thread "main"
>org.apache.hadoop.HadoopIllegalArgumentException: Could not get the
>namenode ID of this node. You may run zkfc on the node other than
>namenode.
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.create(DFSZKFailoverC
>ontroller.java:137)
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.main(DFSZKFailoverCon
>troller.java:186)
>
>There wasn't any information regarding this on the web. I am running the
>namenode, journalnode, datanode and zkfc within separate docker
>containers.
>
>This is the configuration in the zkfc container (I have launched 3 of
>these):
>
><?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
><configuration>
> <property>
> <name>dfs.permissions.superusergroup</name>
> <value>hadoop</value>
> </property>
> <property>
><name>dfs.client.failover.proxy.provider.cluster</name>
><value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyPr
>ovider</value>
> </property>
> <property>
> <name>dfs.ha.fencing.methods</name>
> <value>shell(/bin/true)</value>
> </property>
> <property>
> <name>ha.zookeeper.quorum</name>
> <value>zookeeper:2181</value>
> </property>
> <property>
> <name>dfs.nameservices</name>
> <value>cluster</value>
> </property>
> <property>
> <name>dfs.ha.namenodescluster</name>
> <value>nn1,nn2</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn1</name>
> <value>namenode1:8020</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn2</name>
> <value>namenode2:8020</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn1</name>
> <value>namenode1:50070</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn2</name>
> <value>namenode2:50070</value>
> </property>
></configuration>
>
>Any ideas how I can get this working?
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>For additional commands, e-mail: user-help@hadoop.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: How do I run ZKFC on a different node that is not the name
node?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello,
I can see 2 potential problems.
1. I see property "dfs.ha.namenodescluster" below. If that is really what
you have in your hdfs-site.xml, and not just a copy-paste error, then it
looks like there is a missing period. "dfs.ha.namenodes.cluster" would be
the correct property name.
2. Standard deployment is to run ZKFC on the same host as the NameNode
that it watches. This eliminates a potential failure mode of network
partition between the ZKFC running on one host and the NameNode running on
another host. The ZKFC startup logic tries to pair up automatically with
the correct NameNode by scanning the configured NameNode addresses to see
which one is a network address on the same host. If you aren't running
the ZKFC on the same host as the NameNode, then this automatic detection
logic won't work and it will abort. If you can't arrange to run the ZKFC
on the same host as the NameNode, then it's possible to override this
automatic detection by explicitly pointing the ZKFC to one of the
NameNodes in hdfs-site.xml. You would need to set property
dfs.ha.namenode.id to either nn1 or nn2.
<property>
<name>dfs.ha.namenode.id</name>
<value></value>
<description>
The ID of this namenode. If the namenode ID is not configured it
is determined automatically by matching the local node's address
with the configured address.
</description>
</property>
I hope this helps.
--Chris Nauroth
On 12/9/15, 8:07 PM, "F21" <f2...@gmail.com> wrote:
>I am receiving this error when trying to format zookeeper using the zkfc:
>
>Exception in thread "main"
>org.apache.hadoop.HadoopIllegalArgumentException: Could not get the
>namenode ID of this node. You may run zkfc on the node other than
>namenode.
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.create(DFSZKFailoverC
>ontroller.java:137)
> at
>org.apache.hadoop.hdfs.tools.DFSZKFailoverController.main(DFSZKFailoverCon
>troller.java:186)
>
>There wasn't any information regarding this on the web. I am running the
>namenode, journalnode, datanode and zkfc within separate docker
>containers.
>
>This is the configuration in the zkfc container (I have launched 3 of
>these):
>
><?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
><configuration>
> <property>
> <name>dfs.permissions.superusergroup</name>
> <value>hadoop</value>
> </property>
> <property>
><name>dfs.client.failover.proxy.provider.cluster</name>
><value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyPr
>ovider</value>
> </property>
> <property>
> <name>dfs.ha.fencing.methods</name>
> <value>shell(/bin/true)</value>
> </property>
> <property>
> <name>ha.zookeeper.quorum</name>
> <value>zookeeper:2181</value>
> </property>
> <property>
> <name>dfs.nameservices</name>
> <value>cluster</value>
> </property>
> <property>
> <name>dfs.ha.namenodescluster</name>
> <value>nn1,nn2</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn1</name>
> <value>namenode1:8020</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.cluster.nn2</name>
> <value>namenode2:8020</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn1</name>
> <value>namenode1:50070</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.cluster.nn2</name>
> <value>namenode2:50070</value>
> </property>
></configuration>
>
>Any ideas how I can get this working?
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>For additional commands, e-mail: user-help@hadoop.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org