You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by lei liu <li...@gmail.com> on 2012/10/24 05:56:34 UTC
AvatarNode configuration
I use facebook AvatarNode, my configuration is below:
<property>
<name>dfs.name.dir.shared0</name>
<value>/data6/hdfs_ha1/ha_share1/share0/namenode</value>
</property>
<property>
<name>dfs.name.edits.dir.shared0</name>
<value>/data6/hdfs_ha1/ha_share1/share0/editlog</value>
</property>
<property>
<name>dfs.name.dir.shared1</name>
<value>/data6/hdfs_ha1/ha_share1/share1/namenode</value>
</property>
<property>
<name>dfs.name.edits.dir.shared1</name>
<value>/data6/hdfs_ha1/ha_share1/share1/editlog</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/musa.ll/hdfs_ha/local/name</value>
</property>
<property>
<name>dfs.name.edits.dir</name>
<value>/home/musa.ll/hdfs_ha/local/name</value>
</property>
The local name dir and edits dir are same.
When I start standby namenode, the standby namenode throw below exception:
Ingest: Exception while processing transactions (true)
java.io.IOException: Ingest: Timestamp of transaction log on localmachine
is 2012-10-24 11:41:28:271 and on remote namenode is 2012-10-24
11:41:28:146. But this can never happen.
at
org.apache.hadoop.hdfs.server.namenode.Ingest.ingestFSEdits(Ingest.java:316)
at
org.apache.hadoop.hdfs.server.namenode.Ingest.loadFSEdits(Ingest.java:216)
at org.apache.hadoop.hdfs.server.namenode.Ingest.run(Ingest.java:86)
at java.lang.Thread.run(Unknown Source)
If I config different local name dir and edits dir, thta is ok. Must be
the local name dir and edits dir different?
Thanks,
LiuLei
Re: AvatarNode configuration
Posted by Harsh J <ha...@cloudera.com>.
Hey LiuLei,
AvatarNode isn't part of the Apache Hadoop project yet - you may find
better responses by communicating with the facebook/hadoop repository
maintainers directly. However, you may be interested in the HA
solution carried by Apache Hadoop itself, documented at
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html.
On Wed, Oct 24, 2012 at 9:26 AM, lei liu <li...@gmail.com> wrote:
> I use facebook AvatarNode, my configuration is below:
>
> <property>
> <name>dfs.name.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
> <property>
> <name>dfs.name.edits.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
>
> The local name dir and edits dir are same.
>
> When I start standby namenode, the standby namenode throw below exception:
> Ingest: Exception while processing transactions (true) java.io.IOException:
> Ingest: Timestamp of transaction log on localmachine is 2012-10-24
> 11:41:28:271 and on remote namenode is 2012-10-24 11:41:28:146. But this can
> never happen.
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.ingestFSEdits(Ingest.java:316)
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.loadFSEdits(Ingest.java:216)
> at org.apache.hadoop.hdfs.server.namenode.Ingest.run(Ingest.java:86)
> at java.lang.Thread.run(Unknown Source)
>
> If I config different local name dir and edits dir, thta is ok. Must be the
> local name dir and edits dir different?
>
> Thanks,
> LiuLei
>
>
>
>
--
Harsh J
Re: AvatarNode configuration
Posted by Harsh J <ha...@cloudera.com>.
Hey LiuLei,
AvatarNode isn't part of the Apache Hadoop project yet - you may find
better responses by communicating with the facebook/hadoop repository
maintainers directly. However, you may be interested in the HA
solution carried by Apache Hadoop itself, documented at
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html.
On Wed, Oct 24, 2012 at 9:26 AM, lei liu <li...@gmail.com> wrote:
> I use facebook AvatarNode, my configuration is below:
>
> <property>
> <name>dfs.name.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
> <property>
> <name>dfs.name.edits.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
>
> The local name dir and edits dir are same.
>
> When I start standby namenode, the standby namenode throw below exception:
> Ingest: Exception while processing transactions (true) java.io.IOException:
> Ingest: Timestamp of transaction log on localmachine is 2012-10-24
> 11:41:28:271 and on remote namenode is 2012-10-24 11:41:28:146. But this can
> never happen.
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.ingestFSEdits(Ingest.java:316)
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.loadFSEdits(Ingest.java:216)
> at org.apache.hadoop.hdfs.server.namenode.Ingest.run(Ingest.java:86)
> at java.lang.Thread.run(Unknown Source)
>
> If I config different local name dir and edits dir, thta is ok. Must be the
> local name dir and edits dir different?
>
> Thanks,
> LiuLei
>
>
>
>
--
Harsh J
Re: AvatarNode configuration
Posted by Harsh J <ha...@cloudera.com>.
Hey LiuLei,
AvatarNode isn't part of the Apache Hadoop project yet - you may find
better responses by communicating with the facebook/hadoop repository
maintainers directly. However, you may be interested in the HA
solution carried by Apache Hadoop itself, documented at
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html.
On Wed, Oct 24, 2012 at 9:26 AM, lei liu <li...@gmail.com> wrote:
> I use facebook AvatarNode, my configuration is below:
>
> <property>
> <name>dfs.name.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
> <property>
> <name>dfs.name.edits.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
>
> The local name dir and edits dir are same.
>
> When I start standby namenode, the standby namenode throw below exception:
> Ingest: Exception while processing transactions (true) java.io.IOException:
> Ingest: Timestamp of transaction log on localmachine is 2012-10-24
> 11:41:28:271 and on remote namenode is 2012-10-24 11:41:28:146. But this can
> never happen.
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.ingestFSEdits(Ingest.java:316)
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.loadFSEdits(Ingest.java:216)
> at org.apache.hadoop.hdfs.server.namenode.Ingest.run(Ingest.java:86)
> at java.lang.Thread.run(Unknown Source)
>
> If I config different local name dir and edits dir, thta is ok. Must be the
> local name dir and edits dir different?
>
> Thanks,
> LiuLei
>
>
>
>
--
Harsh J
Re: AvatarNode configuration
Posted by Harsh J <ha...@cloudera.com>.
Hey LiuLei,
AvatarNode isn't part of the Apache Hadoop project yet - you may find
better responses by communicating with the facebook/hadoop repository
maintainers directly. However, you may be interested in the HA
solution carried by Apache Hadoop itself, documented at
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailability.html.
On Wed, Oct 24, 2012 at 9:26 AM, lei liu <li...@gmail.com> wrote:
> I use facebook AvatarNode, my configuration is below:
>
> <property>
> <name>dfs.name.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared0</name>
> <value>/data6/hdfs_ha1/ha_share1/share0/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/namenode</value>
> </property>
> <property>
> <name>dfs.name.edits.dir.shared1</name>
> <value>/data6/hdfs_ha1/ha_share1/share1/editlog</value>
> </property>
>
> <property>
> <name>dfs.name.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
> <property>
> <name>dfs.name.edits.dir</name>
> <value>/home/musa.ll/hdfs_ha/local/name</value>
> </property>
>
> The local name dir and edits dir are same.
>
> When I start standby namenode, the standby namenode throw below exception:
> Ingest: Exception while processing transactions (true) java.io.IOException:
> Ingest: Timestamp of transaction log on localmachine is 2012-10-24
> 11:41:28:271 and on remote namenode is 2012-10-24 11:41:28:146. But this can
> never happen.
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.ingestFSEdits(Ingest.java:316)
> at
> org.apache.hadoop.hdfs.server.namenode.Ingest.loadFSEdits(Ingest.java:216)
> at org.apache.hadoop.hdfs.server.namenode.Ingest.run(Ingest.java:86)
> at java.lang.Thread.run(Unknown Source)
>
> If I config different local name dir and edits dir, thta is ok. Must be the
> local name dir and edits dir different?
>
> Thanks,
> LiuLei
>
>
>
>
--
Harsh J