You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by long thai <th...@gmail.com> on 2010/04/11 12:05:45 UTC
"Too many fetch-failures" ERROR
Hi all.
The error I encounter is so common, however, after 2 weeks of searching and
following all solution, I still stuck at it. So, I hope that anyone can help
me to overcome this issue :)
First, I use Ubuntu 9.04 x86_64 and hadoop-0.20.2. I successfully setup for
single node based on instruction of Michael G. Noll.
Second, I setup Hadoop for multi nodes, following Noll's instruction, and
encounter the error. This is my config files
/etc/hosts
127.0.0.1 localhost
127.0.1.1 thailong-desktop
#192.168.1.2 localhost
#192.168.1.2 thailong-desktop
# The following lines are desirable for IPv6 capable hosts
#::1 localhost ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
#ff02::3 ip6-allhosts
192.168.1.4 node1
192.168.1.2 master
core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop-datastore/hadoop-${user.name}</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://master:54310</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri's scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri's authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>
</configuration>
mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- In: conf/mapred-site.xml -->
<property>
<name>mapred.job.tracker</name>
<value>master:54311</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>
</configuration>
hdfs.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- In: conf/hdfs-site.xml -->
<property>
<name>dfs.replication</name>
<value>1</value>
<description>Default block replication.
The actual number of replications can be specified when the file is
created.
The default is used if replication is not specified in create time.
</description>
</property>
</configuration>
I try to setup the Hadoop on single node again, but at this time, instead of
using localhost, I set all value to master, which is the host name of the
local machine, and the error is still there. It seems that there is a
problem in mapred-site.xml, if I change mapred.job.tracker to localhost, or
change the IP address of master in /etc/hosts to 127.0.1.1, the system runs
with error. is there something that I missed?
This problem has haunted me for weeks, any help from you is precious to me.
Regards
Re: "Too many fetch-failures" ERROR
Posted by long thai <th...@gmail.com>.
Hi.
For single node installation using localhost in cofig files, Hadoop run very
well. However, If I change localhost to the hostname which is assigned to
local machine in /etc/hosts file, in my case it is master, I receive "Too
many fetch-failure" error. I think there is a problem with transferring data
to mapred process. Am I right? Is there any way to solve it?
Regards.
On Mon, Apr 12, 2010 at 2:40 AM, Raghava Mutharaju <
m.vijayaraghava@gmail.com> wrote:
> Hi,
>
> I followed Michael G. Noll's blog post to set up a single node
> installation on my laptop. Sometimes I did encounter this error. I just
> used
> to restart hadoop and that used to fix it. But I don't know the exact
> reason
> behind this.
>
> Regards,
> Raghava.
>
> On Sun, Apr 11, 2010 at 6:05 AM, long thai <thaithanhlong2501@gmail.com
> >wrote:
>
> > Hi all.
> >
> > The error I encounter is so common, however, after 2 weeks of searching
> and
> > following all solution, I still stuck at it. So, I hope that anyone can
> > help
> > me to overcome this issue :)
> >
> > First, I use Ubuntu 9.04 x86_64 and hadoop-0.20.2. I successfully setup
> for
> > single node based on instruction of Michael G. Noll.
> >
> > Second, I setup Hadoop for multi nodes, following Noll's instruction, and
> > encounter the error. This is my config files
> >
> > /etc/hosts
> > 127.0.0.1 localhost
> > 127.0.1.1 thailong-desktop
> > #192.168.1.2 localhost
> > #192.168.1.2 thailong-desktop
> >
> > # The following lines are desirable for IPv6 capable hosts
> > #::1 localhost ip6-localhost ip6-loopback
> > #fe00::0 ip6-localnet
> > #ff00::0 ip6-mcastprefix
> > #ff02::1 ip6-allnodes
> > #ff02::2 ip6-allrouters
> > #ff02::3 ip6-allhosts
> > 192.168.1.4 node1
> > 192.168.1.2 master
> >
> > core-site.xml
> > <?xml version="1.0"?>
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > <!-- Put site-specific property overrides in this file. -->
> > <configuration>
> > <property>
> > <name>hadoop.tmp.dir</name>
> > <value>/usr/local/hadoop-datastore/hadoop-${user.name}</value>
> > <description>A base for other temporary directories.</description>
> > </property>
> > <property>
> > <name>fs.default.name</name>
> > <value>hdfs://master:54310</value>
> > <description>The name of the default file system. A URI whose
> > scheme and authority determine the FileSystem implementation. The
> > uri's scheme determines the config property (fs.SCHEME.impl) naming
> > the FileSystem implementation class. The uri's authority is used to
> > determine the host, port, etc. for a filesystem.</description>
> > </property>
> > </configuration>
> >
> > mapred-site.xml
> > <?xml version="1.0"?>
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > <!-- Put site-specific property overrides in this file. -->
> > <configuration>
> > <!-- In: conf/mapred-site.xml -->
> > <property>
> > <name>mapred.job.tracker</name>
> > <value>master:54311</value>
> > <description>The host and port that the MapReduce job tracker runs
> > at. If "local", then jobs are run in-process as a single map
> > and reduce task.
> > </description>
> > </property>
> > </configuration>
> >
> > hdfs.xml
> > <?xml version="1.0"?>
> > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > <!-- Put site-specific property overrides in this file. -->
> > <configuration>
> > <!-- In: conf/hdfs-site.xml -->
> > <property>
> > <name>dfs.replication</name>
> > <value>1</value>
> > <description>Default block replication.
> > The actual number of replications can be specified when the file is
> > created.
> > The default is used if replication is not specified in create time.
> > </description>
> > </property>
> > </configuration>
> >
> > I try to setup the Hadoop on single node again, but at this time, instead
> > of
> > using localhost, I set all value to master, which is the host name of the
> > local machine, and the error is still there. It seems that there is a
> > problem in mapred-site.xml, if I change mapred.job.tracker to localhost,
> or
> > change the IP address of master in /etc/hosts to 127.0.1.1, the system
> runs
> > with error. is there something that I missed?
> >
> > This problem has haunted me for weeks, any help from you is precious to
> me.
> >
> > Regards
> >
>
Re: "Too many fetch-failures" ERROR
Posted by Raghava Mutharaju <m....@gmail.com>.
Hi,
I followed Michael G. Noll's blog post to set up a single node
installation on my laptop. Sometimes I did encounter this error. I just used
to restart hadoop and that used to fix it. But I don't know the exact reason
behind this.
Regards,
Raghava.
On Sun, Apr 11, 2010 at 6:05 AM, long thai <th...@gmail.com>wrote:
> Hi all.
>
> The error I encounter is so common, however, after 2 weeks of searching and
> following all solution, I still stuck at it. So, I hope that anyone can
> help
> me to overcome this issue :)
>
> First, I use Ubuntu 9.04 x86_64 and hadoop-0.20.2. I successfully setup for
> single node based on instruction of Michael G. Noll.
>
> Second, I setup Hadoop for multi nodes, following Noll's instruction, and
> encounter the error. This is my config files
>
> /etc/hosts
> 127.0.0.1 localhost
> 127.0.1.1 thailong-desktop
> #192.168.1.2 localhost
> #192.168.1.2 thailong-desktop
>
> # The following lines are desirable for IPv6 capable hosts
> #::1 localhost ip6-localhost ip6-loopback
> #fe00::0 ip6-localnet
> #ff00::0 ip6-mcastprefix
> #ff02::1 ip6-allnodes
> #ff02::2 ip6-allrouters
> #ff02::3 ip6-allhosts
> 192.168.1.4 node1
> 192.168.1.2 master
>
> core-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/usr/local/hadoop-datastore/hadoop-${user.name}</value>
> <description>A base for other temporary directories.</description>
> </property>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://master:54310</value>
> <description>The name of the default file system. A URI whose
> scheme and authority determine the FileSystem implementation. The
> uri's scheme determines the config property (fs.SCHEME.impl) naming
> the FileSystem implementation class. The uri's authority is used to
> determine the host, port, etc. for a filesystem.</description>
> </property>
> </configuration>
>
> mapred-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
> <!-- In: conf/mapred-site.xml -->
> <property>
> <name>mapred.job.tracker</name>
> <value>master:54311</value>
> <description>The host and port that the MapReduce job tracker runs
> at. If "local", then jobs are run in-process as a single map
> and reduce task.
> </description>
> </property>
> </configuration>
>
> hdfs.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
> <!-- In: conf/hdfs-site.xml -->
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> <description>Default block replication.
> The actual number of replications can be specified when the file is
> created.
> The default is used if replication is not specified in create time.
> </description>
> </property>
> </configuration>
>
> I try to setup the Hadoop on single node again, but at this time, instead
> of
> using localhost, I set all value to master, which is the host name of the
> local machine, and the error is still there. It seems that there is a
> problem in mapred-site.xml, if I change mapred.job.tracker to localhost, or
> change the IP address of master in /etc/hosts to 127.0.1.1, the system runs
> with error. is there something that I missed?
>
> This problem has haunted me for weeks, any help from you is precious to me.
>
> Regards
>