You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by MirrorX <mi...@gmail.com> on 2011/12/21 02:21:43 UTC
network configuration (etc/hosts) ?
dear all
i am trying for many days to get a simple hadoop cluster (with 2 nodes) to
work but i have trouble configuring the network parameters. i have properly
configured the ssh keys, and the /etc/hosts files are:
master->
127.0.0.1 localhost6.localdomain6 localhost
127.0.1.1 localhost4.localdomain4 master-pc
192.168.7.110 master
192.168.7.157 slave
slave->
127.0.1.1 localhost5.localdomain5 lab-pc
127.0.0.1 localhost3.localdomain3 localhost
192.168.7.110 master
192.168.7.157 slave
i have tried all possible combinations on the /etc/hosts files but i still
cannot make it work. i either get errors 'too many fetch failures' and by
examining the logs of the slave i see this
' INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201112210259_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at
0.03MB/s)'
or i get errors like ->
'INFO mapred.JobClient: Task Id : attempt_201112210308_0001_r_000000_0,
Status : FAILED
Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.'
i have searched into many similar posts on the web but i still cannot find
the solution. could you please help me?
when i run the same job only on the master it is completed fine, and i can
connect via ssh from every node to every node and from each node to itself,
that's why i think there is something wrong with the network configuration
thank you in advance for your help
--
View this message in context: http://old.nabble.com/network-configuration-%28etc-hosts%29---tp33013719p33013719.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
Re: network configuration (etc/hosts) ?
Posted by Joey Echeverria <jo...@cloudera.com>.
Can you run the hostname command on both servers and send their output?
-Joey
On Tue, Dec 20, 2011 at 8:21 PM, MirrorX <mi...@gmail.com> wrote:
>
> dear all
>
> i am trying for many days to get a simple hadoop cluster (with 2 nodes) to
> work but i have trouble configuring the network parameters. i have properly
> configured the ssh keys, and the /etc/hosts files are:
>
> master->
> 127.0.0.1 localhost6.localdomain6 localhost
> 127.0.1.1 localhost4.localdomain4 master-pc
> 192.168.7.110 master
> 192.168.7.157 slave
>
> slave->
> 127.0.1.1 localhost5.localdomain5 lab-pc
> 127.0.0.1 localhost3.localdomain3 localhost
> 192.168.7.110 master
> 192.168.7.157 slave
>
> i have tried all possible combinations on the /etc/hosts files but i still
> cannot make it work. i either get errors 'too many fetch failures' and by
> examining the logs of the slave i see this
> ' INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201112210259_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at
> 0.03MB/s)'
>
> or i get errors like ->
> 'INFO mapred.JobClient: Task Id : attempt_201112210308_0001_r_000000_0,
> Status : FAILED
> Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.'
>
> i have searched into many similar posts on the web but i still cannot find
> the solution. could you please help me?
>
> when i run the same job only on the master it is completed fine, and i can
> connect via ssh from every node to every node and from each node to itself,
> that's why i think there is something wrong with the network configuration
>
> thank you in advance for your help
>
>
>
> --
> View this message in context: http://old.nabble.com/network-configuration-%28etc-hosts%29---tp33013719p33013719.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434
Re: network configuration (etc/hosts) ?
Posted by ArunKumar <ar...@gmail.com>.
MirrorX,
Try out adding hostname of your master and slave system also to /etc/hosts/
That fixed same error for me.
master->
127.0.0.1 localhost6.localdomain6 localhost
127.0.1.1 localhost4.localdomain4 master-pc
192.168.7.110 master master-pc
192.168.7.157 slave lab-pc
slave->
127.0.1.1 localhost5.localdomain5 lab-pc
127.0.0.1 localhost3.localdomain3 localhost
192.168.7.110 master master-pc
192.168.7.157 slave lab-pc
Arun
--
View this message in context: http://lucene.472066.n3.nabble.com/network-configuration-etc-hosts-tp3602798p3605901.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.