You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Arjun Reddy <ar...@gmail.com> on 2012/08/08 23:39:30 UTC
Problem running PI example in Hadoop 2.0.0
I am trying to setup a small cluster using hadoop 2.0.0 and using PI
example to validate the setup. When I have 1 master and 1 slave the
example works fine. I am getting exceptions with the PI example when
additional slave nodes are added to the cluster. The syslogs for failed
tasks are as follows. Any ideas why this is happening.
2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:19,973 WARN [main]
org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5
configuration not found, setting default realm to empty
2012-08-08 15:41:20,142 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties
2012-08-08 15:41:20,221 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
period at 10 second(s).
2012-08-08 15:41:20,221 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
started
2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter: dfs.datanode.data.dir;
Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
job.xml:an attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild:
Sleeping for 0ms before retrying again. Got null now.
2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying
connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.net.ConnectException: Call From node2/
127.0.1.1 to node2:45965 failed on connection exception:
java.net.ConnectException: Connection refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at
org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
at
org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
at org.apache.hadoop.ipc.Client.call(Client.java:1141)
... 3 more
2012-08-08 15:41:30,493 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
2012-08-08 15:41:30,494 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
stopped.
2012-08-08 15:41:30,494 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
shutdown complete.
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
that's ubuntu for you. I'll update the wiki
On 10 August 2012 10:54, Arjun Reddy <ar...@gmail.com> wrote:
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.
>
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I did go through the 6 steps listed at the link and telnet to port
>>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>>
>>
>>
>> That's a high number - I wonder if it's something transient that didn't
>> start.
>>
>> I worry about the 127.0.1.1 number in the hostname -try editing
>> /etc/hosts to 127.0.0.1 to see if that helps; it could just be some
>> listening on a different port
>>
>>
>>>
>>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>>> am new to 2.0.0 version. I followed the document
>>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>>> setup the config files. Is there any other good document that describes
>>> how to set up version 2.0.0.
>>>
>>> Thanks
>>> Arjun
>>>
>>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <stevel@hortonworks.com
>>> > wrote:
>>>
>>>>
>>>>
>>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>>
>>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>>> example works fine. I am getting exceptions with the PI example when
>>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>>> tasks are as follows. Any ideas why this is happening.
>>>>>
>>>>>
>>>>> .
>>>>> 2012-08-08 15:41:30,492 WARN [main]
>>>>> org.apache.hadoop.mapred.YarnChild: Exception running child :
>>>>> java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965
>>>>> failed on connection exception: java.net.ConnectException: Connection
>>>>> refused; For more details see:
>>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>>> at
>>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>>> at
>>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>>> at $Proxy6.getTask(Unknown Source)
>>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>>
>>>>> did you follow the link in this message?
>>>>
>>>> If so, did you follow all the actions?
>>>>
>>>>
>>>
>>
>>
>> --
>> Steve Loughran
>> Hortonworks Inc
>> stevel@hortonworks.com
>> skype: steve_loughran
>> tel: +1 408 400 3721
>>
>> <http://hortonworks.com/download/>
>>
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
Re: Problem running PI example in Hadoop 2.0.0
Posted by anil gupta <an...@gmail.com>.
Hi Sathya,
It wont help you even little bit in unsubscribing from Hadoop list even if
you send a zillion unsubscribe email to user@hadoop.apache.org address.
Unsubscribe instructions are present on the Apache Hadoop website.
Here's what you are supposed to do for unsubscribing:
To unsubscribe, you have to send a mail to user-unsubscribe
@hadoop.apache.org with subject line as unsubscribe.
~Anil
On Fri, Aug 10, 2012 at 9:09 PM, sathyavageeswaran
<sa...@morisonmenon.com>wrote:
> unsubscribe****
>
> ** **
>
> *From:* Arjun Reddy [mailto:arjunreddy0768@gmail.com]
> *Sent:* 10 August 2012 23:25
> *To:* user@hadoop.apache.org
> *Subject:* Re: Problem running PI example in Hadoop 2.0.0****
>
> ** **
>
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.****
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965. ****
>
> ** **
>
> ** **
>
> That's a high number - I wonder if it's something transient that didn't
> start.****
>
> ** **
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port****
>
> ****
>
> ****
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0. ****
>
> ****
>
> Thanks****
>
> Arjun****
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.****
>
> ** **
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)****
>
> did you follow the link in this message? ****
>
> ** **
>
> If so, did you follow all the actions?****
>
> ** **
>
> ** **
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Steve Loughran****
>
> Hortonworks Inc****
>
> stevel@hortonworks.com****
>
> skype: steve_loughran****
>
> tel: +1 408 400 3721****
>
> ** **
>
> **** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
> ------------------------------
> <http://hortonworks.com/download/>
>
> No virus found in this message.
> Checked by AVG - *www.avg.com*
> Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12*
> *** <http://hortonworks.com/download/>
>
--
Thanks & Regards,
Anil Gupta
Re: Problem running PI example in Hadoop 2.0.0
Posted by anil gupta <an...@gmail.com>.
Hi Sathya,
It wont help you even little bit in unsubscribing from Hadoop list even if
you send a zillion unsubscribe email to user@hadoop.apache.org address.
Unsubscribe instructions are present on the Apache Hadoop website.
Here's what you are supposed to do for unsubscribing:
To unsubscribe, you have to send a mail to user-unsubscribe
@hadoop.apache.org with subject line as unsubscribe.
~Anil
On Fri, Aug 10, 2012 at 9:09 PM, sathyavageeswaran
<sa...@morisonmenon.com>wrote:
> unsubscribe****
>
> ** **
>
> *From:* Arjun Reddy [mailto:arjunreddy0768@gmail.com]
> *Sent:* 10 August 2012 23:25
> *To:* user@hadoop.apache.org
> *Subject:* Re: Problem running PI example in Hadoop 2.0.0****
>
> ** **
>
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.****
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965. ****
>
> ** **
>
> ** **
>
> That's a high number - I wonder if it's something transient that didn't
> start.****
>
> ** **
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port****
>
> ****
>
> ****
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0. ****
>
> ****
>
> Thanks****
>
> Arjun****
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.****
>
> ** **
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)****
>
> did you follow the link in this message? ****
>
> ** **
>
> If so, did you follow all the actions?****
>
> ** **
>
> ** **
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Steve Loughran****
>
> Hortonworks Inc****
>
> stevel@hortonworks.com****
>
> skype: steve_loughran****
>
> tel: +1 408 400 3721****
>
> ** **
>
> **** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
> ------------------------------
> <http://hortonworks.com/download/>
>
> No virus found in this message.
> Checked by AVG - *www.avg.com*
> Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12*
> *** <http://hortonworks.com/download/>
>
--
Thanks & Regards,
Anil Gupta
Re: Problem running PI example in Hadoop 2.0.0
Posted by anil gupta <an...@gmail.com>.
Hi Sathya,
It wont help you even little bit in unsubscribing from Hadoop list even if
you send a zillion unsubscribe email to user@hadoop.apache.org address.
Unsubscribe instructions are present on the Apache Hadoop website.
Here's what you are supposed to do for unsubscribing:
To unsubscribe, you have to send a mail to user-unsubscribe
@hadoop.apache.org with subject line as unsubscribe.
~Anil
On Fri, Aug 10, 2012 at 9:09 PM, sathyavageeswaran
<sa...@morisonmenon.com>wrote:
> unsubscribe****
>
> ** **
>
> *From:* Arjun Reddy [mailto:arjunreddy0768@gmail.com]
> *Sent:* 10 August 2012 23:25
> *To:* user@hadoop.apache.org
> *Subject:* Re: Problem running PI example in Hadoop 2.0.0****
>
> ** **
>
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.****
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965. ****
>
> ** **
>
> ** **
>
> That's a high number - I wonder if it's something transient that didn't
> start.****
>
> ** **
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port****
>
> ****
>
> ****
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0. ****
>
> ****
>
> Thanks****
>
> Arjun****
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.****
>
> ** **
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)****
>
> did you follow the link in this message? ****
>
> ** **
>
> If so, did you follow all the actions?****
>
> ** **
>
> ** **
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Steve Loughran****
>
> Hortonworks Inc****
>
> stevel@hortonworks.com****
>
> skype: steve_loughran****
>
> tel: +1 408 400 3721****
>
> ** **
>
> **** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
> ------------------------------
> <http://hortonworks.com/download/>
>
> No virus found in this message.
> Checked by AVG - *www.avg.com*
> Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12*
> *** <http://hortonworks.com/download/>
>
--
Thanks & Regards,
Anil Gupta
Re: Problem running PI example in Hadoop 2.0.0
Posted by anil gupta <an...@gmail.com>.
Hi Sathya,
It wont help you even little bit in unsubscribing from Hadoop list even if
you send a zillion unsubscribe email to user@hadoop.apache.org address.
Unsubscribe instructions are present on the Apache Hadoop website.
Here's what you are supposed to do for unsubscribing:
To unsubscribe, you have to send a mail to user-unsubscribe
@hadoop.apache.org with subject line as unsubscribe.
~Anil
On Fri, Aug 10, 2012 at 9:09 PM, sathyavageeswaran
<sa...@morisonmenon.com>wrote:
> unsubscribe****
>
> ** **
>
> *From:* Arjun Reddy [mailto:arjunreddy0768@gmail.com]
> *Sent:* 10 August 2012 23:25
> *To:* user@hadoop.apache.org
> *Subject:* Re: Problem running PI example in Hadoop 2.0.0****
>
> ** **
>
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.****
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965. ****
>
> ** **
>
> ** **
>
> That's a high number - I wonder if it's something transient that didn't
> start.****
>
> ** **
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port****
>
> ****
>
> ****
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0. ****
>
> ****
>
> Thanks****
>
> Arjun****
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
> wrote:****
>
> ** **
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:****
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.****
>
> ** **
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)****
>
> did you follow the link in this message? ****
>
> ** **
>
> If so, did you follow all the actions?****
>
> ** **
>
> ** **
>
>
>
> ****
>
> ** **
>
> -- ****
>
> Steve Loughran****
>
> Hortonworks Inc****
>
> stevel@hortonworks.com****
>
> skype: steve_loughran****
>
> tel: +1 408 400 3721****
>
> ** **
>
> **** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
>
> ** ** <http://hortonworks.com/download/>
> ------------------------------
> <http://hortonworks.com/download/>
>
> No virus found in this message.
> Checked by AVG - *www.avg.com*
> Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12*
> *** <http://hortonworks.com/download/>
>
--
Thanks & Regards,
Anil Gupta
RE: Problem running PI example in Hadoop 2.0.0
Posted by sathyavageeswaran <sa...@morisonmenon.com>.
unsubscribe
From: Arjun Reddy [mailto:arjunreddy0768@gmail.com]
Sent: 10 August 2012 23:25
To: user@hadoop.apache.org
Subject: Re: Problem running PI example in Hadoop 2.0.0
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the cluster
is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
wrote:
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-si
te/ClusterSetup.html to setup the config files. Is there any other good
document that describes how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
to validate the setup. When I have 1 master and 1 slave the example works
fine. I am getting exceptions with the PI example when additional slave
nodes are added to the cluster. The syslogs for failed tasks are as
follows. Any ideas why this is happening.
.
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.net.ConnectException: Call From
node2/127.0.1.1 <http://127.0.1.1/> to node2:45965 failed on connection
exception: java.net.ConnectException: Connection refused; For more details
see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.jav
a:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
did you follow the link in this message?
If so, did you follow all the actions?
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721 <tel:%2B1%20408%20400%203721>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
_____
<http://hortonworks.com/download/> No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
that's ubuntu for you. I'll update the wiki
On 10 August 2012 10:54, Arjun Reddy <ar...@gmail.com> wrote:
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.
>
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I did go through the 6 steps listed at the link and telnet to port
>>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>>
>>
>>
>> That's a high number - I wonder if it's something transient that didn't
>> start.
>>
>> I worry about the 127.0.1.1 number in the hostname -try editing
>> /etc/hosts to 127.0.0.1 to see if that helps; it could just be some
>> listening on a different port
>>
>>
>>>
>>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>>> am new to 2.0.0 version. I followed the document
>>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>>> setup the config files. Is there any other good document that describes
>>> how to set up version 2.0.0.
>>>
>>> Thanks
>>> Arjun
>>>
>>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <stevel@hortonworks.com
>>> > wrote:
>>>
>>>>
>>>>
>>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>>
>>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>>> example works fine. I am getting exceptions with the PI example when
>>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>>> tasks are as follows. Any ideas why this is happening.
>>>>>
>>>>>
>>>>> .
>>>>> 2012-08-08 15:41:30,492 WARN [main]
>>>>> org.apache.hadoop.mapred.YarnChild: Exception running child :
>>>>> java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965
>>>>> failed on connection exception: java.net.ConnectException: Connection
>>>>> refused; For more details see:
>>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>>> at
>>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>>> at
>>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>>> at $Proxy6.getTask(Unknown Source)
>>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>>
>>>>> did you follow the link in this message?
>>>>
>>>> If so, did you follow all the actions?
>>>>
>>>>
>>>
>>
>>
>> --
>> Steve Loughran
>> Hortonworks Inc
>> stevel@hortonworks.com
>> skype: steve_loughran
>> tel: +1 408 400 3721
>>
>> <http://hortonworks.com/download/>
>>
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
that's ubuntu for you. I'll update the wiki
On 10 August 2012 10:54, Arjun Reddy <ar...@gmail.com> wrote:
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.
>
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I did go through the 6 steps listed at the link and telnet to port
>>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>>
>>
>>
>> That's a high number - I wonder if it's something transient that didn't
>> start.
>>
>> I worry about the 127.0.1.1 number in the hostname -try editing
>> /etc/hosts to 127.0.0.1 to see if that helps; it could just be some
>> listening on a different port
>>
>>
>>>
>>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>>> am new to 2.0.0 version. I followed the document
>>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>>> setup the config files. Is there any other good document that describes
>>> how to set up version 2.0.0.
>>>
>>> Thanks
>>> Arjun
>>>
>>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <stevel@hortonworks.com
>>> > wrote:
>>>
>>>>
>>>>
>>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>>
>>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>>> example works fine. I am getting exceptions with the PI example when
>>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>>> tasks are as follows. Any ideas why this is happening.
>>>>>
>>>>>
>>>>> .
>>>>> 2012-08-08 15:41:30,492 WARN [main]
>>>>> org.apache.hadoop.mapred.YarnChild: Exception running child :
>>>>> java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965
>>>>> failed on connection exception: java.net.ConnectException: Connection
>>>>> refused; For more details see:
>>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>>> at
>>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>>> at
>>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>>> at $Proxy6.getTask(Unknown Source)
>>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>>
>>>>> did you follow the link in this message?
>>>>
>>>> If so, did you follow all the actions?
>>>>
>>>>
>>>
>>
>>
>> --
>> Steve Loughran
>> Hortonworks Inc
>> stevel@hortonworks.com
>> skype: steve_loughran
>> tel: +1 408 400 3721
>>
>> <http://hortonworks.com/download/>
>>
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
RE: Problem running PI example in Hadoop 2.0.0
Posted by sathyavageeswaran <sa...@morisonmenon.com>.
unsubscribe
From: Arjun Reddy [mailto:arjunreddy0768@gmail.com]
Sent: 10 August 2012 23:25
To: user@hadoop.apache.org
Subject: Re: Problem running PI example in Hadoop 2.0.0
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the cluster
is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
wrote:
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-si
te/ClusterSetup.html to setup the config files. Is there any other good
document that describes how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
to validate the setup. When I have 1 master and 1 slave the example works
fine. I am getting exceptions with the PI example when additional slave
nodes are added to the cluster. The syslogs for failed tasks are as
follows. Any ideas why this is happening.
.
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.net.ConnectException: Call From
node2/127.0.1.1 <http://127.0.1.1/> to node2:45965 failed on connection
exception: java.net.ConnectException: Connection refused; For more details
see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.jav
a:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
did you follow the link in this message?
If so, did you follow all the actions?
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721 <tel:%2B1%20408%20400%203721>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
_____
<http://hortonworks.com/download/> No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12
RE: Problem running PI example in Hadoop 2.0.0
Posted by sathyavageeswaran <sa...@morisonmenon.com>.
unsubscribe
From: Arjun Reddy [mailto:arjunreddy0768@gmail.com]
Sent: 10 August 2012 23:25
To: user@hadoop.apache.org
Subject: Re: Problem running PI example in Hadoop 2.0.0
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the cluster
is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
wrote:
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-si
te/ClusterSetup.html to setup the config files. Is there any other good
document that describes how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
to validate the setup. When I have 1 master and 1 slave the example works
fine. I am getting exceptions with the PI example when additional slave
nodes are added to the cluster. The syslogs for failed tasks are as
follows. Any ideas why this is happening.
.
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.net.ConnectException: Call From
node2/127.0.1.1 <http://127.0.1.1/> to node2:45965 failed on connection
exception: java.net.ConnectException: Connection refused; For more details
see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.jav
a:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
did you follow the link in this message?
If so, did you follow all the actions?
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721 <tel:%2B1%20408%20400%203721>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
_____
<http://hortonworks.com/download/> No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12
RE: Problem running PI example in Hadoop 2.0.0
Posted by sathyavageeswaran <sa...@morisonmenon.com>.
unsubscribe
From: Arjun Reddy [mailto:arjunreddy0768@gmail.com]
Sent: 10 August 2012 23:25
To: user@hadoop.apache.org
Subject: Re: Problem running PI example in Hadoop 2.0.0
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the cluster
is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>
wrote:
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-si
te/ClusterSetup.html to setup the config files. Is there any other good
document that describes how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
to validate the setup. When I have 1 master and 1 slave the example works
fine. I am getting exceptions with the PI example when additional slave
nodes are added to the cluster. The syslogs for failed tasks are as
follows. Any ideas why this is happening.
.
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.net.ConnectException: Call From
node2/127.0.1.1 <http://127.0.1.1/> to node2:45965 failed on connection
exception: java.net.ConnectException: Connection refused; For more details
see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.jav
a:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
did you follow the link in this message?
If so, did you follow all the actions?
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721 <tel:%2B1%20408%20400%203721>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
<http://hortonworks.com/download/>
_____
<http://hortonworks.com/download/> No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2197 / Virus Database: 2437/5190 - Release Date: 08/09/12
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
that's ubuntu for you. I'll update the wiki
On 10 August 2012 10:54, Arjun Reddy <ar...@gmail.com> wrote:
> I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
> cluster is working fine now. Thank you very much for your help.
>
>
> On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I did go through the 6 steps listed at the link and telnet to port
>>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>>
>>
>>
>> That's a high number - I wonder if it's something transient that didn't
>> start.
>>
>> I worry about the 127.0.1.1 number in the hostname -try editing
>> /etc/hosts to 127.0.0.1 to see if that helps; it could just be some
>> listening on a different port
>>
>>
>>>
>>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>>> am new to 2.0.0 version. I followed the document
>>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>>> setup the config files. Is there any other good document that describes
>>> how to set up version 2.0.0.
>>>
>>> Thanks
>>> Arjun
>>>
>>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <stevel@hortonworks.com
>>> > wrote:
>>>
>>>>
>>>>
>>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>>
>>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>>> example works fine. I am getting exceptions with the PI example when
>>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>>> tasks are as follows. Any ideas why this is happening.
>>>>>
>>>>>
>>>>> .
>>>>> 2012-08-08 15:41:30,492 WARN [main]
>>>>> org.apache.hadoop.mapred.YarnChild: Exception running child :
>>>>> java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965
>>>>> failed on connection exception: java.net.ConnectException: Connection
>>>>> refused; For more details see:
>>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>>> at
>>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>>> at
>>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>>> at $Proxy6.getTask(Unknown Source)
>>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>>
>>>>> did you follow the link in this message?
>>>>
>>>> If so, did you follow all the actions?
>>>>
>>>>
>>>
>>
>>
>> --
>> Steve Loughran
>> Hortonworks Inc
>> stevel@hortonworks.com
>> skype: steve_loughran
>> tel: +1 408 400 3721
>>
>> <http://hortonworks.com/download/>
>>
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
cluster is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I did go through the 6 steps listed at the link and telnet to port
>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>
>
>
> That's a high number - I wonder if it's something transient that didn't
> start.
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port
>
>
>>
>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>> am new to 2.0.0 version. I followed the document
>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>> setup the config files. Is there any other good document that describes
>> how to set up version 2.0.0.
>>
>> Thanks
>> Arjun
>>
>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>>
>>>
>>>
>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>
>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>> example works fine. I am getting exceptions with the PI example when
>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>> tasks are as follows. Any ideas why this is happening.
>>>>
>>>>
>>>> .
>>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>>> Exception running child : java.net.ConnectException: Call From node2/
>>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>>> java.net.ConnectException: Connection refused; For more details see:
>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>> at
>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>> at
>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>> at $Proxy6.getTask(Unknown Source)
>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>
>>>> did you follow the link in this message?
>>>
>>> If so, did you follow all the actions?
>>>
>>>
>>
>
>
> --
> Steve Loughran
> Hortonworks Inc
> stevel@hortonworks.com
> skype: steve_loughran
> tel: +1 408 400 3721
>
> <http://hortonworks.com/download/>
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
cluster is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I did go through the 6 steps listed at the link and telnet to port
>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>
>
>
> That's a high number - I wonder if it's something transient that didn't
> start.
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port
>
>
>>
>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>> am new to 2.0.0 version. I followed the document
>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>> setup the config files. Is there any other good document that describes
>> how to set up version 2.0.0.
>>
>> Thanks
>> Arjun
>>
>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>>
>>>
>>>
>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>
>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>> example works fine. I am getting exceptions with the PI example when
>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>> tasks are as follows. Any ideas why this is happening.
>>>>
>>>>
>>>> .
>>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>>> Exception running child : java.net.ConnectException: Call From node2/
>>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>>> java.net.ConnectException: Connection refused; For more details see:
>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>> at
>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>> at
>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>> at $Proxy6.getTask(Unknown Source)
>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>
>>>> did you follow the link in this message?
>>>
>>> If so, did you follow all the actions?
>>>
>>>
>>
>
>
> --
> Steve Loughran
> Hortonworks Inc
> stevel@hortonworks.com
> skype: steve_loughran
> tel: +1 408 400 3721
>
> <http://hortonworks.com/download/>
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
cluster is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I did go through the 6 steps listed at the link and telnet to port
>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>
>
>
> That's a high number - I wonder if it's something transient that didn't
> start.
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port
>
>
>>
>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>> am new to 2.0.0 version. I followed the document
>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>> setup the config files. Is there any other good document that describes
>> how to set up version 2.0.0.
>>
>> Thanks
>> Arjun
>>
>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>>
>>>
>>>
>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>
>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>> example works fine. I am getting exceptions with the PI example when
>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>> tasks are as follows. Any ideas why this is happening.
>>>>
>>>>
>>>> .
>>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>>> Exception running child : java.net.ConnectException: Call From node2/
>>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>>> java.net.ConnectException: Connection refused; For more details see:
>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>> at
>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>> at
>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>> at $Proxy6.getTask(Unknown Source)
>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>
>>>> did you follow the link in this message?
>>>
>>> If so, did you follow all the actions?
>>>
>>>
>>
>
>
> --
> Steve Loughran
> Hortonworks Inc
> stevel@hortonworks.com
> skype: steve_loughran
> tel: +1 408 400 3721
>
> <http://hortonworks.com/download/>
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I removed the entry 127.0.1.1 in /etc/hosts on all the nodes and the
cluster is working fine now. Thank you very much for your help.
On Thu, Aug 9, 2012 at 3:15 PM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I did go through the 6 steps listed at the link and telnet to port
>> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>>
>
>
> That's a high number - I wonder if it's something transient that didn't
> start.
>
> I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
> to 127.0.0.1 to see if that helps; it could just be some listening on a
> different port
>
>
>>
>> We can run hadoop 1.0.3 version without any issues on this cluster and I
>> am new to 2.0.0 version. I followed the document
>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
>> setup the config files. Is there any other good document that describes
>> how to set up version 2.0.0.
>>
>> Thanks
>> Arjun
>>
>> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>>
>>>
>>>
>>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>>
>>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>>> example to validate the setup. When I have 1 master and 1 slave the
>>>> example works fine. I am getting exceptions with the PI example when
>>>> additional slave nodes are added to the cluster. The syslogs for failed
>>>> tasks are as follows. Any ideas why this is happening.
>>>>
>>>>
>>>> .
>>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>>> Exception running child : java.net.ConnectException: Call From node2/
>>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>>> java.net.ConnectException: Connection refused; For more details see:
>>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>>> at
>>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>>> at
>>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>>> at $Proxy6.getTask(Unknown Source)
>>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>>
>>>> did you follow the link in this message?
>>>
>>> If so, did you follow all the actions?
>>>
>>>
>>
>
>
> --
> Steve Loughran
> Hortonworks Inc
> stevel@hortonworks.com
> skype: steve_loughran
> tel: +1 408 400 3721
>
> <http://hortonworks.com/download/>
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0.
>
> Thanks
> Arjun
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>> example to validate the setup. When I have 1 master and 1 slave the
>>> example works fine. I am getting exceptions with the PI example when
>>> additional slave nodes are added to the cluster. The syslogs for failed
>>> tasks are as follows. Any ideas why this is happening.
>>>
>>>
>>> .
>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>> Exception running child : java.net.ConnectException: Call From node2/
>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>> java.net.ConnectException: Connection refused; For more details see:
>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>> at
>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>> at
>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>> at $Proxy6.getTask(Unknown Source)
>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>
>>> did you follow the link in this message?
>>
>> If so, did you follow all the actions?
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0.
>
> Thanks
> Arjun
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>> example to validate the setup. When I have 1 master and 1 slave the
>>> example works fine. I am getting exceptions with the PI example when
>>> additional slave nodes are added to the cluster. The syslogs for failed
>>> tasks are as follows. Any ideas why this is happening.
>>>
>>>
>>> .
>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>> Exception running child : java.net.ConnectException: Call From node2/
>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>> java.net.ConnectException: Connection refused; For more details see:
>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>> at
>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>> at
>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>> at $Proxy6.getTask(Unknown Source)
>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>
>>> did you follow the link in this message?
>>
>> If so, did you follow all the actions?
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0.
>
> Thanks
> Arjun
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>> example to validate the setup. When I have 1 master and 1 slave the
>>> example works fine. I am getting exceptions with the PI example when
>>> additional slave nodes are added to the cluster. The syslogs for failed
>>> tasks are as follows. Any ideas why this is happening.
>>>
>>>
>>> .
>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>> Exception running child : java.net.ConnectException: Call From node2/
>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>> java.net.ConnectException: Connection refused; For more details see:
>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>> at
>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>> at
>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>> at $Proxy6.getTask(Unknown Source)
>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>
>>> did you follow the link in this message?
>>
>> If so, did you follow all the actions?
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 9 August 2012 10:31, Arjun Reddy <ar...@gmail.com> wrote:
> I did go through the 6 steps listed at the link and telnet to port
> 45965 is failing. Netstat -a shows nothing is listing on port 45965.
>
That's a high number - I wonder if it's something transient that didn't
start.
I worry about the 127.0.1.1 number in the hostname -try editing /etc/hosts
to 127.0.0.1 to see if that helps; it could just be some listening on a
different port
>
> We can run hadoop 1.0.3 version without any issues on this cluster and I
> am new to 2.0.0 version. I followed the document
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html to
> setup the config files. Is there any other good document that describes
> how to set up version 2.0.0.
>
> Thanks
> Arjun
>
> On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>>
>>
>> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>>
>>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>>> example to validate the setup. When I have 1 master and 1 slave the
>>> example works fine. I am getting exceptions with the PI example when
>>> additional slave nodes are added to the cluster. The syslogs for failed
>>> tasks are as follows. Any ideas why this is happening.
>>>
>>>
>>> .
>>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>> Exception running child : java.net.ConnectException: Call From node2/
>>> 127.0.1.1 to node2:45965 failed on connection exception:
>>> java.net.ConnectException: Connection refused; For more details see:
>>> http://wiki.apache.org/hadoop/ConnectionRefused
>>> at
>>> org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>>> at
>>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>>> at $Proxy6.getTask(Unknown Source)
>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>>
>>> did you follow the link in this message?
>>
>> If so, did you follow all the actions?
>>
>>
>
--
Steve Loughran
Hortonworks Inc
stevel@hortonworks.com
skype: steve_loughran
tel: +1 408 400 3721
<http://hortonworks.com/download/>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
to
setup the config files. Is there any other good document that describes
how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>> example to validate the setup. When I have 1 master and 1 slave the
>> example works fine. I am getting exceptions with the PI example when
>> additional slave nodes are added to the cluster. The syslogs for failed
>> tasks are as follows. Any ideas why this is happening.
>>
>>
>> .
>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>> Exception running child : java.net.ConnectException: Call From node2/
>> 127.0.1.1 to node2:45965 failed on connection exception:
>> java.net.ConnectException: Connection refused; For more details see:
>> http://wiki.apache.org/hadoop/ConnectionRefused
>> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>> at
>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>> at $Proxy6.getTask(Unknown Source)
>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>
>> did you follow the link in this message?
>
> If so, did you follow all the actions?
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
to
setup the config files. Is there any other good document that describes
how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>> example to validate the setup. When I have 1 master and 1 slave the
>> example works fine. I am getting exceptions with the PI example when
>> additional slave nodes are added to the cluster. The syslogs for failed
>> tasks are as follows. Any ideas why this is happening.
>>
>>
>> .
>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>> Exception running child : java.net.ConnectException: Call From node2/
>> 127.0.1.1 to node2:45965 failed on connection exception:
>> java.net.ConnectException: Connection refused; For more details see:
>> http://wiki.apache.org/hadoop/ConnectionRefused
>> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>> at
>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>> at $Proxy6.getTask(Unknown Source)
>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>
>> did you follow the link in this message?
>
> If so, did you follow all the actions?
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
to
setup the config files. Is there any other good document that describes
how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>> example to validate the setup. When I have 1 master and 1 slave the
>> example works fine. I am getting exceptions with the PI example when
>> additional slave nodes are added to the cluster. The syslogs for failed
>> tasks are as follows. Any ideas why this is happening.
>>
>>
>> .
>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>> Exception running child : java.net.ConnectException: Call From node2/
>> 127.0.1.1 to node2:45965 failed on connection exception:
>> java.net.ConnectException: Connection refused; For more details see:
>> http://wiki.apache.org/hadoop/ConnectionRefused
>> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>> at
>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>> at $Proxy6.getTask(Unknown Source)
>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>
>> did you follow the link in this message?
>
> If so, did you follow all the actions?
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Arjun Reddy <ar...@gmail.com>.
I did go through the 6 steps listed at the link and telnet to port 45965 is
failing. Netstat -a shows nothing is listing on port 45965.
We can run hadoop 1.0.3 version without any issues on this cluster and I am
new to 2.0.0 version. I followed the document
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html
to
setup the config files. Is there any other good document that describes
how to set up version 2.0.0.
Thanks
Arjun
On Thu, Aug 9, 2012 at 11:47 AM, Steve Loughran <st...@hortonworks.com>wrote:
>
>
> On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
>
>> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
>> example to validate the setup. When I have 1 master and 1 slave the
>> example works fine. I am getting exceptions with the PI example when
>> additional slave nodes are added to the cluster. The syslogs for failed
>> tasks are as follows. Any ideas why this is happening.
>>
>>
>> .
>> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
>> Exception running child : java.net.ConnectException: Call From node2/
>> 127.0.1.1 to node2:45965 failed on connection exception:
>> java.net.ConnectException: Connection refused; For more details see:
>> http://wiki.apache.org/hadoop/ConnectionRefused
>> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
>> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
>> at
>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
>> at $Proxy6.getTask(Unknown Source)
>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>>
>> did you follow the link in this message?
>
> If so, did you follow all the actions?
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.
>
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>
did you follow the link in this message?
If so, did you follow all the actions?
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.
>
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>
did you follow the link in this message?
If so, did you follow all the actions?
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.
>
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>
did you follow the link in this message?
If so, did you follow all the actions?
Re: Problem running PI example in Hadoop 2.0.0
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Arjun,
Seems 'loopback' prob to me..Make a check on what Harit has pointed.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 3:17 AM, Harit Himanshu
<ha...@gmail.com> wrote:
> - when adding slaves are you adding slaves in conf/slaves file? (I assume
> yes)
> - are these slaves mapped in your /etc/hosts file?
> - in your slaves con/masters do you have entry for master host?
> - in your slaves /etc/hosts does master resolves to the correct IP?
>
>
> On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
> to validate the setup. When I have 1 master and 1 slave the example works
> fine. I am getting exceptions with the PI example when additional slave
> nodes are added to the cluster. The syslogs for failed tasks are as
> follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main]
> org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5
> configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period
> at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.datanode.data.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From
> node2/127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
> system...
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> stopped.
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> shutdown complete.
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Arjun,
Seems 'loopback' prob to me..Make a check on what Harit has pointed.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 3:17 AM, Harit Himanshu
<ha...@gmail.com> wrote:
> - when adding slaves are you adding slaves in conf/slaves file? (I assume
> yes)
> - are these slaves mapped in your /etc/hosts file?
> - in your slaves con/masters do you have entry for master host?
> - in your slaves /etc/hosts does master resolves to the correct IP?
>
>
> On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
> to validate the setup. When I have 1 master and 1 slave the example works
> fine. I am getting exceptions with the PI example when additional slave
> nodes are added to the cluster. The syslogs for failed tasks are as
> follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main]
> org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5
> configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period
> at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.datanode.data.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From
> node2/127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
> system...
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> stopped.
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> shutdown complete.
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Arjun,
Seems 'loopback' prob to me..Make a check on what Harit has pointed.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 3:17 AM, Harit Himanshu
<ha...@gmail.com> wrote:
> - when adding slaves are you adding slaves in conf/slaves file? (I assume
> yes)
> - are these slaves mapped in your /etc/hosts file?
> - in your slaves con/masters do you have entry for master host?
> - in your slaves /etc/hosts does master resolves to the correct IP?
>
>
> On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
> to validate the setup. When I have 1 master and 1 slave the example works
> fine. I am getting exceptions with the PI example when additional slave
> nodes are added to the cluster. The syslogs for failed tasks are as
> follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main]
> org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5
> configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period
> at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.datanode.data.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From
> node2/127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
> system...
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> stopped.
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> shutdown complete.
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Arjun,
Seems 'loopback' prob to me..Make a check on what Harit has pointed.
Regards,
Mohammad Tariq
On Thu, Aug 9, 2012 at 3:17 AM, Harit Himanshu
<ha...@gmail.com> wrote:
> - when adding slaves are you adding slaves in conf/slaves file? (I assume
> yes)
> - are these slaves mapped in your /etc/hosts file?
> - in your slaves con/masters do you have entry for master host?
> - in your slaves /etc/hosts does master resolves to the correct IP?
>
>
> On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
>
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example
> to validate the setup. When I have 1 master and 1 slave the example works
> fine. I am getting exceptions with the PI example when additional slave
> nodes are added to the cluster. The syslogs for failed tasks are as
> follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main]
> org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5
> configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period
> at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.datanode.data.dir;
> Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From
> node2/127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
> system...
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> stopped.
> 2012-08-08 15:41:30,494 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> shutdown complete.
>
>
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Harit Himanshu <ha...@gmail.com>.
- when adding slaves are you adding slaves in conf/slaves file? (I assume yes)
- are these slaves mapped in your /etc/hosts file?
- in your slaves con/masters do you have entry for master host?
- in your slaves /etc/hosts does master resolves to the correct IP?
On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Harit Himanshu <ha...@gmail.com>.
- when adding slaves are you adding slaves in conf/slaves file? (I assume yes)
- are these slaves mapped in your /etc/hosts file?
- in your slaves con/masters do you have entry for master host?
- in your slaves /etc/hosts does master resolves to the correct IP?
On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 8 August 2012 14:39, Arjun Reddy <ar...@gmail.com> wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI
> example to validate the setup. When I have 1 master and 1 slave the
> example works fine. I am getting exceptions with the PI example when
> additional slave nodes are added to the cluster. The syslogs for failed
> tasks are as follows. Any ideas why this is happening.
>
>
> .
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.net.ConnectException: Call From node2/
> 127.0.1.1 to node2:45965 failed on connection exception:
> java.net.ConnectException: Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
>
did you follow the link in this message?
If so, did you follow all the actions?
Re: Problem running PI example in Hadoop 2.0.0
Posted by Harit Himanshu <ha...@gmail.com>.
- when adding slaves are you adding slaves in conf/slaves file? (I assume yes)
- are these slaves mapped in your /etc/hosts file?
- in your slaves con/masters do you have entry for master host?
- in your slaves /etc/hosts does master resolves to the correct IP?
On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Harit Himanshu <ha...@gmail.com>.
- when adding slaves are you adding slaves in conf/slaves file? (I assume yes)
- are these slaves mapped in your /etc/hosts file?
- in your slaves con/masters do you have entry for master host?
- in your slaves /etc/hosts does master resolves to the correct IP?
On Aug 8, 2012, at 2:39 PM, Arjun Reddy wrote:
> I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
>
> 2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
> 2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
> 2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
> 2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
> 2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
> 2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 0 time(s).
> 2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 1 time(s).
> 2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 2 time(s).
> 2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 3 time(s).
> 2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 4 time(s).
> 2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 5 time(s).
> 2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 6 time(s).
> 2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 7 time(s).
> 2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 8 time(s).
> 2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965. Already tried 9 time(s).
> 2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1 to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
> at org.apache.hadoop.ipc.Client.call(Client.java:1165)
> at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
> at $Proxy6.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
> at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
> at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
> at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
> at org.apache.hadoop.ipc.Client.call(Client.java:1141)
> ... 3 more
>
> 2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
> 2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
>
Re: Problem running PI example in Hadoop 2.0.0
Posted by Serge Blazhiyevskyy <Se...@nice.com>.
Looks like misconfiguration
Make sure that your master and slaves connect to externally available IP
Thanks
Serge
From: Arjun Reddy <ar...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wed, 8 Aug 2012 15:39:30 -0600
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Problem running PI example in Hadoop 2.0.0
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 0 time(s).
2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 1 time(s).
2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 2 time(s).
2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 3 time(s).
2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 4 time(s).
2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 5 time(s).
2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 6 time(s).
2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 7 time(s).
2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 8 time(s).
2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 9 time(s).
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1<http://127.0.1.1> to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
at org.apache.hadoop.ipc.Client.call(Client.java:1141)
... 3 more
2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
Re: Problem running PI example in Hadoop 2.0.0
Posted by Serge Blazhiyevskyy <Se...@nice.com>.
Looks like misconfiguration
Make sure that your master and slaves connect to externally available IP
Thanks
Serge
From: Arjun Reddy <ar...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wed, 8 Aug 2012 15:39:30 -0600
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Problem running PI example in Hadoop 2.0.0
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 0 time(s).
2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 1 time(s).
2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 2 time(s).
2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 3 time(s).
2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 4 time(s).
2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 5 time(s).
2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 6 time(s).
2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 7 time(s).
2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 8 time(s).
2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 9 time(s).
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1<http://127.0.1.1> to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
at org.apache.hadoop.ipc.Client.call(Client.java:1141)
... 3 more
2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
Re: Problem running PI example in Hadoop 2.0.0
Posted by Serge Blazhiyevskyy <Se...@nice.com>.
Looks like misconfiguration
Make sure that your master and slaves connect to externally available IP
Thanks
Serge
From: Arjun Reddy <ar...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wed, 8 Aug 2012 15:39:30 -0600
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Problem running PI example in Hadoop 2.0.0
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 0 time(s).
2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 1 time(s).
2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 2 time(s).
2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 3 time(s).
2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 4 time(s).
2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 5 time(s).
2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 6 time(s).
2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 7 time(s).
2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 8 time(s).
2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 9 time(s).
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1<http://127.0.1.1> to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
at org.apache.hadoop.ipc.Client.call(Client.java:1141)
... 3 more
2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.
Re: Problem running PI example in Hadoop 2.0.0
Posted by Serge Blazhiyevskyy <Se...@nice.com>.
Looks like misconfiguration
Make sure that your master and slaves connect to externally available IP
Thanks
Serge
From: Arjun Reddy <ar...@gmail.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Wed, 8 Aug 2012 15:39:30 -0600
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Problem running PI example in Hadoop 2.0.0
I am trying to setup a small cluster using hadoop 2.0.0 and using PI example to validate the setup. When I have 1 master and 1 slave the example works fine. I am getting exceptions with the PI example when additional slave nodes are added to the cluster. The syslogs for failed tasks are as follows. Any ideas why this is happening.
2012-08-08 15:41:19,914 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:19,915 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:19,973 WARN [main] org.apache.hadoop.security.authentication.util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
2012-08-08 15:41:20,142 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2012-08-08 15:41:20,221 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2012-08-08 15:41:20,377 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.namenode.name.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: dfs.datanode.data.dir; Ignoring.
2012-08-08 15:41:20,378 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2012-08-08 15:41:20,435 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2012-08-08 15:41:21,483 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 0 time(s).
2012-08-08 15:41:22,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 1 time(s).
2012-08-08 15:41:23,484 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 2 time(s).
2012-08-08 15:41:24,485 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 3 time(s).
2012-08-08 15:41:25,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 4 time(s).
2012-08-08 15:41:26,486 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 5 time(s).
2012-08-08 15:41:27,487 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 6 time(s).
2012-08-08 15:41:28,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 7 time(s).
2012-08-08 15:41:29,488 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 8 time(s).
2012-08-08 15:41:30,489 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: node2/127.0.1.1:45965<http://127.0.1.1:45965>. Already tried 9 time(s).
2012-08-08 15:41:30,492 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.net.ConnectException: Call From node2/127.0.1.1<http://127.0.1.1> to node2:45965 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
at $Proxy6.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:123)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
at org.apache.hadoop.ipc.Client.call(Client.java:1141)
... 3 more
2012-08-08 15:41:30,493 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.
2012-08-08 15:41:30,494 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.