You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by sandeep reddy <la...@yahoo.com> on 2013/04/12 11:45:47 UTC
Map tasks never getting started
Hi,
While running a mapred job map task never getting assigned. The message for a map in web interface is "No Task Attempts found".
When I check the logs there are couple of errors as following:
2013-04-12 02:29:04,454 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:jobs cause:java.io.IOException: java.lang.NullPointerException
2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 4571, call heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false, true, 4) from 192.168.2.20:59968: error: java.io.IOException: java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
at org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
at org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
at org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
at org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201304120228_0001_m_000002_0'
Some please help me to resolve this issue.
I am using hadoop-1.0.2
Thanks,
Sandeep.
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
The default mapping comes from the rack awareness script, not the
topology data file it uses. Also, note that if you start a TT or DN
before you update the topology data and refresh it, the default rack
value gets assigned instead.
On Fri, Apr 12, 2013 at 3:42 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Harsh,
>
> My topology file looks similar as following:
>
> 192.168.2.20 host1.example.com host1 /tc/rack1
> 192.168.2.21 host2.example.com host2 /tc/rack1
> 192.168.2.22 host3.example.com host3 /tc/rack1
> 192.168.2.23 host4.example.com host4 /tc/rack2
> 192.168.2.24 host5.example.com host5 /tc/rack2
>
> If I am not wrong all hosts have the same levers.
>
> Can you please let me know how to including the default rack value used
> upon a mismatch?
>
> Thanks,
> Sandeep.
> ________________________________
> From: Harsh J <ha...@cloudera.com>
> To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy
> <la...@yahoo.com>
> Sent: Friday, April 12, 2013 3:22 PM
> Subject: Re: Map tasks never getting started
>
> Your rack topology configuration is broken (mismatching levels) and
> you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
> current release, or fix your rack configuration to be of the same
> levels for all mapped nodes (including the default rack value used
> upon a mismatch) and restart the JobTracker.
>
> On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
> <la...@yahoo.com> wrote:
>> Hi,
>>
>> While running a mapred job map task never getting assigned. The message
>> for
>> a map in web interface is "No Task Attempts found".
>> When I check the logs there are couple of errors as following:
>>
>> 2013-04-12 02:29:04,454 ERROR
>> org.apache.hadoop.security.UserGroupInformation:
>> PriviledgedActionException
>> as:jobs cause:java.io.IOException: java.lang.NullPointerException
>> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 8 on 4571, call
>> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false,
>> false,
>> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
>> java.lang.NullPointerException
>> java.io.IOException: java.lang.NullPointerException
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
>> at
>>
>> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
>> at
>> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
>> task 'attempt_201304120228_0001_m_000002_0'
>>
>> Some please help me to resolve this issue.
>> I am using hadoop-1.0.2
>>
>> Thanks,
>> Sandeep.
>
>
>
> --
> Harsh J
>
>
--
Harsh J
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
The default mapping comes from the rack awareness script, not the
topology data file it uses. Also, note that if you start a TT or DN
before you update the topology data and refresh it, the default rack
value gets assigned instead.
On Fri, Apr 12, 2013 at 3:42 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Harsh,
>
> My topology file looks similar as following:
>
> 192.168.2.20 host1.example.com host1 /tc/rack1
> 192.168.2.21 host2.example.com host2 /tc/rack1
> 192.168.2.22 host3.example.com host3 /tc/rack1
> 192.168.2.23 host4.example.com host4 /tc/rack2
> 192.168.2.24 host5.example.com host5 /tc/rack2
>
> If I am not wrong all hosts have the same levers.
>
> Can you please let me know how to including the default rack value used
> upon a mismatch?
>
> Thanks,
> Sandeep.
> ________________________________
> From: Harsh J <ha...@cloudera.com>
> To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy
> <la...@yahoo.com>
> Sent: Friday, April 12, 2013 3:22 PM
> Subject: Re: Map tasks never getting started
>
> Your rack topology configuration is broken (mismatching levels) and
> you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
> current release, or fix your rack configuration to be of the same
> levels for all mapped nodes (including the default rack value used
> upon a mismatch) and restart the JobTracker.
>
> On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
> <la...@yahoo.com> wrote:
>> Hi,
>>
>> While running a mapred job map task never getting assigned. The message
>> for
>> a map in web interface is "No Task Attempts found".
>> When I check the logs there are couple of errors as following:
>>
>> 2013-04-12 02:29:04,454 ERROR
>> org.apache.hadoop.security.UserGroupInformation:
>> PriviledgedActionException
>> as:jobs cause:java.io.IOException: java.lang.NullPointerException
>> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 8 on 4571, call
>> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false,
>> false,
>> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
>> java.lang.NullPointerException
>> java.io.IOException: java.lang.NullPointerException
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
>> at
>>
>> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
>> at
>> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
>> task 'attempt_201304120228_0001_m_000002_0'
>>
>> Some please help me to resolve this issue.
>> I am using hadoop-1.0.2
>>
>> Thanks,
>> Sandeep.
>
>
>
> --
> Harsh J
>
>
--
Harsh J
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
The default mapping comes from the rack awareness script, not the
topology data file it uses. Also, note that if you start a TT or DN
before you update the topology data and refresh it, the default rack
value gets assigned instead.
On Fri, Apr 12, 2013 at 3:42 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Harsh,
>
> My topology file looks similar as following:
>
> 192.168.2.20 host1.example.com host1 /tc/rack1
> 192.168.2.21 host2.example.com host2 /tc/rack1
> 192.168.2.22 host3.example.com host3 /tc/rack1
> 192.168.2.23 host4.example.com host4 /tc/rack2
> 192.168.2.24 host5.example.com host5 /tc/rack2
>
> If I am not wrong all hosts have the same levers.
>
> Can you please let me know how to including the default rack value used
> upon a mismatch?
>
> Thanks,
> Sandeep.
> ________________________________
> From: Harsh J <ha...@cloudera.com>
> To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy
> <la...@yahoo.com>
> Sent: Friday, April 12, 2013 3:22 PM
> Subject: Re: Map tasks never getting started
>
> Your rack topology configuration is broken (mismatching levels) and
> you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
> current release, or fix your rack configuration to be of the same
> levels for all mapped nodes (including the default rack value used
> upon a mismatch) and restart the JobTracker.
>
> On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
> <la...@yahoo.com> wrote:
>> Hi,
>>
>> While running a mapred job map task never getting assigned. The message
>> for
>> a map in web interface is "No Task Attempts found".
>> When I check the logs there are couple of errors as following:
>>
>> 2013-04-12 02:29:04,454 ERROR
>> org.apache.hadoop.security.UserGroupInformation:
>> PriviledgedActionException
>> as:jobs cause:java.io.IOException: java.lang.NullPointerException
>> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 8 on 4571, call
>> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false,
>> false,
>> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
>> java.lang.NullPointerException
>> java.io.IOException: java.lang.NullPointerException
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
>> at
>>
>> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
>> at
>> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
>> task 'attempt_201304120228_0001_m_000002_0'
>>
>> Some please help me to resolve this issue.
>> I am using hadoop-1.0.2
>>
>> Thanks,
>> Sandeep.
>
>
>
> --
> Harsh J
>
>
--
Harsh J
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
The default mapping comes from the rack awareness script, not the
topology data file it uses. Also, note that if you start a TT or DN
before you update the topology data and refresh it, the default rack
value gets assigned instead.
On Fri, Apr 12, 2013 at 3:42 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Harsh,
>
> My topology file looks similar as following:
>
> 192.168.2.20 host1.example.com host1 /tc/rack1
> 192.168.2.21 host2.example.com host2 /tc/rack1
> 192.168.2.22 host3.example.com host3 /tc/rack1
> 192.168.2.23 host4.example.com host4 /tc/rack2
> 192.168.2.24 host5.example.com host5 /tc/rack2
>
> If I am not wrong all hosts have the same levers.
>
> Can you please let me know how to including the default rack value used
> upon a mismatch?
>
> Thanks,
> Sandeep.
> ________________________________
> From: Harsh J <ha...@cloudera.com>
> To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy
> <la...@yahoo.com>
> Sent: Friday, April 12, 2013 3:22 PM
> Subject: Re: Map tasks never getting started
>
> Your rack topology configuration is broken (mismatching levels) and
> you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
> current release, or fix your rack configuration to be of the same
> levels for all mapped nodes (including the default rack value used
> upon a mismatch) and restart the JobTracker.
>
> On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
> <la...@yahoo.com> wrote:
>> Hi,
>>
>> While running a mapred job map task never getting assigned. The message
>> for
>> a map in web interface is "No Task Attempts found".
>> When I check the logs there are couple of errors as following:
>>
>> 2013-04-12 02:29:04,454 ERROR
>> org.apache.hadoop.security.UserGroupInformation:
>> PriviledgedActionException
>> as:jobs cause:java.io.IOException: java.lang.NullPointerException
>> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 8 on 4571, call
>> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false,
>> false,
>> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
>> java.lang.NullPointerException
>> java.io.IOException: java.lang.NullPointerException
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
>> at
>>
>> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
>> at
>>
>> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
>> at
>> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
>> task 'attempt_201304120228_0001_m_000002_0'
>>
>> Some please help me to resolve this issue.
>> I am using hadoop-1.0.2
>>
>> Thanks,
>> Sandeep.
>
>
>
> --
> Harsh J
>
>
--
Harsh J
Re: Map tasks never getting started
Posted by sandeep reddy <la...@yahoo.com>.
Harsh,
My topology file looks similar as following:
192.168.2.20 host1.example.com host1 /tc/rack1
192.168.2.21 host2.example.com host2 /tc/rack1
192.168.2.22 host3.example.com host3 /tc/rack1
192.168.2.23 host4.example.com host4 /tc/rack2
192.168.2.24 host5.example.com host5 /tc/rack2
If I am not wrong all hosts have the same levers.
Can you please let me know how to including the default rack value used
upon a mismatch?
Thanks,
Sandeep.
________________________________
From: Harsh J <ha...@cloudera.com>
To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy <la...@yahoo.com>
Sent: Friday, April 12, 2013 3:22 PM
Subject: Re: Map tasks never getting started
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J
Re: Map tasks never getting started
Posted by sandeep reddy <la...@yahoo.com>.
Harsh,
My topology file looks similar as following:
192.168.2.20 host1.example.com host1 /tc/rack1
192.168.2.21 host2.example.com host2 /tc/rack1
192.168.2.22 host3.example.com host3 /tc/rack1
192.168.2.23 host4.example.com host4 /tc/rack2
192.168.2.24 host5.example.com host5 /tc/rack2
If I am not wrong all hosts have the same levers.
Can you please let me know how to including the default rack value used
upon a mismatch?
Thanks,
Sandeep.
________________________________
From: Harsh J <ha...@cloudera.com>
To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy <la...@yahoo.com>
Sent: Friday, April 12, 2013 3:22 PM
Subject: Re: Map tasks never getting started
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J
Re: Map tasks never getting started
Posted by sandeep reddy <la...@yahoo.com>.
Harsh,
My topology file looks similar as following:
192.168.2.20 host1.example.com host1 /tc/rack1
192.168.2.21 host2.example.com host2 /tc/rack1
192.168.2.22 host3.example.com host3 /tc/rack1
192.168.2.23 host4.example.com host4 /tc/rack2
192.168.2.24 host5.example.com host5 /tc/rack2
If I am not wrong all hosts have the same levers.
Can you please let me know how to including the default rack value used
upon a mismatch?
Thanks,
Sandeep.
________________________________
From: Harsh J <ha...@cloudera.com>
To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy <la...@yahoo.com>
Sent: Friday, April 12, 2013 3:22 PM
Subject: Re: Map tasks never getting started
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J
Re: Map tasks never getting started
Posted by sandeep reddy <la...@yahoo.com>.
Harsh,
My topology file looks similar as following:
192.168.2.20 host1.example.com host1 /tc/rack1
192.168.2.21 host2.example.com host2 /tc/rack1
192.168.2.22 host3.example.com host3 /tc/rack1
192.168.2.23 host4.example.com host4 /tc/rack2
192.168.2.24 host5.example.com host5 /tc/rack2
If I am not wrong all hosts have the same levers.
Can you please let me know how to including the default rack value used
upon a mismatch?
Thanks,
Sandeep.
________________________________
From: Harsh J <ha...@cloudera.com>
To: "<us...@hadoop.apache.org>" <us...@hadoop.apache.org>; sandeep reddy <la...@yahoo.com>
Sent: Friday, April 12, 2013 3:22 PM
Subject: Re: Map tasks never getting started
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J
Re: Map tasks never getting started
Posted by Harsh J <ha...@cloudera.com>.
Your rack topology configuration is broken (mismatching levels) and
you're therefore hitting MAPREDUCE-1740. Either upgrade to a more
current release, or fix your rack configuration to be of the same
levels for all mapped nodes (including the default rack value used
upon a mismatch) and restart the JobTracker.
On Fri, Apr 12, 2013 at 3:15 PM, sandeep reddy
<la...@yahoo.com> wrote:
> Hi,
>
> While running a mapred job map task never getting assigned. The message for
> a map in web interface is "No Task Attempts found".
> When I check the logs there are couple of errors as following:
>
> 2013-04-12 02:29:04,454 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:jobs cause:java.io.IOException: java.lang.NullPointerException
> 2013-04-12 02:29:04,455 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 4571, call
> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@8f9c09e, false, false,
> true, 4) from 192.168.2.20:59968: error: java.io.IOException:
> java.lang.NullPointerException
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.JobInProgress.getMatchingLevelForNodes(JobInProgress.java:1699)
> at
> org.apache.hadoop.mapred.JobInProgress.addRunningTaskToTIP(JobInProgress.java:1784)
> at
> org.apache.hadoop.mapred.JobInProgress.obtainNewNonLocalMapTask(JobInProgress.java:1440)
> at
> org.apache.hadoop.mapred.JobQueueTaskScheduler.assignTasks(JobQueueTaskScheduler.java:189)
> at
> org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3398)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
> 2013-04-12 02:29:04,463 INFO org.apache.hadoop.mapred.JobTracker: Removing
> task 'attempt_201304120228_0001_m_000002_0'
>
> Some please help me to resolve this issue.
> I am using hadoop-1.0.2
>
> Thanks,
> Sandeep.
--
Harsh J