You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Ramya S <ra...@suntecgroup.com> on 2013/07/12 11:16:21 UTC
Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
Re: Taktracker in namenode failure
Posted by Ram <pr...@gmail.com>.
Hi,
The problem is with jar file only, to check run any other MR job or
sample wordcount job on namenode tasktracker, if it is running no problem
with namenode tasktracker, if not running there may be problem with
tasktracker configuration, then compare with other node tasktracker
configuration. i.e tasktracker configuration means mapred configuration.
Hi,
From,
Ramesh.
On Fri, Jul 12, 2013 at 3:37 PM, Devaraj k <de...@huawei.com> wrote:
> I think, there is mismatch of jar’s coming in the classpath for the map
> tasks when it runs in different machines. You can find out this, by giving
> some unique name for your Mapper class, Job Submit class and then submit
> the Job.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com]
> *Sent:* 12 July 2013 15:27
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> ** **
>
> Both the map output value class configured and the output value written
> from the mapper is Text class. So there is no mismatch in the value class.
> ****
>
> ****
>
> But when the same MR program is run with 2 tasktrackers(without
> tasktracker in namenode) exception is not occuring.****
>
> ****
>
> The problem is only with the tasktracker running in the namenode.****
>
> ****
>
> ****
>
> ****
>
> *Thanks & Regards*****
>
> ****
>
> *Ramya.S*****
>
> ** **
> ------------------------------
>
> *From:* Devaraj k [mailto:devaraj.k@huawei.com <de...@huawei.com>]
> *Sent:* Fri 7/12/2013 3:04 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> Could you tell, what is the Map Output Value class you are configuring
> while submitting Job and what is the type of the value writing from the
> Mapper. If both of these mismatches then it will trow the below error.****
>
> ****
>
> Thanks****
>
> Devaraj k****
>
> ****
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com <ra...@suntecgroup.com>]
> *Sent:* 12 July 2013 14:46
> *To:* user@hadoop.apache.org
> *Subject:* Taktracker in namenode failure****
>
> ****
>
> Hi,****
>
> ****
>
> Why only tasktracker in namenode faill during job execution with error.**
> **
>
> I have attached the snapshot of error screen with this mail****
>
> java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable****
>
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)****
>
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)****
>
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:30)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:19)****
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)****
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)****
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)****
>
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at javax.security.auth.Subject.doAs(Subject.java:416)****
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)****
>
> at org.apache.hadoop.mapred.Child.main(Child.java:249)****
>
> ****
>
> but this same task is reassigned to another tasktracker and getting executed. why?****
>
> ****
>
> ****
>
> *Best Regards,*****
>
> *Ramya*****
>
Re: Taktracker in namenode failure
Posted by Ram <pr...@gmail.com>.
Hi,
The problem is with jar file only, to check run any other MR job or
sample wordcount job on namenode tasktracker, if it is running no problem
with namenode tasktracker, if not running there may be problem with
tasktracker configuration, then compare with other node tasktracker
configuration. i.e tasktracker configuration means mapred configuration.
Hi,
From,
Ramesh.
On Fri, Jul 12, 2013 at 3:37 PM, Devaraj k <de...@huawei.com> wrote:
> I think, there is mismatch of jar’s coming in the classpath for the map
> tasks when it runs in different machines. You can find out this, by giving
> some unique name for your Mapper class, Job Submit class and then submit
> the Job.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com]
> *Sent:* 12 July 2013 15:27
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> ** **
>
> Both the map output value class configured and the output value written
> from the mapper is Text class. So there is no mismatch in the value class.
> ****
>
> ****
>
> But when the same MR program is run with 2 tasktrackers(without
> tasktracker in namenode) exception is not occuring.****
>
> ****
>
> The problem is only with the tasktracker running in the namenode.****
>
> ****
>
> ****
>
> ****
>
> *Thanks & Regards*****
>
> ****
>
> *Ramya.S*****
>
> ** **
> ------------------------------
>
> *From:* Devaraj k [mailto:devaraj.k@huawei.com <de...@huawei.com>]
> *Sent:* Fri 7/12/2013 3:04 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> Could you tell, what is the Map Output Value class you are configuring
> while submitting Job and what is the type of the value writing from the
> Mapper. If both of these mismatches then it will trow the below error.****
>
> ****
>
> Thanks****
>
> Devaraj k****
>
> ****
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com <ra...@suntecgroup.com>]
> *Sent:* 12 July 2013 14:46
> *To:* user@hadoop.apache.org
> *Subject:* Taktracker in namenode failure****
>
> ****
>
> Hi,****
>
> ****
>
> Why only tasktracker in namenode faill during job execution with error.**
> **
>
> I have attached the snapshot of error screen with this mail****
>
> java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable****
>
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)****
>
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)****
>
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:30)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:19)****
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)****
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)****
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)****
>
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at javax.security.auth.Subject.doAs(Subject.java:416)****
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)****
>
> at org.apache.hadoop.mapred.Child.main(Child.java:249)****
>
> ****
>
> but this same task is reassigned to another tasktracker and getting executed. why?****
>
> ****
>
> ****
>
> *Best Regards,*****
>
> *Ramya*****
>
Re: Taktracker in namenode failure
Posted by Ram <pr...@gmail.com>.
Hi,
The problem is with jar file only, to check run any other MR job or
sample wordcount job on namenode tasktracker, if it is running no problem
with namenode tasktracker, if not running there may be problem with
tasktracker configuration, then compare with other node tasktracker
configuration. i.e tasktracker configuration means mapred configuration.
Hi,
From,
Ramesh.
On Fri, Jul 12, 2013 at 3:37 PM, Devaraj k <de...@huawei.com> wrote:
> I think, there is mismatch of jar’s coming in the classpath for the map
> tasks when it runs in different machines. You can find out this, by giving
> some unique name for your Mapper class, Job Submit class and then submit
> the Job.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com]
> *Sent:* 12 July 2013 15:27
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> ** **
>
> Both the map output value class configured and the output value written
> from the mapper is Text class. So there is no mismatch in the value class.
> ****
>
> ****
>
> But when the same MR program is run with 2 tasktrackers(without
> tasktracker in namenode) exception is not occuring.****
>
> ****
>
> The problem is only with the tasktracker running in the namenode.****
>
> ****
>
> ****
>
> ****
>
> *Thanks & Regards*****
>
> ****
>
> *Ramya.S*****
>
> ** **
> ------------------------------
>
> *From:* Devaraj k [mailto:devaraj.k@huawei.com <de...@huawei.com>]
> *Sent:* Fri 7/12/2013 3:04 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> Could you tell, what is the Map Output Value class you are configuring
> while submitting Job and what is the type of the value writing from the
> Mapper. If both of these mismatches then it will trow the below error.****
>
> ****
>
> Thanks****
>
> Devaraj k****
>
> ****
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com <ra...@suntecgroup.com>]
> *Sent:* 12 July 2013 14:46
> *To:* user@hadoop.apache.org
> *Subject:* Taktracker in namenode failure****
>
> ****
>
> Hi,****
>
> ****
>
> Why only tasktracker in namenode faill during job execution with error.**
> **
>
> I have attached the snapshot of error screen with this mail****
>
> java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable****
>
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)****
>
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)****
>
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:30)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:19)****
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)****
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)****
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)****
>
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at javax.security.auth.Subject.doAs(Subject.java:416)****
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)****
>
> at org.apache.hadoop.mapred.Child.main(Child.java:249)****
>
> ****
>
> but this same task is reassigned to another tasktracker and getting executed. why?****
>
> ****
>
> ****
>
> *Best Regards,*****
>
> *Ramya*****
>
Re: Taktracker in namenode failure
Posted by Ram <pr...@gmail.com>.
Hi,
The problem is with jar file only, to check run any other MR job or
sample wordcount job on namenode tasktracker, if it is running no problem
with namenode tasktracker, if not running there may be problem with
tasktracker configuration, then compare with other node tasktracker
configuration. i.e tasktracker configuration means mapred configuration.
Hi,
From,
Ramesh.
On Fri, Jul 12, 2013 at 3:37 PM, Devaraj k <de...@huawei.com> wrote:
> I think, there is mismatch of jar’s coming in the classpath for the map
> tasks when it runs in different machines. You can find out this, by giving
> some unique name for your Mapper class, Job Submit class and then submit
> the Job.****
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> ** **
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com]
> *Sent:* 12 July 2013 15:27
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> ** **
>
> Both the map output value class configured and the output value written
> from the mapper is Text class. So there is no mismatch in the value class.
> ****
>
> ****
>
> But when the same MR program is run with 2 tasktrackers(without
> tasktracker in namenode) exception is not occuring.****
>
> ****
>
> The problem is only with the tasktracker running in the namenode.****
>
> ****
>
> ****
>
> ****
>
> *Thanks & Regards*****
>
> ****
>
> *Ramya.S*****
>
> ** **
> ------------------------------
>
> *From:* Devaraj k [mailto:devaraj.k@huawei.com <de...@huawei.com>]
> *Sent:* Fri 7/12/2013 3:04 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: Taktracker in namenode failure****
>
> Could you tell, what is the Map Output Value class you are configuring
> while submitting Job and what is the type of the value writing from the
> Mapper. If both of these mismatches then it will trow the below error.****
>
> ****
>
> Thanks****
>
> Devaraj k****
>
> ****
>
> *From:* Ramya S [mailto:ramyas@suntecgroup.com <ra...@suntecgroup.com>]
> *Sent:* 12 July 2013 14:46
> *To:* user@hadoop.apache.org
> *Subject:* Taktracker in namenode failure****
>
> ****
>
> Hi,****
>
> ****
>
> Why only tasktracker in namenode faill during job execution with error.**
> **
>
> I have attached the snapshot of error screen with this mail****
>
> java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable****
>
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)****
>
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)****
>
> at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:30)****
>
> at WordCount$TokenizerMapper.map(WordCount.java:19)****
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)****
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)****
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)****
>
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at javax.security.auth.Subject.doAs(Subject.java:416)****
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)****
>
> at org.apache.hadoop.mapred.Child.main(Child.java:249)****
>
> ****
>
> but this same task is reassigned to another tasktracker and getting executed. why?****
>
> ****
>
> ****
>
> *Best Regards,*****
>
> *Ramya*****
>
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
I think, there is mismatch of jar’s coming in the classpath for the map tasks when it runs in different machines. You can find out this, by giving some unique name for your Mapper class, Job Submit class and then submit the Job.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 15:27
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
I think, there is mismatch of jar’s coming in the classpath for the map tasks when it runs in different machines. You can find out this, by giving some unique name for your Mapper class, Job Submit class and then submit the Job.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 15:27
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
I think, there is mismatch of jar’s coming in the classpath for the map tasks when it runs in different machines. You can find out this, by giving some unique name for your Mapper class, Job Submit class and then submit the Job.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 15:27
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
I think, there is mismatch of jar’s coming in the classpath for the map tasks when it runs in different machines. You can find out this, by giving some unique name for your Mapper class, Job Submit class and then submit the Job.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 15:27
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Ramya S <ra...@suntecgroup.com>.
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Ramya S <ra...@suntecgroup.com>.
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Ramya S <ra...@suntecgroup.com>.
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Ramya S <ra...@suntecgroup.com>.
Both the map output value class configured and the output value written from the mapper is Text class. So there is no mismatch in the value class.
But when the same MR program is run with 2 tasktrackers(without tasktracker in namenode) exception is not occuring.
The problem is only with the tasktracker running in the namenode.
Thanks & Regards
Ramya.S
________________________________
From: Devaraj k [mailto:devaraj.k@huawei.com]
Sent: Fri 7/12/2013 3:04 PM
To: user@hadoop.apache.org
Subject: RE: Taktracker in namenode failure
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya
RE: Taktracker in namenode failure
Posted by Devaraj k <de...@huawei.com>.
Could you tell, what is the Map Output Value class you are configuring while submitting Job and what is the type of the value writing from the Mapper. If both of these mismatches then it will trow the below error.
Thanks
Devaraj k
From: Ramya S [mailto:ramyas@suntecgroup.com]
Sent: 12 July 2013 14:46
To: user@hadoop.apache.org
Subject: Taktracker in namenode failure
Hi,
Why only tasktracker in namenode faill during job execution with error.
I have attached the snapshot of error screen with this mail
java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.IntWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1019)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at WordCount$TokenizerMapper.map(WordCount.java:30)
at WordCount$TokenizerMapper.map(WordCount.java:19)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
but this same task is reassigned to another tasktracker and getting executed. why?
Best Regards,
Ramya