You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Prasan Ary <vo...@yahoo.com> on 2008/03/08 00:50:02 UTC

Map/Reduce Type Mismatch error

  Hi All,
  I am running a Map/Reduce on a textfile.
  Map takes <Text,Text> as (key,value) input pair , and outputs <Text,IntWritable> as (key,value) output pair.
   
  Reduce takes <Text,IntWritable> as (key,value) input pair, and outputs <Text,Text> as (key,value) output pair.
   
  I am getting a type mismatch error.
   
  Any suggestion?
   
   
  JobConf job = new JobConf(..............
   
  job.setOutputKeyClass(Text.class); 
  job.setOutputValueClass(Text.class); 
   
  -----------------------------
  public static class Map extends MapReduceBase implements Mapper<Text, Text, Text, IntWritable> {
    ......
    public void map(Text key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { ......
   
  output.collect(key,new IntWritable(1));
   
  ----------------------------
   
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, Text> { 
  public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text,Text> output, Reporter reporter) throws IOException {  ..............
   
  output.collect(key, new Text("SomeText");

       
---------------------------------
Never miss a thing.   Make Yahoo your homepage.

Re: Map/Reduce Type Mismatch error

Posted by Amar Kamat <am...@yahoo-inc.com>.
Look at WordCount.java in src/examples/org/apache/hadoop/examples. Whether 
you need a new InputFormat depends on what you want to do.
Amar
On Fri, 7 Mar 2008, Prasan Ary wrote:

>  Hi All,
>  I am running a Map/Reduce on a textfile.
>  Map takes <Text,Text> as (key,value) input pair , and outputs <Text,IntWritable> as (key,value) output pair.
>
>  Reduce takes <Text,IntWritable> as (key,value) input pair, and outputs <Text,Text> as (key,value) output pair.
>
>  I am getting a type mismatch error.
>
>  Any suggestion?
>
>
>  JobConf job = new JobConf(..............
>
>  job.setOutputKeyClass(Text.class);
>  job.setOutputValueClass(Text.class);
>
>  -----------------------------
>  public static class Map extends MapReduceBase implements Mapper<Text, Text, Text, IntWritable> {
>    ......
>    public void map(Text key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { ......
>
>  output.collect(key,new IntWritable(1));
>
>  ----------------------------
>
>  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, Text> {
>  public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text,Text> output, Reporter reporter) throws IOException {  ..............
>
>  output.collect(key, new Text("SomeText");
>
>
> ---------------------------------
> Never miss a thing.   Make Yahoo your homepage.

RE: Map/Reduce Type Mismatch error

Posted by Jeff Eastman <je...@collab.net>.
The key provided by the default FileInputFormat is not Text, but an
integer offset into the split(which is not very usful IMHO). Try
changing your mapper back to <WritableComparable, Text>. If you are
expecting the file name to be the key, you will (I think) need to write
your own InputFormat.

Jeff

-----Original Message-----
From: Prasan Ary [mailto:voicesnthedark@yahoo.com] 
Sent: Friday, March 07, 2008 3:50 PM
To: hadoop
Subject: Map/Reduce Type Mismatch error

  Hi All,
  I am running a Map/Reduce on a textfile.
  Map takes <Text,Text> as (key,value) input pair , and outputs
<Text,IntWritable> as (key,value) output pair.
   
  Reduce takes <Text,IntWritable> as (key,value) input pair, and outputs
<Text,Text> as (key,value) output pair.
   
  I am getting a type mismatch error.
   
  Any suggestion?
   
   
  JobConf job = new JobConf(..............
   
  job.setOutputKeyClass(Text.class); 
  job.setOutputValueClass(Text.class); 
   
  -----------------------------
  public static class Map extends MapReduceBase implements Mapper<Text,
Text, Text, IntWritable> {
    ......
    public void map(Text key, Text value, OutputCollector<Text,
IntWritable> output, Reporter reporter) throws IOException { ......
   
  output.collect(key,new IntWritable(1));
   
  ----------------------------
   
  public static class Reduce extends MapReduceBase implements
Reducer<Text, IntWritable, Text, Text> { 
  public void reduce(Text key, Iterator<IntWritable> values,
OutputCollector<Text,Text> output, Reporter reporter) throws IOException
{  ..............
   
  output.collect(key, new Text("SomeText");

       
---------------------------------
Never miss a thing.   Make Yahoo your homepage.