You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Prasan Ary <vo...@yahoo.com> on 2008/03/08 00:50:02 UTC
Map/Reduce Type Mismatch error
Hi All,
I am running a Map/Reduce on a textfile.
Map takes <Text,Text> as (key,value) input pair , and outputs <Text,IntWritable> as (key,value) output pair.
Reduce takes <Text,IntWritable> as (key,value) input pair, and outputs <Text,Text> as (key,value) output pair.
I am getting a type mismatch error.
Any suggestion?
JobConf job = new JobConf(..............
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
-----------------------------
public static class Map extends MapReduceBase implements Mapper<Text, Text, Text, IntWritable> {
......
public void map(Text key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { ......
output.collect(key,new IntWritable(1));
----------------------------
public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, Text> {
public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text,Text> output, Reporter reporter) throws IOException { ..............
output.collect(key, new Text("SomeText");
---------------------------------
Never miss a thing. Make Yahoo your homepage.
Re: Map/Reduce Type Mismatch error
Posted by Amar Kamat <am...@yahoo-inc.com>.
Look at WordCount.java in src/examples/org/apache/hadoop/examples. Whether
you need a new InputFormat depends on what you want to do.
Amar
On Fri, 7 Mar 2008, Prasan Ary wrote:
> Hi All,
> I am running a Map/Reduce on a textfile.
> Map takes <Text,Text> as (key,value) input pair , and outputs <Text,IntWritable> as (key,value) output pair.
>
> Reduce takes <Text,IntWritable> as (key,value) input pair, and outputs <Text,Text> as (key,value) output pair.
>
> I am getting a type mismatch error.
>
> Any suggestion?
>
>
> JobConf job = new JobConf(..............
>
> job.setOutputKeyClass(Text.class);
> job.setOutputValueClass(Text.class);
>
> -----------------------------
> public static class Map extends MapReduceBase implements Mapper<Text, Text, Text, IntWritable> {
> ......
> public void map(Text key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { ......
>
> output.collect(key,new IntWritable(1));
>
> ----------------------------
>
> public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, Text> {
> public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text,Text> output, Reporter reporter) throws IOException { ..............
>
> output.collect(key, new Text("SomeText");
>
>
> ---------------------------------
> Never miss a thing. Make Yahoo your homepage.
RE: Map/Reduce Type Mismatch error
Posted by Jeff Eastman <je...@collab.net>.
The key provided by the default FileInputFormat is not Text, but an
integer offset into the split(which is not very usful IMHO). Try
changing your mapper back to <WritableComparable, Text>. If you are
expecting the file name to be the key, you will (I think) need to write
your own InputFormat.
Jeff
-----Original Message-----
From: Prasan Ary [mailto:voicesnthedark@yahoo.com]
Sent: Friday, March 07, 2008 3:50 PM
To: hadoop
Subject: Map/Reduce Type Mismatch error
Hi All,
I am running a Map/Reduce on a textfile.
Map takes <Text,Text> as (key,value) input pair , and outputs
<Text,IntWritable> as (key,value) output pair.
Reduce takes <Text,IntWritable> as (key,value) input pair, and outputs
<Text,Text> as (key,value) output pair.
I am getting a type mismatch error.
Any suggestion?
JobConf job = new JobConf(..............
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
-----------------------------
public static class Map extends MapReduceBase implements Mapper<Text,
Text, Text, IntWritable> {
......
public void map(Text key, Text value, OutputCollector<Text,
IntWritable> output, Reporter reporter) throws IOException { ......
output.collect(key,new IntWritable(1));
----------------------------
public static class Reduce extends MapReduceBase implements
Reducer<Text, IntWritable, Text, Text> {
public void reduce(Text key, Iterator<IntWritable> values,
OutputCollector<Text,Text> output, Reporter reporter) throws IOException
{ ..............
output.collect(key, new Text("SomeText");
---------------------------------
Never miss a thing. Make Yahoo your homepage.