You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by yz5od2 <wo...@yahoo.com> on 2009/10/15 21:39:22 UTC

newbie help, type mismatch in mapper

Hi,
I am new to Hadoop so this might be an easy question for someone to  
help me with.

I continually am getting this exception (my code follows below)

java.io.IOException: Type mismatch in key from map: expected  
org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable
	at org.apache.hadoop.mapred.MapTask 
$MapOutputBuffer.collect(MapTask.java:807)
	at org.apache.hadoop.mapred.MapTask 
$NewOutputCollector.write(MapTask.java:504)
	at  
org 
.apache 
.hadoop 
.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
	at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
	at org.apache.hadoop.mapred.Child.main(Child.java:170)


Running 0.20.1

I have a text file with lines of data separated by carriage returns.  
This is properly stored in a directory within HDFS. I only have a  
Mapping task for processing this file, after the mapping is done it  
should go straight to output, No reduce or combiner functions.

I am just trying to test to see if this will run. The mapper just  
takes the data sent to it and adds it back to the collector as 2 text  
values.

--------------------
My Mapper:
--------------------

public class MyTestMapper extends Mapper<LongWritable,Text,Text,Text> {

	public void map(Object key, Text value, Context context) throws  
IOException, InterruptedException {
		String key = key.toString();		
		String line = value.toString();

		String id = extractId(line);
		String reformattedLine = reformatLine(line);

		context.write(new Text(id), new Text(reformattedLine));
	}
}

--------------------
My job submission code:
-------------------------------------

Job job = new Job(conf);
job.setJarByClass(MyTestMapper.class);
job.setMapperClass(MyTestMapper.class);	

FileInputFormat.addInputPath(job, new Path("/myDir/sample.txt"));
FileOutputFormat.setOutputPath(job, new Path("/myDir/output/ 
results-"+System.currentTimeMillis()+".txt"));

job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);

job.submit();