You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by Ted Yu <yu...@gmail.com> on 2016/09/14 22:36:44 UTC
enforcing value class type
Hi,
I have a question on the following check in
MapTask$MapOutputBuffer.collect():
if (value.getClass() != valClass) {
throw new IOException("Type mismatch in value from map: expected "
+ valClass.getName() + ", received "
+ value.getClass().getName());
}
See an example of the above in the following:
2016-09-14 22:20:09,643 WARN [Thread-1538] mapred.LocalJobRunner$Job(560):
job_local579834479_0004
java.lang.Exception: java.io.IOException: Type mismatch in value from map:
expected org.apache.hadoop.hbase.KeyValue, received
org.apache.hadoop.hbase.SizeCachedNoTagsKeyValue
at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in value from map: expected
org.apache.hadoop.hbase.KeyValue, received
org.apache.hadoop.hbase.SizeCachedNoTagsKeyValue
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1079)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:715)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.hadoop.hbase.backup.mapreduce.HFileSplitter$HFileCellMapper.map(HFileSplitter.java:93)
at
org.apache.hadoop.hbase.backup.mapreduce.HFileSplitter$HFileCellMapper.map(HFileSplitter.java:77)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
where:
public class SizeCachedNoTagsKeyValue extends SizeCachedKeyValue {
public class SizeCachedKeyValue extends KeyValue {
Can someone highlight why subclass of valClass is rejected by the check ?
Thanks