MultithreadedMapRunner: java.lang.ArrayIndexOutOfBoundsException

[Guo Leitao <le...@gmail.com>]

Hi all,

I just try to use MultithreadedMapRunner for wordcount with 2 threads in one
maptask, but each maptask gets the following exception, and the job failed
eventually. But when running with only 1 thread, there is no such error.

java.lang.ArrayIndexOutOfBoundsException
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1122)
	at java.io.DataOutputStream.write(DataOutputStream.java:90)
	at org.apache.hadoop.io.Text.write(Text.java:282)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:906)
	at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:518)
	at WordCount$Map.map(WordCount.java:25)
	at WordCount$Map.map(WordCount.java:1)
	at org.apache.hadoop.mapred.lib.MultithreadedMapRunner$MapperInvokeRunable.run(MultithreadedMapRunner.java:229)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)


Version: hadoop-0.20.1
OS: CentOS 5.2 (64bit)
Java HotSpot(TM) 64-Bit Server VM (build 14.2-b01, mixed mode)

with thanks, Leitao