You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Devireddy, Bhaskar" <Bh...@amd.com> on 2012/08/30 20:27:51 UTC

Shuffle BAD_ID errors

We are noticing shuffle BAD_ID errors using hadoop trunk and Yarn. The stack traces are as follows and any ideas why these exceptions are happening and how to fix them.

java.lang.IllegalArgumentException: Encoded byte size for String was -42, which is outside of 0..1000 range.
       at org.apache.hadoop.io.WritableUtils.readStringSafely(WritableUtils.java:477)
       at org.apache.hadoop.mapreduce.task.reduce.ShuffleHeader.readFields(ShuffleHeader.java:60)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:363)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)


java.lang.IllegalArgumentException: TaskAttemptId string : ï¿½ï¿½^Zï¿½ï¿½m%attempt_1346265984657_000 is not properly formed
       at org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)


java.lang.IllegalArgumentException: TaskAttemptId string : @ is not properly formed
       at org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)
       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)

Thanks
Bhaskar Devireddy