You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by "Devireddy, Bhaskar" <Bh...@amd.com> on 2012/08/31 23:11:23 UTC

Shuffle BAD_ID errors

We are noticing shuffle BAD_ID errors using hadoop trunk and Yarn. The stack traces are as follows and any ideas why these exceptions are happening and how to fix them.



java.lang.IllegalArgumentException: Encoded byte size for String was -42, which is outside of 0..1000 range.

       at org.apache.hadoop.io.WritableUtils.readStringSafely(WritableUtils.java:477)

       at org.apache.hadoop.mapreduce.task.reduce.ShuffleHeader.readFields(ShuffleHeader.java:60)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:363)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)





java.lang.IllegalArgumentException: TaskAttemptId string : ï¿½ï¿½^Zï¿½ï¿½m%attempt_1346265984657_000 is not properly formed

       at org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)





java.lang.IllegalArgumentException: TaskAttemptId string : @ is not properly formed

       at org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)

       at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)



Thanks

Bhaskar Devireddy