You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by sam liu <sa...@gmail.com> on 2013/09/12 03:37:12 UTC
ArrayIndexOutOfBoundsException occurs from MapTask
Hi,
We are using Hadoop-1.1.1. When run a big workload mapreduce application,
it threw a ArrayIndexOutOfBoundsException as below. Seems the issue was
threw in line 'System.arraycopy(b, off, kvbuffer, bufindex, gaplen);' in
MapTask. I am not familiar with the code of MapTask.java, and want to know
which situations will cause this kind of exception?
Caused by: java.lang.ArrayIndexOutOfBoundsException
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
at java.io.DataOutputStream.write(DataOutputStream.java:82)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
Thanks!
Re: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by sam liu <sa...@gmail.com>.
Thanks for your information!
However, if so, it should be a Hadoop bug, because there is no restrictions
on setting the value to "mapreduce.task.io.sort.mb"
2013/9/12 Brahma Reddy Battula <br...@huawei.com>
> I had also faced same issue, after chnaging "*mapreduce.task.io.sort.mb*"
> from 2047 to 512, then jobs are executed success
>
>
>
> It might help in your case also..
>
>
>
> ------------------------------
>
> *From:* sam liu [samliuhadoop@gmail.com]
> *Sent:* Thursday, September 12, 2013 9:37 AM
> *To:* user@hadoop.apache.org
> *Subject:* ArrayIndexOutOfBoundsException occurs from MapTask
>
> Hi,
>
> We are using Hadoop-1.1.1. When run a big workload mapreduce
> application, it threw a ArrayIndexOutOfBoundsException as below. Seems the
> issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex,
> gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and
> want to know which situations will cause this kind of exception?
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
> at java.io.DataOutputStream.write(DataOutputStream.java:82)
> at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
>
> Thanks!
>
Re: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by sam liu <sa...@gmail.com>.
Thanks for your information!
However, if so, it should be a Hadoop bug, because there is no restrictions
on setting the value to "mapreduce.task.io.sort.mb"
2013/9/12 Brahma Reddy Battula <br...@huawei.com>
> I had also faced same issue, after chnaging "*mapreduce.task.io.sort.mb*"
> from 2047 to 512, then jobs are executed success
>
>
>
> It might help in your case also..
>
>
>
> ------------------------------
>
> *From:* sam liu [samliuhadoop@gmail.com]
> *Sent:* Thursday, September 12, 2013 9:37 AM
> *To:* user@hadoop.apache.org
> *Subject:* ArrayIndexOutOfBoundsException occurs from MapTask
>
> Hi,
>
> We are using Hadoop-1.1.1. When run a big workload mapreduce
> application, it threw a ArrayIndexOutOfBoundsException as below. Seems the
> issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex,
> gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and
> want to know which situations will cause this kind of exception?
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
> at java.io.DataOutputStream.write(DataOutputStream.java:82)
> at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
>
> Thanks!
>
Re: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by sam liu <sa...@gmail.com>.
Thanks for your information!
However, if so, it should be a Hadoop bug, because there is no restrictions
on setting the value to "mapreduce.task.io.sort.mb"
2013/9/12 Brahma Reddy Battula <br...@huawei.com>
> I had also faced same issue, after chnaging "*mapreduce.task.io.sort.mb*"
> from 2047 to 512, then jobs are executed success
>
>
>
> It might help in your case also..
>
>
>
> ------------------------------
>
> *From:* sam liu [samliuhadoop@gmail.com]
> *Sent:* Thursday, September 12, 2013 9:37 AM
> *To:* user@hadoop.apache.org
> *Subject:* ArrayIndexOutOfBoundsException occurs from MapTask
>
> Hi,
>
> We are using Hadoop-1.1.1. When run a big workload mapreduce
> application, it threw a ArrayIndexOutOfBoundsException as below. Seems the
> issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex,
> gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and
> want to know which situations will cause this kind of exception?
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
> at java.io.DataOutputStream.write(DataOutputStream.java:82)
> at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
>
> Thanks!
>
Re: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by sam liu <sa...@gmail.com>.
Thanks for your information!
However, if so, it should be a Hadoop bug, because there is no restrictions
on setting the value to "mapreduce.task.io.sort.mb"
2013/9/12 Brahma Reddy Battula <br...@huawei.com>
> I had also faced same issue, after chnaging "*mapreduce.task.io.sort.mb*"
> from 2047 to 512, then jobs are executed success
>
>
>
> It might help in your case also..
>
>
>
> ------------------------------
>
> *From:* sam liu [samliuhadoop@gmail.com]
> *Sent:* Thursday, September 12, 2013 9:37 AM
> *To:* user@hadoop.apache.org
> *Subject:* ArrayIndexOutOfBoundsException occurs from MapTask
>
> Hi,
>
> We are using Hadoop-1.1.1. When run a big workload mapreduce
> application, it threw a ArrayIndexOutOfBoundsException as below. Seems the
> issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex,
> gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and
> want to know which situations will cause this kind of exception?
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
> at java.io.DataOutputStream.write(DataOutputStream.java:82)
> at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
>
> Thanks!
>
RE: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by Brahma Reddy Battula <br...@huawei.com>.
I had also faced same issue, after chnaging "mapreduce.task.io.sort.mb" from 2047 to 512, then jobs are executed success
It might help in your case also..
________________________________
From: sam liu [samliuhadoop@gmail.com]
Sent: Thursday, September 12, 2013 9:37 AM
To: user@hadoop.apache.org
Subject: ArrayIndexOutOfBoundsException occurs from MapTask
Hi,
We are using Hadoop-1.1.1. When run a big workload mapreduce application, it threw a ArrayIndexOutOfBoundsException as below. Seems the issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex, gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and want to know which situations will cause this kind of exception?
Caused by: java.lang.ArrayIndexOutOfBoundsException
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
at java.io.DataOutputStream.write(DataOutputStream.java:82)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
Thanks!
RE: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by Brahma Reddy Battula <br...@huawei.com>.
I had also faced same issue, after chnaging "mapreduce.task.io.sort.mb" from 2047 to 512, then jobs are executed success
It might help in your case also..
________________________________
From: sam liu [samliuhadoop@gmail.com]
Sent: Thursday, September 12, 2013 9:37 AM
To: user@hadoop.apache.org
Subject: ArrayIndexOutOfBoundsException occurs from MapTask
Hi,
We are using Hadoop-1.1.1. When run a big workload mapreduce application, it threw a ArrayIndexOutOfBoundsException as below. Seems the issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex, gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and want to know which situations will cause this kind of exception?
Caused by: java.lang.ArrayIndexOutOfBoundsException
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
at java.io.DataOutputStream.write(DataOutputStream.java:82)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
Thanks!
RE: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by Brahma Reddy Battula <br...@huawei.com>.
I had also faced same issue, after chnaging "mapreduce.task.io.sort.mb" from 2047 to 512, then jobs are executed success
It might help in your case also..
________________________________
From: sam liu [samliuhadoop@gmail.com]
Sent: Thursday, September 12, 2013 9:37 AM
To: user@hadoop.apache.org
Subject: ArrayIndexOutOfBoundsException occurs from MapTask
Hi,
We are using Hadoop-1.1.1. When run a big workload mapreduce application, it threw a ArrayIndexOutOfBoundsException as below. Seems the issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex, gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and want to know which situations will cause this kind of exception?
Caused by: java.lang.ArrayIndexOutOfBoundsException
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
at java.io.DataOutputStream.write(DataOutputStream.java:82)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
Thanks!
RE: ArrayIndexOutOfBoundsException occurs from MapTask
Posted by Brahma Reddy Battula <br...@huawei.com>.
I had also faced same issue, after chnaging "mapreduce.task.io.sort.mb" from 2047 to 512, then jobs are executed success
It might help in your case also..
________________________________
From: sam liu [samliuhadoop@gmail.com]
Sent: Thursday, September 12, 2013 9:37 AM
To: user@hadoop.apache.org
Subject: ArrayIndexOutOfBoundsException occurs from MapTask
Hi,
We are using Hadoop-1.1.1. When run a big workload mapreduce application, it threw a ArrayIndexOutOfBoundsException as below. Seems the issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex, gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and want to know which situations will cause this kind of exception?
Caused by: java.lang.ArrayIndexOutOfBoundsException
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
at java.io.DataOutputStream.write(DataOutputStream.java:82)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)
Thanks!