You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Sowmya Ramesh <sr...@hortonworks.com> on 2015/09/12 00:15:56 UTC
Review Request 38323: BigDecimalSplitter
java.lang.ArrayIndexOutOfBoundsException
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38323/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-2399
https://issues.apache.org/jira/browse/SQOOP-2399
Repository: sqoop-trunk
Description
-------
With a relatively small table and a text primary key this error happens:
15/06/14 03:00:25 WARN db.BigDecimalSplitter: Set BigDecimal splitSize to MIN_INCREMENT
15/06/14 03:00:25 INFO mapred.JobClient: Cleaning up the staging area maprfs:/var/mapr/cluster/mapred/jobTracker/staging/mapr/.staging/job_201404251052_5654
15/06/14 03:00:25 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.ArrayIndexOutOfBoundsException: -1
java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.elementData(ArrayList.java:371)
at java.util.ArrayList.get(ArrayList.java:384)
at org.apache.sqoop.mapreduce.db.BigDecimalSplitter.split(BigDecimalSplitter.java:143)
at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:151)
at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:113)
Diffs
-----
src/java/org/apache/sqoop/mapreduce/db/BigDecimalSplitter.java ebe6c40
src/java/org/apache/sqoop/mapreduce/db/DBSplitter.java b121d4b
src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java db96e41
src/java/org/apache/sqoop/mapreduce/db/TextSplitter.java d3085cd
src/test/org/apache/sqoop/mapreduce/db/TestTextSplitter.java 9c007d3
Diff: https://reviews.apache.org/r/38323/diff/
Testing
-------
Unit test
Thanks,
Sowmya Ramesh
Re: Review Request 38323: BigDecimalSplitter
java.lang.ArrayIndexOutOfBoundsException
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38323/#review99586
-----------------------------------------------------------
Ship it!
Will fix the comment before committing.
- Venkat Ranganathan
On Sept. 18, 2015, 11:17 a.m., Sowmya Ramesh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38323/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2015, 11:17 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2399
> https://issues.apache.org/jira/browse/SQOOP-2399
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> With a relatively small table and a text primary key this error happens:
>
> 15/06/14 03:00:25 WARN db.BigDecimalSplitter: Set BigDecimal splitSize to MIN_INCREMENT
> 15/06/14 03:00:25 INFO mapred.JobClient: Cleaning up the staging area maprfs:/var/mapr/cluster/mapred/jobTracker/staging/mapr/.staging/job_201404251052_5654
> 15/06/14 03:00:25 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.ArrayIndexOutOfBoundsException: -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:371)
> at java.util.ArrayList.get(ArrayList.java:384)
> at org.apache.sqoop.mapreduce.db.BigDecimalSplitter.split(BigDecimalSplitter.java:143)
> at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:151)
> at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:113)
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/mapreduce/db/BigDecimalSplitter.java 7e3048a
> src/java/org/apache/sqoop/mapreduce/db/DBSplitter.java b121d4b
> src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java 2c59fe5
> src/java/org/apache/sqoop/mapreduce/db/TextSplitter.java d3085cd
> src/test/org/apache/sqoop/mapreduce/db/TestTextSplitter.java 9c007d3
>
> Diff: https://reviews.apache.org/r/38323/diff/
>
>
> Testing
> -------
>
> Unit test
>
>
> Thanks,
>
> Sowmya Ramesh
>
>
Re: Review Request 38323: BigDecimalSplitter
java.lang.ArrayIndexOutOfBoundsException
Posted by Sowmya Ramesh <sr...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38323/
-----------------------------------------------------------
(Updated Sept. 18, 2015, 6:17 p.m.)
Review request for Sqoop.
Changes
-------
Applied feedback!
Bugs: SQOOP-2399
https://issues.apache.org/jira/browse/SQOOP-2399
Repository: sqoop-trunk
Description
-------
With a relatively small table and a text primary key this error happens:
15/06/14 03:00:25 WARN db.BigDecimalSplitter: Set BigDecimal splitSize to MIN_INCREMENT
15/06/14 03:00:25 INFO mapred.JobClient: Cleaning up the staging area maprfs:/var/mapr/cluster/mapred/jobTracker/staging/mapr/.staging/job_201404251052_5654
15/06/14 03:00:25 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.ArrayIndexOutOfBoundsException: -1
java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.elementData(ArrayList.java:371)
at java.util.ArrayList.get(ArrayList.java:384)
at org.apache.sqoop.mapreduce.db.BigDecimalSplitter.split(BigDecimalSplitter.java:143)
at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:151)
at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:113)
Diffs (updated)
-----
src/java/org/apache/sqoop/mapreduce/db/BigDecimalSplitter.java 7e3048a
src/java/org/apache/sqoop/mapreduce/db/DBSplitter.java b121d4b
src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java 2c59fe5
src/java/org/apache/sqoop/mapreduce/db/TextSplitter.java d3085cd
src/test/org/apache/sqoop/mapreduce/db/TestTextSplitter.java 9c007d3
Diff: https://reviews.apache.org/r/38323/diff/
Testing
-------
Unit test
Thanks,
Sowmya Ramesh
Re: Review Request 38323: BigDecimalSplitter
java.lang.ArrayIndexOutOfBoundsException
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38323/#review99141
-----------------------------------------------------------
Thanks Sowmya Ramesh for working on this. Just one small nit
src/java/org/apache/sqoop/mapreduce/db/BigDecimalSplitter.java (line 144)
<https://reviews.apache.org/r/38323/#comment156026>
Can you add a comment saying when splits.size can be 1? Essentially if the collation and sort orders differ we can hit this
- Venkat Ranganathan
On Sept. 11, 2015, 3:15 p.m., Sowmya Ramesh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38323/
> -----------------------------------------------------------
>
> (Updated Sept. 11, 2015, 3:15 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2399
> https://issues.apache.org/jira/browse/SQOOP-2399
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> With a relatively small table and a text primary key this error happens:
>
> 15/06/14 03:00:25 WARN db.BigDecimalSplitter: Set BigDecimal splitSize to MIN_INCREMENT
> 15/06/14 03:00:25 INFO mapred.JobClient: Cleaning up the staging area maprfs:/var/mapr/cluster/mapred/jobTracker/staging/mapr/.staging/job_201404251052_5654
> 15/06/14 03:00:25 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.ArrayIndexOutOfBoundsException: -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:371)
> at java.util.ArrayList.get(ArrayList.java:384)
> at org.apache.sqoop.mapreduce.db.BigDecimalSplitter.split(BigDecimalSplitter.java:143)
> at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:151)
> at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:113)
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/mapreduce/db/BigDecimalSplitter.java ebe6c40
> src/java/org/apache/sqoop/mapreduce/db/DBSplitter.java b121d4b
> src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java db96e41
> src/java/org/apache/sqoop/mapreduce/db/TextSplitter.java d3085cd
> src/test/org/apache/sqoop/mapreduce/db/TestTextSplitter.java 9c007d3
>
> Diff: https://reviews.apache.org/r/38323/diff/
>
>
> Testing
> -------
>
> Unit test
>
>
> Thanks,
>
> Sowmya Ramesh
>
>