You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Micah Kornfield (Jira)" <ji...@apache.org> on 2020/10/22 04:22:00 UTC
[jira] [Resolved] (ARROW-10153) [Java] Adding values to
VarCharVector beyond 2GB results in IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/ARROW-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Micah Kornfield resolved ARROW-10153.
-------------------------------------
Resolution: Not A Problem
Closing as this was intended behavior and all questions have been answered.
> [Java] Adding values to VarCharVector beyond 2GB results in IndexOutOfBoundsException
> -------------------------------------------------------------------------------------
>
> Key: ARROW-10153
> URL: https://issues.apache.org/jira/browse/ARROW-10153
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
> Affects Versions: 1.0.0
> Reporter: Samarth Jain
> Priority: Major
>
> On executing the below test case, one can see that on adding the 2049th string of size 1MB, it fails.
> {code:java}
> int length = 1024 * 1024;
> StringBuilder sb = new StringBuilder(length);
> for (int i = 0; i < length; i++) {
> sb.append("a");
> }
> byte[] str = sb.toString().getBytes();
> VarCharVector vector = new VarCharVector("v", new RootAllocator(Long.MAX_VALUE));
> vector.allocateNew(3000);
> for (int i = 0; i < 3000; i++) {
> vector.setSafe(i, str);
> }{code}
>
> {code:java}
> Exception in thread "main" java.lang.IndexOutOfBoundsException: index: -2147483648, length: 1048576 (expected: range(0, 2147483648))Exception in thread "main" java.lang.IndexOutOfBoundsException: index: -2147483648, length: 1048576 (expected: range(0, 2147483648)) at org.apache.arrow.memory.ArrowBuf.checkIndex(ArrowBuf.java:699) at org.apache.arrow.memory.ArrowBuf.setBytes(ArrowBuf.java:762) at org.apache.arrow.vector.BaseVariableWidthVector.setBytes(BaseVariableWidthVector.java:1212) at org.apache.arrow.vector.BaseVariableWidthVector.setSafe(BaseVariableWidthVector.java:1011)
> {code}
> Stepping through the code,
> [https://github.com/apache/arrow/blob/master/java/memory/memory-core/src/main/java/org/apache/arrow/memory/ArrowBuf.java#L425]
> returns the negative index `-2147483648`
--
This message was sent by Atlassian Jira
(v8.3.4#803005)