You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Litchy Soong (Jira)" <ji...@apache.org> on 2020/09/14 03:06:00 UTC
[jira] [Updated] (ARROW-9989) Arrow
[ https://issues.apache.org/jira/browse/ARROW-9989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Litchy Soong updated ARROW-9989:
--------------------------------
Description:
In scala, (Java of Arrow), following code work
{quote}object A {
def write(): = {
val vectorSchemaRoot = VectorSchemaRoot.create(getSchema, allocator) val writer = new ArrowStreamWriter(vectorSchemaRoot, null, out) }
}
{quote}
But following does not work
{quote}object A {
var vectorSchemaRoot: VectorSchemaRoot = null
var writer: ArrowStreamWriter = null
def write(): = {
vectorSchemaRoot = VectorSchemaRoot.create(getSchema, allocator) writer = new ArrowStreamWriter(vectorSchemaRoot, null, out) }
}
{quote}
The error is ,
{quote}java.lang.IllegalStateException: wrong buffer size: 601 != 4081java.lang.IllegalStateException: wrong buffer size: 601 != 4081 at org.apache.arrow.vector.ipc.message.MessageSerializer.writeBatchBuffers(MessageSerializer.java:297) at org.apache.arrow.vector.ipc.message.MessageSerializer.serialize(MessageSerializer.java:267) at org.apache.arrow.vector.ipc.ArrowWriter.writeRecordBatch(ArrowWriter.java:132) at org.apache.arrow.vector.ipc.ArrowWriter.writeBatch(ArrowWriter.java:120)
java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0)) at io.netty.buffer.ArrowBuf.checkIndexD(ArrowBuf.java:337) at io.netty.buffer.ArrowBuf.chk(ArrowBuf.java:324) at io.netty.buffer.ArrowBuf.getByte(ArrowBuf.java:526) at org.apache.arrow.vector.BitVectorHelper.setBit(BitVectorHelper.java:70) at org.apache.arrow.vector.Float4Vector.set(Float4Vector.java:168)
java.lang.IllegalStateException: RefCnt has gone negativejava.lang.IllegalStateException: RefCnt has gone negative at org.apache.arrow.util.Preconditions.checkState(Preconditions.java:458) at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:134) at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:108) at org.apache.arrow.vector.BaseFixedWidthVector.reAlloc(BaseFixedWidthVector.java:441) at org.apache.arrow.vector.BaseFixedWidthVector.setValueCount(BaseFixedWidthVector.java:708) at org.apache.arrow.vector.VectorSchemaRoot.setRowCount(VectorSchemaRoot.java:226)
{quote}
and it is raised every second time when I call the method. And seems both ArrowStreamWriter and VectorSchemaRoot could not be initialized in this way. Why?
was:
In scala, (Java of Arrow), following code work
{quote}object A {
def write(): = {
val vectorSchemaRoot = VectorSchemaRoot.create(getSchema, allocator)
val writer = new ArrowStreamWriter(vectorSchemaRoot, null, out)
}
}
{quote}
But following does not work
{quote}object A {
var vectorSchemaRoot: VectorSchemaRoot = null
var writer: ArrowStreamWriter = null
def write(): = {
vectorSchemaRoot = VectorSchemaRoot.create(getSchema, allocator)
writer = new ArrowStreamWriter(vectorSchemaRoot, null, out)
}
}
{quote}
The error is ,
{quote}java.lang.IllegalStateException: wrong buffer size: 601 != 4081java.lang.IllegalStateException: wrong buffer size: 601 != 4081 at org.apache.arrow.vector.ipc.message.MessageSerializer.writeBatchBuffers(MessageSerializer.java:297) at org.apache.arrow.vector.ipc.message.MessageSerializer.serialize(MessageSerializer.java:267) at org.apache.arrow.vector.ipc.ArrowWriter.writeRecordBatch(ArrowWriter.java:132) at org.apache.arrow.vector.ipc.ArrowWriter.writeBatch(ArrowWriter.java:120)
java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0)) at io.netty.buffer.ArrowBuf.checkIndexD(ArrowBuf.java:337) at io.netty.buffer.ArrowBuf.chk(ArrowBuf.java:324) at io.netty.buffer.ArrowBuf.getByte(ArrowBuf.java:526) at org.apache.arrow.vector.BitVectorHelper.setBit(BitVectorHelper.java:70) at org.apache.arrow.vector.Float4Vector.set(Float4Vector.java:168)
java.lang.IllegalStateException: RefCnt has gone negativejava.lang.IllegalStateException: RefCnt has gone negative at org.apache.arrow.util.Preconditions.checkState(Preconditions.java:458) at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:134) at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:108) at org.apache.arrow.vector.BaseFixedWidthVector.reAlloc(BaseFixedWidthVector.java:441) at org.apache.arrow.vector.BaseFixedWidthVector.setValueCount(BaseFixedWidthVector.java:708) at org.apache.arrow.vector.VectorSchemaRoot.setRowCount(VectorSchemaRoot.java:226)
{quote}
and it is raised every second time when I call the method. And seems both ArrowStreamWriter and VectorSchemaRoot could not be initialized in this way. Why?
> Arrow
> ------
>
> Key: ARROW-9989
> URL: https://issues.apache.org/jira/browse/ARROW-9989
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
> Affects Versions: 0.14.0
> Environment: Linux 18.04, Arrow from maven 0.14.0
> Reporter: Litchy Soong
> Priority: Major
>
> In scala, (Java of Arrow), following code work
> {quote}object A {
> def write(): = {
> val vectorSchemaRoot = VectorSchemaRoot.create(getSchema, allocator) val writer = new ArrowStreamWriter(vectorSchemaRoot, null, out) }
> }
> {quote}
> But following does not work
> {quote}object A {
> var vectorSchemaRoot: VectorSchemaRoot = null
> var writer: ArrowStreamWriter = null
> def write(): = {
> vectorSchemaRoot = VectorSchemaRoot.create(getSchema, allocator) writer = new ArrowStreamWriter(vectorSchemaRoot, null, out) }
> }
> {quote}
> The error is ,
> {quote}java.lang.IllegalStateException: wrong buffer size: 601 != 4081java.lang.IllegalStateException: wrong buffer size: 601 != 4081 at org.apache.arrow.vector.ipc.message.MessageSerializer.writeBatchBuffers(MessageSerializer.java:297) at org.apache.arrow.vector.ipc.message.MessageSerializer.serialize(MessageSerializer.java:267) at org.apache.arrow.vector.ipc.ArrowWriter.writeRecordBatch(ArrowWriter.java:132) at org.apache.arrow.vector.ipc.ArrowWriter.writeBatch(ArrowWriter.java:120)
>
> java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0)) at io.netty.buffer.ArrowBuf.checkIndexD(ArrowBuf.java:337) at io.netty.buffer.ArrowBuf.chk(ArrowBuf.java:324) at io.netty.buffer.ArrowBuf.getByte(ArrowBuf.java:526) at org.apache.arrow.vector.BitVectorHelper.setBit(BitVectorHelper.java:70) at org.apache.arrow.vector.Float4Vector.set(Float4Vector.java:168)
>
> java.lang.IllegalStateException: RefCnt has gone negativejava.lang.IllegalStateException: RefCnt has gone negative at org.apache.arrow.util.Preconditions.checkState(Preconditions.java:458) at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:134) at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:108) at org.apache.arrow.vector.BaseFixedWidthVector.reAlloc(BaseFixedWidthVector.java:441) at org.apache.arrow.vector.BaseFixedWidthVector.setValueCount(BaseFixedWidthVector.java:708) at org.apache.arrow.vector.VectorSchemaRoot.setRowCount(VectorSchemaRoot.java:226)
> {quote}
>
> and it is raised every second time when I call the method. And seems both ArrowStreamWriter and VectorSchemaRoot could not be initialized in this way. Why?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)