You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "bodduv (via GitHub)" <gi...@apache.org> on 2025/03/27 11:54:51 UTC
[I] [arrow-vector] Zero-copy transfer of DecimalVector and Decimal256Vector is losing nullability information [arrow-java]
bodduv opened a new issue, #692:
URL: https://github.com/apache/arrow-java/issues/692
### Describe the bug, including details regarding any error messages, version, and platform.
In the main branch, `DecimalVector` and `Decimal256Vector` are losing nullability information during a zero-copy transfer. It currently **forces** a nullable `FieldType` onto the "to" `ValueVector`.
The deficiency in the code is in `DecimalVector.TransferImpl` class when trying to get a transfer `getTransferPair(String, BufferAllocator)` creating a `DecimalVector` with a nullable `FieldType` (although the "from" ValueVector's FieldType is non-nullable):
https://github.com/apache/arrow-java/blob/48a20ba91605ebb44df845671782acd8f869bd4b/vector/src/main/java/org/apache/arrow/vector/DecimalVector.java#L563-L569
Note that all the other (at least the primitive) `ValueVector`s are transferred without losing nullability information by using the "from" `ValueVector`'s field. E.g.:
https://github.com/apache/arrow-java/blob/48a20ba91605ebb44df845671782acd8f869bd4b/vector/src/main/java/org/apache/arrow/vector/BigIntVector.java#L319-L324
---
To **reproduce** the issue: add the following test in `src/test/java/org/apache/arrow/vector/TestDecimalVector.java` to see it fail.
```java
@Test
public void testGetTransferPairWithoutFieldNonNullable() {
final FieldType decimalNonNullableType =
new FieldType(false, new ArrowType.Decimal(10, scale), null);
final DecimalVector fromVector =
new DecimalVector("decimal", decimalNonNullableType, allocator);
final TransferPair transferPair =
fromVector.getTransferPair(fromVector.getField().getName(), allocator);
final DecimalVector toVector = (DecimalVector) transferPair.getTo();
// These asserts fail
assertSame(fromVector.getField().isNullable(), toVector.getField().isNullable());
assertSame(fromVector.getField().getFieldType(), toVector.getField().getFieldType());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [arrow-vector] Zero-copy transfer of DecimalVector and Decimal256Vector is losing nullability information [arrow-java]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre closed issue #692: [arrow-vector] Zero-copy transfer of DecimalVector and Decimal256Vector is losing nullability information
URL: https://github.com/apache/arrow-java/issues/692
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org