You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/09/14 17:12:00 UTC
[jira] [Commented] (PARQUET-1417)
BINARY_AS_SIGNED_INTEGER_COMPARATOR fails with IOBE for the same arrays
with the different length
[ https://issues.apache.org/jira/browse/PARQUET-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615096#comment-16615096 ]
ASF GitHub Bot commented on PARQUET-1417:
-----------------------------------------
vvysotskyi opened a new pull request #522: PARQUET-1417: BINARY_AS_SIGNED_INTEGER_COMPARATOR fails with IOBE for the same arrays with the different length
URL: https://github.com/apache/parquet-mr/pull/522
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> BINARY_AS_SIGNED_INTEGER_COMPARATOR fails with IOBE for the same arrays with the different length
> -------------------------------------------------------------------------------------------------
>
> Key: PARQUET-1417
> URL: https://issues.apache.org/jira/browse/PARQUET-1417
> Project: Parquet
> Issue Type: Bug
> Reporter: Volodymyr Vysotskyi
> Priority: Major
> Labels: pull-request-available
>
> {{BINARY_AS_SIGNED_INTEGER_COMPARATOR}} fails when the same byte arrays but with the different number leading zeros are compared:
> {code:java}
> BINARY_AS_SIGNED_INTEGER_COMPARATOR.compare(
> Binary.fromConstantByteBuffer(ByteBuffer.wrap(new byte[] { 0, 0, -108 })),
> Binary.fromConstantByteBuffer(ByteBuffer.wrap(new byte[] { 0, -108 })));
> {code}
> Error is:
> {noformat}
> java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkIndex(Buffer.java:540)
> at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:139)
> at org.apache.parquet.schema.PrimitiveComparator$9.compare(PrimitiveComparator.java:280)
> at org.apache.parquet.schema.PrimitiveComparator$9.compare(PrimitiveComparator.java:262)
> at org.apache.parquet.schema.PrimitiveComparator$BinaryComparator.compareNotNulls(PrimitiveComparator.java:186)
> at org.apache.parquet.schema.PrimitiveComparator$BinaryComparator.compareNotNulls(PrimitiveComparator.java:183)
> at org.apache.parquet.schema.PrimitiveComparator.compare(PrimitiveComparator.java:63)
> {noformat}
> The problem is that {{BINARY_AS_SIGNED_INTEGER_COMPARATOR.compare(ByteBuffer b1, ByteBuffer b2)}} method passes the length of the first {{ByteBuffer}}, but it should pass the less length since padding was calculated and passed for the {{ByteBuffer}} with greater length to the {{compare(int length, ByteBuffer b1, int p1, ByteBuffer b2, int p2)}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)