You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/04/10 10:06:00 UTC
[jira] [Commented] (PARQUET-415) ByteBufferBackedBinary
serialization is broken
[ https://issues.apache.org/jira/browse/PARQUET-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432019#comment-16432019 ]
ASF GitHub Bot commented on PARQUET-415:
----------------------------------------
jinxing64 commented on issue #305: PARQUET-415: Fix ByteBuffer Binary serialization.
URL: https://github.com/apache/parquet-mr/pull/305#issuecomment-380045863
@rdblue
Thanks for fixing this problem, may I ask a question?
1. What parquet version includes this change? is it 1.9.0?
2. Without this change, what exception will be thrown? is it below?
```
Caused by: java.nio.BufferUnderflowException
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:151)
at java.nio.ByteBuffer.get(ByteBuffer.java:715)
at org.apache.parquet.io.api.Binary$ByteBufferBackedBinary.getBytes(Binary.java:405)
at org.apache.parquet.io.api.Binary$ByteBufferBackedBinary.getBytesUnsafe(Binary.java:414)
at org.apache.parquet.io.api.Binary$ByteBufferBackedBinary.writeObject(Binary.java:484)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
......
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> ByteBufferBackedBinary serialization is broken
> ----------------------------------------------
>
> Key: PARQUET-415
> URL: https://issues.apache.org/jira/browse/PARQUET-415
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: 1.8.1
> Reporter: Ryan Blue
> Assignee: Ryan Blue
> Priority: Blocker
> Fix For: 1.9.0
>
>
> While working on logical types for parquet-avro, I updated the {{FromStringBinary}} to work with a {{CharSequence}} instead of a {{String}}, which broke {{TestFilterApiMethods#testSerializable}}. The underlying problem is that when the buffer-backed binary is deserialized, {{length}} and {{offset}} are not initialized so the buffer is correct but the apparent length of the binary is 0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)