[jira] [Comment Edited] (AVRO-2592) Avro decimal fails on certain conditions - ByteBuffer.position() is the root cause

["Ryan Skraba (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984900#comment-16984900 ] 

Ryan Skraba edited comment on AVRO-2592 at 11/29/19 11:06 AM:
--------------------------------------------------------------

bq. Am I correct with the statement "The ByteBuffer does wrap one Datum only and always?". In my opinion, the ByteBuffer position handling is neat for cases where you have one very large buffer and you append data. Even better, in cases where a cyclic buffer is required and you flip its contents.
bq. 
This is a compelling argument, and I agree.  Fixing this case would be simple and wouldn't break any useful behaviour -- I couldn't find any way that the ByteBuffer being used for the decimal could be an isolated part of another buffer, and I really can't imagine a use case where a user would *want* a decimal to be consumed only once and then fail.

(Note that, unlike my claim, needing to rewind the ByteBuffers used by Avro is no longer "consistent", and thank goodness: AVRO-1799 fixed a bug where even {{record.toString()}} used to consume them!)


was (Author: ryanskraba):
> Am I correct with the statement "The ByteBuffer does wrap one Datum only and always?". In my opinion, the ByteBuffer position handling is neat for cases where you have one very large buffer and you append data. Even better, in cases where a cyclic buffer is required and you flip its contents.

This is a compelling argument, and I agree.  Fixing this case would be simple and wouldn't break any useful behaviour -- I couldn't find any way that the ByteBuffer being used for the decimal could be an isolated part of another buffer, and I really can't imagine a use case where a user would *want* a decimal to be consumed only once and then fail.

(Note that, unlike my claim, needing to rewind the ByteBuffers used by Avro is no longer "consistent", and thank goodness: AVRO-1799 fixed a bug where even {{record.toString()}} used to consume them!)

> Avro decimal fails on certain conditions - ByteBuffer.position() is the root cause
> ----------------------------------------------------------------------------------
>
>                 Key: AVRO-2592
>                 URL: https://issues.apache.org/jira/browse/AVRO-2592
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.9.1
>            Reporter: Werner Daehn
>            Priority: Blocker
>
> The Decimal Conversion from/to Bytebuffer is using the methods that consider the current position, e.g. remaining().
> [https://github.com/apache/avro/blob/release-1.9.1/lang/java/avro/src/main/java/org/apache/avro/Conversions.java#L82]
> At first sight that looks like a good idea. But actually it creates all sorts of problems.
> For example this code fails with "Zero Length BigInteger":
>  
> {code:java}
> BigDecimal d = BigDecimal.valueOf(3.1415); BigDecimal d = BigDecimal.valueOf(3.1415);
> Decimal decimaltype = LogicalTypes.decimal(7, 4); ByteBuffer buffer = DECIMAL_CONVERTER.toBytes(d, null, decimaltype);
> System.out.println(DECIMAL_CONVERTER.fromBytes(buffer, null, decimaltype).toString());
> BigDecimal n = DECIMAL_CONVERTER.fromBytes(buffer, null, decimaltype);
> System.out.println(n.toString());{code}
>  
> Reason is obvious. The first call to fromBytes() moves the position from 0 to the last byte. The second invocation reads from the last position, hence zero records.
> There are other situations this might cause issues, e.g. a user might create the ByteBuffer via other means and it is normal that the position is after the last byte. Then the serialization would not work either. And many other.
> As the ByteBuffer is used to wrap a single BigDecimal, I would suggest to remove all position-aware/setting methods and read/write from position zero.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)