[jira] [Commented] (AVRO-2592) Avro decimal fails on certain conditions - ByteBuffer.position() is the root cause

["Ryan Skraba (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950989#comment-16950989 ] 

Ryan Skraba commented on AVRO-2592:
-----------------------------------

Hello!  I'm not _sure_ this is a bug -- the Java implementation consistently takes the current position inside ByteBuffers into account and consumes the bytes when they're read.

I've hit this problem in the past, when logging a record containing BYTES fields and inadvertently consuming the buffer, and we've _very_ recently seen AVRO-2588.  It isn't limited to DecimalConversion.... 

This *could* be a point of confusion, but at least it's consistent: ByteBuffer datum "disappears" once you've consumed it, and you must reset/rewind the position if you plan to use it again.  Maybe we should revisit *that* decision?

In either case, I wouldn't consider this a Blocker -- a workaround exists and is pretty simple.

> Avro decimal fails on certain conditions - ByteBuffer.position() is the root cause
> ----------------------------------------------------------------------------------
>
>                 Key: AVRO-2592
>                 URL: https://issues.apache.org/jira/browse/AVRO-2592
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.9.1
>            Reporter: Werner Daehn
>            Priority: Blocker
>
> The Decimal Conversion from/to Bytebuffer is using the methods that consider the current position, e.g. remaining().
> [https://github.com/apache/avro/blob/release-1.9.1/lang/java/avro/src/main/java/org/apache/avro/Conversions.java#L82]
> At first sight that looks like a good idea. But actually it creates all sorts of problems.
> For example this code fails with "Zero Length BigInteger":
>  
> {code:java}
> BigDecimal d = BigDecimal.valueOf(3.1415); BigDecimal d = BigDecimal.valueOf(3.1415);
> Decimal decimaltype = LogicalTypes.decimal(7, 4); ByteBuffer buffer = DECIMAL_CONVERTER.toBytes(d, null, decimaltype);
> System.out.println(DECIMAL_CONVERTER.fromBytes(buffer, null, decimaltype).toString());
> BigDecimal n = DECIMAL_CONVERTER.fromBytes(buffer, null, decimaltype);
> System.out.println(n.toString());{code}
>  
> Reason is obvious. The first call to fromBytes() moves the position from 0 to the last byte. The second invocation reads from the last position, hence zero records.
> There are other situations this might cause issues, e.g. a user might create the ByteBuffer via other means and it is normal that the position is after the last byte. Then the serialization would not work either. And many other.
> As the ByteBuffer is used to wrap a single BigDecimal, I would suggest to remove all position-aware/setting methods and read/write from position zero.
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)