You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/06/01 06:43:00 UTC
[jira] [Commented] (AVRO-2837) Java DecimalConversion handling of
scale and precision
[ https://issues.apache.org/jira/browse/AVRO-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120784#comment-17120784 ]
ASF subversion and git services commented on AVRO-2837:
-------------------------------------------------------
Commit 2a766be2c417c71b42b49ed2898ad3e8ed0d0113 in avro's branch refs/heads/master from Matthew McMahon
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=2a766be ]
AVRO-2837: DecimalConversion handling of scale and precision (#884)
* AVRO-2837: DecimalConversion handling of scale and precision
Improve the handling to check precision and not error if scale
of value is less
* AVRO-2837: DecimalConversion handling of scale and precision
Scale needs to be set correctly for serialization in order to
deserialize as expected. Added additional tests and extended
error messages
> Java DecimalConversion handling of scale and precision
> ------------------------------------------------------
>
> Key: AVRO-2837
> URL: https://issues.apache.org/jira/browse/AVRO-2837
> Project: Apache Avro
> Issue Type: Bug
> Components: java, logical types
> Affects Versions: 1.8.2, 1.9.2
> Reporter: Matthew McMahon
> Priority: Major
> Attachments: AVRO-2837.patch, AVRO-2837.patch
>
>
> Came across an interesting issue in Avro 1.8.2
> Configured a decimal logical type (Fixed type of size 12 with scale of 15 and precision of 28).
> Due to an upstream bug, a value of 1070464558597365250.000000000000000 (1.07046455859736525E+18 that is then rescaled to 15) appears, and the DecimalConversion attempts to turn it into a Fixed type.
> This should have failed, as it has a precision of 34 and won't fit into the 12 bytes (needs 14). However in 1.8.2 this still writes a value that downstream processing then works out is invalid and errors.
> Basically the top 2 bytes are thrown away.
> This problem is fixed in 1.9.2 due to the change in https://issues.apache.org/jira/browse/AVRO-2309 as this particular issue fails when it attempts to pass an offset of -2 to the System.arraycopy method.
> That seems ok, but is a bit of a red herring to the actual issue, and precision is still not actually being checked.
> Proposing a couple changes to the DecimalConversion:
> * Check precision set on the decimal logical type. If value has greater precision then error with more informative message
> * Still check scale and error if value has a greater scale. However if the scale in value is less, than it seems safe to update the scale and pad zero's rather than error
> * Do this for both Bytes and Fixed types
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)