[jira] [Commented] (AVRO-2164) Make Decimal a first class type.

["Zoltan Farkas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675887#comment-16675887 ] 

Zoltan Farkas commented on AVRO-2164:
-------------------------------------

I am in favor of making decimal a first class type. 

One extra benefit is that we would be able to properly serialize the decimals in the json format. (Number instead of binary)

Regarding evolution, we should allow for increases of precision and scale. 

why can't this be part of avro 2.0?

Side note:

In my own decimal logical type implementation, I serialize the scale along with the unscaled value.  So if I have a decimal ( 16, 8) and I serialize a decimal value with a scale <= 8, the value will be serialized with its original scale, for everything with a larger scale, I either error or round depending if a rounding mode is specified via a type attribute... 




> Make Decimal a first class type.
> --------------------------------
>
>                 Key: AVRO-2164
>                 URL: https://issues.apache.org/jira/browse/AVRO-2164
>             Project: Avro
>          Issue Type: Improvement
>          Components: logical types
>    Affects Versions: 1.8.2
>            Reporter: Andy Coates
>            Priority: Major
>
> I'd be interested to hear the communities thoughts on making decimal a first class type. 
> The current logical type encodes a decimal into a _bytes_ or _fixed_. This encoding does not include any information about the scale, i.e. this encoding is lossy. 
> There are open issues around the compatibility / evolvability of schemas containing decimal logical types, (e.g. AVRO-2078 & AVRO-1721), that mean reading data that was previously written with a different scale will result in data corruption.
> If these issues were fixed, with suitable compatibility checks put in place, this would then make it impossible to evolve an Avro schema where the scale needs to be changed. This inability to evolve the scale is very restrictive, and can result in high overhead for organizations that _need_ to change the scale, i.e. they may potentially need to copy their entire data set, deserializing with the old scale and re-serializing with the new.
> If _decimal_ were promoted to a first class type, this would allow the scale to be captured in the serialized form, allow for schema evolution support.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)