You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Xuefu Zhang (JIRA)" <ji...@apache.org> on 2014/04/01 01:21:16 UTC
[jira] [Commented] (AVRO-1402) Support for DECIMAL primitive type
[ https://issues.apache.org/jira/browse/AVRO-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955862#comment-13955862 ]
Xuefu Zhang commented on AVRO-1402:
-----------------------------------
{quote}
The scale and precision specified in the column definition are maximums. My understanding is that decimals with smaller scale and precision may be stored in such columns. For example in Hive if I have a DECIMAL(5,3) column then 12.4 is stored as [124, 1] as the unscaled int-scale pair (so precision 3, scale 1), and not as [12400, 3] So in effect there are per-record precision and scale values, unless I'm missing something.
{quote}
Sorry for being late on this. While it's possible that a column of type decimal might have different scales from row to row, I'm not sure if there is any use case for that. At least, majority of use cases is that all rows will have the same scale. So for majority of use cases, storage efficiency will suffer a little for storing scale on per row basis. Most of time, same application (such as Hive AvroSerde) will be used to write/read the data, so the way data is stored can be controlled. Of course, storing scale per row is more generic. If that's case, the need for maxPrecision/maxScale at schema is less meaningful, as consumer of the decimal data will need and be able to figure out the precision/scale on a per-row basis.
If we do choose to storing scale per row, I'm wondering if byte instead of int can be used as the type, saving some storage scale.
> Support for DECIMAL primitive type
> ----------------------------------
>
> Key: AVRO-1402
> URL: https://issues.apache.org/jira/browse/AVRO-1402
> Project: Avro
> Issue Type: New Feature
> Affects Versions: 1.7.5
> Reporter: Mariano Dominguez
> Priority: Minor
> Labels: Hive
> Attachments: AVRO-1402.patch, AVRO-1402.patch, AVRO-1402.patch
>
>
> Currently, Avro does not seem to support a DECIMAL type or equivalent.
> http://avro.apache.org/docs/1.7.5/spec.html#schema_primitive
> Adding DECIMAL support would be particularly interesting when converting types from Avro to Hive, since DECIMAL is already a supported data type in Hive (0.11.0).
--
This message was sent by Atlassian JIRA
(v6.2#6252)