DecimalType.MAX_PRECISION, why 38?

[Yaroslav Tkachenko <ya...@goldsky.io>]

Hey everyone,

I'm curious if anyone knows the reason behind choosing 38 as
a MAX_PRECISION for DecimalType in the Table API?

I needed to process decimal types larger than that and I ended up
monkey-patching DecimalType to use a higher precision. I understand it adds
a bit of overhead, but I haven't noticed any serious problems.

Maybe this number can be configured. I noticed it's hardcoded in a few
places, but it should be a fairly straightforward change.

Thanks!

Re: DecimalType.MAX_PRECISION, why 38?

[Martijn Visser <ma...@apache.org>]

Hi Yaroslav,

I did a quick dive into FLIP-37 [1] which was on the rework of the type
system. In the FLIP it's noted:

> Decimal considerations: We use Hive’s maximums:
https://github.com/apache/hive/blob/master/storage-api/src/java/org/apache/hadoop/hive/common/type/HiveDecimal.java

So that's the reason why it was set to 38 :)

Best regards,

Martijn Visser
https://twitter.com/MartijnVisser82
https://github.com/MartijnVisser

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-37%3A+Rework+of+the+Table+API+Type+System

On Tue, 19 Apr 2022 at 02:42, Yaroslav Tkachenko <ya...@goldsky.io>
wrote:

> Hey everyone,
>
> I'm curious if anyone knows the reason behind choosing 38 as
> a MAX_PRECISION for DecimalType in the Table API?
>
> I needed to process decimal types larger than that and I ended up
> monkey-patching DecimalType to use a higher precision. I understand it adds
> a bit of overhead, but I haven't noticed any serious problems.
>
> Maybe this number can be configured. I noticed it's hardcoded in a few
> places, but it should be a fairly straightforward change.
>
> Thanks!
>