Re: [parquet-mr] Support writing INT96 type by parquet-avro

[Gabor Szadovszky <ga...@apache.org>]

I've created PARQUET-2037
<https://issues.apache.org/jira/browse/PARQUET-2037> about this topic. Also
created a potential implementation: #901
<https://github.com/apache/parquet-mr/pull/901>
Any reviews/comments are welcomed.

On Tue, Apr 20, 2021 at 4:48 PM Gabor Szadovszky <ga...@apache.org> wrote:

> Hi everyone,
>
> We have faced a customer issue that we need to write INT96 values using
> parquet-avro. I don't have too much experience in parquet-avro so I am
> curious about your opinions.
>
> After PARQUET-1928 we are able to read INT96 values into Avro FIXED values
> by using the configuration "parquet.avro.readInt96AsFixed". Now we need a
> similar option where we can write the FIXED values into the original INT96
> values. There are two problems I can see with this.
> 1. INT96 is deprecated for a while now. Previously we did not want to
> implement anything for additional support of INT96. What are your feelings
> on this?
> 2. How can we distinguish the "original" FIXED values from the ones
> created from INT96 values? Of course we only need to convert FIXED values
> that have the length of 12bytes but I don't think it is enough.
>
> Thanks a lot,
> Gabor
>