You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Stefan Ziegler <st...@gmail.com> on 2023/04/10 16:11:26 UTC

AvroParquetWriter and local timestamp problems

Hi

I'm trying to create some parquet files with the AvroParquetWriter but I'm
struggling with timestamps. It seems that avro supports timetamps without
time zone (aka isAdjustedToUTC=false) and I can create a field with the
according logical type: "LocalTimestampMillis datetimeType =
LogicalTypes.localTimestampMillis();"

But every client I use just interprets the field as bigint or int64. As far
as I understand the AvroSchemaConverter code, the conversion of the "local
timestamp" is missing:
https://github.com/apache/parquet-mr/blob/master/parquet-avro/src/main/java/org/apache/parquet/avro/AvroSchemaConverter.java#L445

Not sure if this is true and if it would need some adjustments in other
areas.

best regards
Stefan