You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Maxim Gekk (Jira)" <ji...@apache.org> on 2019/10/12 10:23:00 UTC

[jira] [Created] (SPARK-29448) Support the `INTERVAL` type by Parquet datasource

Maxim Gekk created SPARK-29448:
----------------------------------

             Summary: Support the `INTERVAL` type by Parquet datasource
                 Key: SPARK-29448
                 URL: https://issues.apache.org/jira/browse/SPARK-29448
             Project: Spark
          Issue Type: New Feature
          Components: SQL
    Affects Versions: 2.4.4
            Reporter: Maxim Gekk


Parquet format allows to store intervals as triple of (milliseconds, days, months) see https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#interval . The `INTERVAL` logical type is used for an interval of time. _It must annotate a fixed_len_byte_array of length 12. This array stores three little-endian unsigned integers that represent durations at different granularities of time. The first stores a number in months, the second stores a number in days, and the third stores a number in milliseconds. This representation is independent of any particular timezone or date._

Need to support writing and reading values of Catalyst's CalendarIntervalType.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org