You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Maxim Gekk (Jira)" <ji...@apache.org> on 2020/10/21 20:07:00 UTC

[jira] [Created] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

Maxim Gekk created SPARK-33210:
----------------------------------

             Summary: Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default
                 Key: SPARK-33210
                 URL: https://issues.apache.org/jira/browse/SPARK-33210
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.1.0
            Reporter: Maxim Gekk


The ticket aims to set the following SQL configs:
- spark.sql.legacy.parquet.int96RebaseModeInWrite
- spark.sql.legacy.parquet.int96RebaseModeInRead
to EXCEPTION by default.

The reason is let users to decide should Spark modify loaded/saved timestamps instead of silently shifting timestamps while rebasing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org