[jira] [Created] (SPARK-10400) Rename or deprecate SQL option "spark.sql.parquet.followParquetFormatSpec"

["Cheng Lian (JIRA)" <ji...@apache.org>]

Cheng Lian created SPARK-10400:
----------------------------------

             Summary: Rename or deprecate SQL option "spark.sql.parquet.followParquetFormatSpec"
                 Key: SPARK-10400
                 URL: https://issues.apache.org/jira/browse/SPARK-10400
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 1.5.0
            Reporter: Cheng Lian
            Assignee: Cheng Lian
            Priority: Minor


We introduced SQL option "spark.sql.parquet.followParquetFormatSpec" while working on implementing Parquet backwards-compatibility rules in SPARK-6777. It indicates whether we should use legacy Parquet format adopted by Spark 1.4 and prior versions or the standard format defined in parquet-format spec. However, the name of this option is somewhat confusing, because it's not super intuitive why we shouldn't follow the spec. Would be nice to rename it to "spark.sql.parquet.writeLegacyFormat" and invert its default value (they have opposite meanings). Note that this option is not "public" ({{isPublic}} is false).

At the moment of writing, 1.5 RC3 has already been cut. If we can't make this one into 1.5, we can deprecate the old option with the new one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org