You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Chenxiao Mao (JIRA)" <ji...@apache.org> on 2018/08/21 09:24:00 UTC

[jira] [Created] (SPARK-25175) Case-insensitive field resolution when reading from ORC

Chenxiao Mao created SPARK-25175:
------------------------------------

             Summary: Case-insensitive field resolution when reading from ORC
                 Key: SPARK-25175
                 URL: https://issues.apache.org/jira/browse/SPARK-25175
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.3.1
            Reporter: Chenxiao Mao


SPARK-25132 adds support for case-insensitive field resolution when reading from Parquet files. We found ORC files have similar issues. Since Spark has 2 OrcFileFormat, we should add support for both.
* Since SPARK-2883, Spark supports ORC inside sql/hive module with Hive dependency. This hive OrcFileFormat doesn't support case-insensitive field resolution at all.
* SPARK-20682 adds a new ORC data source inside sql/core. This native OrcFileFormat supports case-insensitive field resolution, however it cannot handle duplicate fields.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org