[jira] [Created] (SPARK-26230) FileIndex: if case sensitive, validate partitions with original column names

["Gengliang Wang (JIRA)" <ji...@apache.org>]

Gengliang Wang created SPARK-26230:
--------------------------------------

             Summary: FileIndex: if case sensitive, validate partitions with original column names
                 Key: SPARK-26230
                 URL: https://issues.apache.org/jira/browse/SPARK-26230
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.0.0
            Reporter: Gengliang Wang


Partition column name is required to be unique under the same directory. The following paths are invalid partitioned directory:
```
hdfs://host:9000/path/a=1
hdfs://host:9000/path/b=2
```

If case sensitive, the following paths should be invalid too:
```
hdfs://host:9000/path/a=1
hdfs://host:9000/path/A=2
```
Since column 'a' and 'A' are different, and it is wrong to use either one as the column name in partition schema.

Also, there is a `TODO` in the code. This PR is to resolve the problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org