[jira] [Created] (SPARK-11412) mergeSchema option not working for orc format?

["Dave (JIRA)" <ji...@apache.org>]

Dave created SPARK-11412:
----------------------------

             Summary: mergeSchema option not working for orc format?
                 Key: SPARK-11412
                 URL: https://issues.apache.org/jira/browse/SPARK-11412
             Project: Spark
          Issue Type: Bug
            Reporter: Dave


when I tried to load partitioned orc files with a slight difference in a nested column. say 
column 
-- request: struct (nullable = true)
 |    |-- datetime: string (nullable = true)
 |    |-- host: string (nullable = true)
 |    |-- ip: string (nullable = true)
 |    |-- referer: string (nullable = true)
 |    |-- request_uri: string (nullable = true)
 |    |-- uri: string (nullable = true)
 |    |-- useragent: string (nullable = true)
And then there's a page_url_lists attributes in the later partitions.

I tried to use
val s = sqlContext.read.format("orc").option("mergeSchema", "true").load("/data/warehouse/xxxx") to load the data.
But the schema doesn't show request.page_url_lists.
I am wondering if schema merge doesn't work for orc?




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org