[jira] [Resolved] (SPARK-21019) read orc when some of the columns are missing in some files

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-21019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-21019.
----------------------------------
    Resolution: Duplicate

I believe this should be a duplicate of SPARK-11412.

>  read orc when some of the columns are missing in some files
> ------------------------------------------------------------
>
>                 Key: SPARK-21019
>                 URL: https://issues.apache.org/jira/browse/SPARK-21019
>             Project: Spark
>          Issue Type: Bug
>          Components: Input/Output
>    Affects Versions: 2.1.1
>            Reporter: Modi Tamam
>
> I'm using Spark-2.1.1.
> I'm experiencing an issue when I'm reading a bunch of ORC files when some of the fields are missing from some of the files (file-1 has fields 'a' and 'b', file-2 has fields 'a' and 'c'). When I'm running the same flow with JSON files format, every thing is just fine (you can see it at the code snippets , if you'll run it...) My question is whether it's a bug or an expected behavior?
> I'v pushed a maven project, ready for run, you can find it here https://github.com/MordechaiTamam/spark-orc-issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org