You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2017/10/04 05:35:01 UTC
[jira] [Commented] (SPARK-22192) An RDD of nested POJO objects
cannot be converted into a DataFrame using SQLContext.createDataFrame API
[ https://issues.apache.org/jira/browse/SPARK-22192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190838#comment-16190838 ]
Hyukjin Kwon commented on SPARK-22192:
--------------------------------------
Would you mind if I ask a reproducer?
> An RDD of nested POJO objects cannot be converted into a DataFrame using SQLContext.createDataFrame API
> -------------------------------------------------------------------------------------------------------
>
> Key: SPARK-22192
> URL: https://issues.apache.org/jira/browse/SPARK-22192
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Environment: Independent of OS / platform
> Reporter: Asif Hussain Shahid
> Priority: Minor
>
> If an RDD contains nested POJO objects, then SQLContext.createDataFrame(RDD, Class) api only handles the top level POJO object. It throws ScalaMatchError exception when handling the nested POJO object as the code does not recursively handle the nested POJOs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org