You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Rohit Mishra (Jira)" <ji...@apache.org> on 2020/08/15 04:17:00 UTC
[jira] [Commented] (SPARK-32619) converting dataframe to dataset
for the json schema
[ https://issues.apache.org/jira/browse/SPARK-32619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178147#comment-17178147 ]
Rohit Mishra commented on SPARK-32619:
--------------------------------------
[~Manjay7869], Can you please update- Environment detail & reproducible steps. Please rewrite the description by introducing code blocks wherever you have tried to explain any code snippet.
If you are not sure please try to utilize Stack overflow and User mail list to get an answer and probably a solution. Please read the guideline for future reference- [https://spark.apache.org/community.html]
> converting dataframe to dataset for the json schema
> ---------------------------------------------------
>
> Key: SPARK-32619
> URL: https://issues.apache.org/jira/browse/SPARK-32619
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.4.3
> Reporter: Manjay Kumar
> Priority: Minor
>
> have a schema
>
> {
> Details :[{
> phone : "98977999"
> contacts: [{
> name:"manjay"
> ---------- has missing street block in json
> ]}
> ]}
>
> }
>
> Case class , based on schema
> case class Details (
> phone : String,
> contacts : Array[Adress]
> )
>
> case class Adress(
> name : String
> street : String
>
> )
>
>
> this throws : No such struct field street - Analysis exception.
>
> dataframe.as[Details]
>
> Is this a bug ?? or there is a resolution for this.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org