[jira] [Commented] (SPARK-33920) We cannot pass schema to a createDataFrame function in scala, however we can do this in python.

["L. C. Hsieh (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-33920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255321#comment-17255321 ] 

L. C. Hsieh commented on SPARK-33920:
-------------------------------------

There is `{{def createDataFrame(rowRDD: RDD[Row], schema: StructType)}}` in Scala API. If you mean `{{def createDataFrame[A <: Product : TypeTag](data: Seq[A])}}`, Scala API uses Scala reflection to infer the schema of the given Product. Why you need `{{schema}}` parameter here?

> We cannot pass schema to a createDataFrame function in scala, however we can do this in python.
> -----------------------------------------------------------------------------------------------
>
>                 Key: SPARK-33920
>                 URL: https://issues.apache.org/jira/browse/SPARK-33920
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build, SQL
>    Affects Versions: 3.0.1
>            Reporter: Abdul Rafay Abdul Rafay
>            Priority: Critical
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> ~spark.createDataFrame(data, schema)~
> ~I am able to pass schema as a parameter to a function createDataFrame in python but cannot pass this in scala for static data.~



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org