[jira] [Commented] (SPARK-18717) Datasets - crash (compile exception) when mapping to immutable scala map

["Andrew Ray (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-18717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15723499#comment-15723499 ] 

Andrew Ray commented on SPARK-18717:
------------------------------------

Use `scala.collection.Map` as the type in your case class definition (instead of the default `scala.collection.immutable.Map`). Supported scala types are listed in the doc here: https://spark.apache.org/docs/latest/sql-programming-guide.html#data-types although this should probably still be fixed or at least have a better error/warning.

> Datasets - crash (compile exception) when mapping to immutable scala map
> ------------------------------------------------------------------------
>
>                 Key: SPARK-18717
>                 URL: https://issues.apache.org/jira/browse/SPARK-18717
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 2.0.2
>            Reporter: Damian Momot
>
> {code}
> val spark: SparkSession = ???
> case class Test(id: String, map_test: Map[Long, String])
> spark.sql("CREATE TABLE xyz.map_test (id string, map_test map<int, string>) STORED AS PARQUET")
> spark.sql("SELECT * FROM xyz.map_test").as[Test].map(t => t).collect()
> {code}
> {code}
> org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 307, Column 108: No applicable constructor/method found for actual parameters "java.lang.String, scala.collection.Map"; candidates are: "$line14.$read$$iw$$iw$Test(java.lang.String, scala.collection.immutable.Map)"
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org