[GitHub] [spark] planga82 commented on pull request #32599: [SPARK-35320][SQL] Support other data types as key types when parsing map in from_json

[GitBox <gi...@apache.org>]

planga82 commented on pull request #32599:
URL: https://github.com/apache/spark/pull/32599#issuecomment-847322940


   > Just to clarify, it works iff users specify the explicit map type. Schema inference won't infer such schema.
   
   Yes, Schema inference won't infer such schema
   
   
   We have clear then that it is not a Bug, as @cloud-fan and @viirya say, the keys of a Json are always of type String, so this would be an improvement.
   I think that since we allow write operations of type
   `df.withColumn(colName, from_json(col(colName), MapType(DoubleType, StringType))).write.json(jsonDir)`
   it is more consistent to support the reverse operation
   `spark.read.schema(StructType(Seq(StructField(DoubleType, dataType)))).json(jsonDir)`
   
   Anyway, if we decide to not support this, I think we need to improve the error message
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org