[GitHub] [spark] HyukjinKwon commented on issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's input json as literal only

[GitBox <gi...@apache.org>]

HyukjinKwon commented on issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's input json as literal only
URL: https://github.com/apache/spark/pull/22775#issuecomment-594231132
 
 
   @nchammas, thanks for some input here.
   
   > I could do something like this to check whether or not I have a consistent schema:
   >
   > ```
   > df.select(schema_of_json(...)).distinct().count()
   > ```
   
   I wanted to make the expression only for the specific usecase and avoid to have multiple ways for the same thing. Other cases can be easily worked around. For the case you mentioned, it can be worked around as below:
   
   ```scala
   spark.read.json(df.select("json").as[String]).schema == StructType.fromDDL(...)
   ```
   
   I think it is fine to have an expression that takes literals to return a column to support a missing usecase requested multiple times.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org