You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/05 08:53:31 UTC

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29942: [SPARK-33007][SQL] Simplify named_struct + get struct field + from_json expression chain

HyukjinKwon commented on a change in pull request #29942:
URL: https://github.com/apache/spark/pull/29942#discussion_r499440791



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeJsonExprs.scala
##########
@@ -28,10 +28,43 @@ import org.apache.spark.sql.types.{ArrayType, StructType}
  * The optimization includes:
  * 1. JsonToStructs(StructsToJson(child)) => child.
  * 2. Prune unnecessary columns from GetStructField/GetArrayStructFields + JsonToStructs.
+ * 3. struct(from_json.col1, from_json.col2, from_json.col3...) => struct(from_json)

Review comment:
       It's a nit but can we match the comment style? 1. and 2. used the expression class names but 3. used function-ish name.
   
   BTW, `sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst` is a private package that doesn't generate Javadoc. So it should be safe to use `[[...]]` as we want (because using `[[..]]` in Scaladoc causes some problems sometimes when it's converted into Javadoc in some cases such as `trait`s).




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org