[GitHub] [spark] viirya commented on a change in pull request #24576: [SPARK-27671][SQL] Fix error when casting from a nested null in a struct

[GitBox <gi...@apache.org>]

viirya commented on a change in pull request #24576: [SPARK-27671][SQL] Fix error when casting from a nested null in a struct
URL: https://github.com/apache/spark/pull/24576#discussion_r283077851
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
 ##########
 @@ -620,6 +620,12 @@ case class Cast(child: Expression, dataType: DataType, timeZoneId: Option[String
     // We can return what the children return. Same thing should happen in the codegen path.
     if (DataType.equalsStructurally(from, to)) {
       identity
+    } else if (from == NullType) {
+      // According to `canCast`, NullType can be casted to any type.
+      // For primitive types, we don't reach here because the guard of `nullSafeEval`.
+      // But for nested types like struct, we might reach here for nested null type field.
+      // We won't call the returned function actually, but returns a placeholder.
+      _ => throw new SparkException(s"should not directly cast from NullType to $to.")
     } else {
       to match {
         case dt if dt == from => identity[Any]
 
 Review comment:
   When `from` is `NullType`, the returned function won't be called, actually, but just a placeholder.
   
   Instead of `identity`, throwing an exception sounds better, so it prevents accidentally calling it, and we don't want it.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org