[GitHub] mgaido91 commented on a change in pull request #23726: [SPARK-26812][SQL] Report correct nullability for complex datatypes in Union

[GitBox <gi...@apache.org>]

mgaido91 commented on a change in pull request #23726: [SPARK-26812][SQL] Report correct nullability for complex datatypes in Union
URL: https://github.com/apache/spark/pull/23726#discussion_r253279139
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala
 ##########
 @@ -230,9 +231,23 @@ case class Union(children: Seq[LogicalPlan]) extends LogicalPlan {
   }
 
   // updating nullability to make all the children consistent
-  override def output: Seq[Attribute] =
-    children.map(_.output).transpose.map(attrs =>
-      attrs.head.withNullability(attrs.exists(_.nullable)))
+  override def output: Seq[Attribute] = {
 
 Review comment:
   I'd say so, but I have not yet been able to find a use case and therefore a UT for that too. If you check the failing case of the JIRA, for instance, the current change works. Because after the analysis all the plans children of union have their attributes casted appropriately. So I am not sure that is really needed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org