[GitHub] [spark] mgaido91 opened a new pull request #24604: [SPARK-26812][SQL] Report correct nullability for complex datatypes i…

[GitBox <gi...@apache.org>]

mgaido91 opened a new pull request #24604: [SPARK-26812][SQL] Report correct nullability for complex datatypes i…
URL: https://github.com/apache/spark/pull/24604
 
 
   …n Union
   
   ## What changes were proposed in this pull request?
   
   When there is a `Union`, the reported output datatypes are the ones of the first plan and the nullability is updated according to all the plans. For complex types, though, the nullability of their elements is not updated using the types from the other plans. This means that the nullability of the inner elements is the one of the first plan. If this is not compatible with the one of other plans, errors can happen (as reported in the JIRA).
   
   The PR proposes to update the nullability of the inner elements of complex datatypes according to most permissive value of all the plans.
   
   ## How was this patch tested?
   
   added UT
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org