[GitHub] [spark] maropu commented on issue #24765: [SPARK-27915][SQL][WIP] Update logical Filter's output nullability based on IsNotNull conditions

[GitBox <gi...@apache.org>]

maropu commented on issue #24765: [SPARK-27915][SQL][WIP] Update logical Filter's output nullability based on IsNotNull conditions
URL: https://github.com/apache/spark/pull/24765#issuecomment-511653938
 
 
   Yea, thanks for revisiting this, @JoshRosen! I remember we have the two suggestions from @gatorsmile and @cloud-fan in the previous discussion; [1) nullability is just a hint for the optimizer and it might be good to add a new trait for this hint](https://github.com/apache/spark/pull/18576#issuecomment-326050659).  And, [2) the optimization for `Filter.output` is not common in use cases and it is more important to fix the same issue in `Join.output`](https://github.com/apache/spark/pull/18576#discussion_r177674483). So, I'm currently not sure that this is a right approach (I agree to fix this issue though).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org