You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/09/13 15:02:49 UTC
[GitHub] [spark] peter-toth opened a new pull request, #37866: [SPARK-40362][SQL][3.3] Fix BinaryComparison canonicalization
peter-toth opened a new pull request, #37866:
URL: https://github.com/apache/spark/pull/37866
### What changes were proposed in this pull request?
Change canonicalization to a one pass process and move logic from `Canonicalize.reorderCommutativeOperators` to the respective commutative operators' `canonicalize`.
### Why are the changes needed?
https://github.com/apache/spark/pull/34883 improved expression canonicalization performance but introduced regression when a commutative operator is under a `BinaryComparison`. This is because children reorder by their hashcode can't happen in `preCanonicalized` phase when children are not yet "final".
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Added new UT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth commented on pull request #37866: [SPARK-40362][SQL][3.3] Fix BinaryComparison canonicalization
Posted by GitBox <gi...@apache.org>.
peter-toth commented on PR #37866:
URL: https://github.com/apache/spark/pull/37866#issuecomment-1246302372
Thanks @cloud-fan, @dongjoon-hyun.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] peter-toth commented on pull request #37866: [SPARK-40362][SQL][3.3] Fix BinaryComparison canonicalization
Posted by GitBox <gi...@apache.org>.
peter-toth commented on PR #37866:
URL: https://github.com/apache/spark/pull/37866#issuecomment-1245693260
> The compilation is broken. Please check the code.
Thanks @dongjoon-hyun for checking the logs, https://github.com/apache/spark/pull/37866/commits/483d6b76573d18fb9cb3d4a01e3a9721b6a30ec5 should fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #37866: [SPARK-40362][SQL][3.3] Fix BinaryComparison canonicalization
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #37866: [SPARK-40362][SQL][3.3] Fix BinaryComparison canonicalization
URL: https://github.com/apache/spark/pull/37866
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org