[GitHub] [spark] peter-toth commented on pull request #37525: [WIP][SPARK-40086][SPARK-42049][SQL] Improve AliasAwareOutputPartitioning and AliasAwareQueryOutputOrdering to take all aliases into account

[GitBox <gi...@apache.org>]

peter-toth commented on PR #37525:
URL: https://github.com/apache/spark/pull/37525#issuecomment-1385458033

   @ulysses-you, @cloud-fan, I've rebased this PR on top of `master`, that now includes `multiTransform()`:
   - The 1st commit of this PR is a cherry-pick of the first commit from https://github.com/apache/spark/pull/39556.
   - The 2nd commit is from the original content of this PR and switches to `multiTransform()`.
   - 3rd is the aliasMap split as distussed here: https://github.com/apache/spark/pull/37525#discussion_r1070970038 and required as `autoComtinue` feature was removed.
   - 4th adds early pruning, but this is a bit different to what we have in https://github.com/apache/spark/pull/39556. It is actually more efficient and we don't need "after transformation pruning".
   - 5th contains test fixes in `PlannerSuite` as some of the previous expected results seems to be wrong. 
   
   Let me know your thougths.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org