[GitHub] [spark] allisonwang-db commented on pull request #30093: [SPARK-33183][SQL] Fix EliminateSorts bug when removing global sorts

[GitBox <gi...@apache.org>]

allisonwang-db commented on pull request #30093:
URL: https://github.com/apache/spark/pull/30093#issuecomment-713147885


   I like the approach of using two simpler rules and removing the special case for the `Range` operator.
   
   @tanelk  I am not sure if I fully understand your proposed logic behind "remove local child sorts inside a local sort". Say we have this case
   ```
   Sort('a.asc, false, _)
     Repartition
        Sort('b.asc, false, _)
   ```
   With the proposed logic, the initial `global` value will be false, which will lead to the `Repartition` case to return `false`, and the `recursiveRemoveSort` will fail to eliminate the child local sort `Sort('b.asc, false, _)`
   
   Can you provide an example when your logic can eliminate local child sorts inside a local sort while the current logic cannot?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org