[GitHub] [spark] wankunde commented on pull request #41786: [WIP][SPARK-44243][CORE] Add a parameter to determine the locality of local shuffle reader

["wankunde (via GitHub)" <gi...@apache.org>]

wankunde commented on PR #41786:
URL: https://github.com/apache/spark/pull/41786#issuecomment-1618666379

   Thanks @mridulm @maryannxue for your review.
   
   If a query contains shuffle A and shuffle B, there are many PartialReducerPartitions after OptimizeSkewedJoin optimization, and shuffle B is a local read shuffle. Enable `spark.shuffle.reduceLocality.enabled` may takes some extra time to get the preferred locations.
   But there is a limit on the number of PartialReducerPartitions in our production environment, so it is okay for me not to make this change.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org