[GitHub] [incubator-uniffle] zuston commented on a diff in pull request #636: [#630] feat(client): Disable the localShuffleReader by default in Spark3 client

["zuston (via GitHub)" <gi...@apache.org>]

zuston commented on code in PR #636:
URL: https://github.com/apache/incubator-uniffle/pull/636#discussion_r1112748100


##########
docs/client_guide.md:
##########
@@ -80,6 +80,10 @@ and Continuous partition assignment mechanism.
         # Default value is 1.0, used to estimate task concurrency, how likely is this part of the resource between spark.dynamicAllocation.minExecutors and spark.dynamicAllocation.maxExecutors to be allocated
         --conf spark.rss.estimate.task.concurrency.dynamic.factor=1.0
         ```
+   
+3. In `RssShuffleManager`, the local shuffle reader has been disabled by default (`spark.sql.adaptive.localShuffleReader.enabled=false`), as it would cause too many random small IOs and too many network connections with shuffle servers. 

Review Comment:
   Nice. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org