You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/21 12:09:33 UTC

[GitHub] [spark] wankunde commented on pull request #34234: [SPARK-36967][CORE] Report accurate shuffle block size if its skewed

wankunde commented on pull request #34234:
URL: https://github.com/apache/spark/pull/34234#issuecomment-948549219


   Hi, @Ngone51 @JoeyValentine @mridulm 
   I add a parameter to limit the number of reported shuffle blocks if there are too many huge skewed blocks.
   I think this is also helpful to limit the memory usage for MapStatus object.
   Could you help me to review this PR again? 
   Many thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org