[GitHub] [incubator-uniffle] advancedxy commented on issue #503: [Bug] Frequent and long lasting full GCs when there are too many partitions on a shuffle server

[GitBox <gi...@apache.org>]

advancedxy commented on issue #503:
URL: https://github.com/apache/incubator-uniffle/issues/503#issuecomment-1396537726

   > 1. Garbage collector changes to CMS
   > 2. Expand the uniffle cluster by adding more shuffle-servers
   > 3. If one shuffle-server has partition number exceeding the threshold, we should make it fallback to ess.
   
   If possible, you could also scale-up the shuffle-server, let's say, bump the maxHeapSize to 60G.
   
   For 1, I doubt its effectiveness. But you experiment report is much appreciated.
   For 2, this could always be a simple and direct solution.
   For 3, you mean coordinator reject the app upon app startup or at app's stage level? Stage level fallback is a big change. Itself may deserve a issue and a design doc.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org