[GitHub] [incubator-uniffle] xianjingfeng commented on issue #503: [Bug] Frequent and long lasting full GCs when there are too many partitions on a shuffle server

["xianjingfeng (via GitHub)" <gi...@apache.org>]

xianjingfeng commented on issue #503:
URL: https://github.com/apache/incubator-uniffle/issues/503#issuecomment-1413043692

   > > > @leixm @xianjingfeng Do you encounter similar problems?
   > > 
   > > 
   > > I haven't found this situation yet. xmx=80g, buffer-capacity=40g, read-capacity=20g
   > 
   > I don't enable the single buffer flush. And I found this occurrence of full GC is always along with the huge partition appearance although I have switched GC to CMS.
   > 
   > So I guess the big memory shuffleBlock reference of huge partition is released but gc throughout is slower than receiving data speed and then gc happened.
   > 
   > Based on this guess, I will enable the single buffer flush to avoid this problem. Could you help give some advice? @advancedxy @xianjingfeng @jerqi
   
   The size of RPC queue is unlimited, is it possible relate to this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org